BGP Dynamic Routing: Not listening on 179 port in router namespace for V6

[vismys]

Trying to configure BGP on Mitaka/Ubuntu setup.

#uname -a
Linux hp3bl2 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

BGP neighborship is not getting established. Neutron router sents RST,ACK on session establishment request from Cisco Router.

Snapshot of /var/log/neutron/neutron-bgp-dragent.log

============================================================
_connect_loop /usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py:1078
2016-09-26 17:29:15.102 2010 INFO bgpspeaker.peer [-] Will try to reconnect to fd00:10:6b50:600a::2 after 30 secs: True
2016-09-26 17:29:22.055 2010 DEBUG neutron.services.bgp.agent.bgp_dragent [req-7b042acb-731e-430c-bfd5-b67359980c69 - - - - -] Report state task started _report_state /usr/lib/python2.7/dist-packages/neutron/services/bgp/agent/bgp_dragent.py:675
2016-09-26 17:29:22.058 2010 DEBUG oslo_messaging._drivers.amqpdriver [req-326e6789-1dd1-4b89-9ab6-82395d96e407 - - - - -] CALL msg_id: 89ef26385d34408cbde5deb5725d1bb2 exchange 'neutron' topic 'q-plugin' _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:454
2016-09-26 17:29:22.082 2010 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 89ef26385d34408cbde5deb5725d1bb2 __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:302
2016-09-26 17:29:33.945 2010 DEBUG oslo_service.periodic_task [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] Running periodic task BgpDrAgentWithStateReport.periodic_resync run_periodic_tasks /usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215
2016-09-26 17:29:33.946 2010 DEBUG neutron.services.bgp.agent.bgp_dragent [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] Started periodic resync. periodic_resync /usr/lib/python2.7/dist-packages/neutron/services/bgp/agent/bgp_dragent.py:173
2016-09-26 17:29:33.947 2010 DEBUG neutron.services.bgp.agent.bgp_dragent [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] resync (b9ed49f6-7a32-4049-a74b-9791dcfaea32): ['Periodic route cache refresh'] _periodic_resync_helper /usr/lib/python2.7/dist-packages/neutron/services/bgp/agent/bgp_dragent.py:165
2016-09-26 17:29:33.948 2010 DEBUG oslo_concurrency.lockutils [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] Lock "bgp-dragent" acquired by "neutron.services.bgp.agent.bgp_dragent.sync_state" :: waited 0.000s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:273
2016-09-26 17:29:33.950 2010 DEBUG oslo_messaging._drivers.amqpdriver [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] CALL msg_id: e9eaf874365f453fb9cf76dce327b53b exchange 'neutron' topic 'q-bgp-plugin' _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:454
2016-09-26 17:29:34.009 2010 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: e9eaf874365f453fb9cf76dce327b53b __call__ /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:302
2016-09-26 17:29:34.011 2010 DEBUG neutron.services.bgp.agent.bgp_dragent [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] Recording resync request for BGP Speaker b9ed49f6-7a32-4049-a74b-9791dcfaea32 with reason=Periodic route cache refresh schedule_resync /usr/lib/python2.7/dist-packages/neutron/services/bgp/agent/bgp_dragent.py:509
2016-09-26 17:29:34.012 2010 DEBUG oslo_concurrency.lockutils [req-089bb213-a2c5-40c8-9e1f-9282e80e4482 - - - - -] Lock "bgp-dragent" released by "neutron.services.bgp.agent.bgp_dragent.sync_state" :: held 0.064s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:285
2016-09-26 17:29:46.103 2010 DEBUG bgpspeaker.peer [-] Peer fd00:10:6b50:600a::2 BGP FSM went from Active to Connect bgp_state /usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py:222
2016-09-26 17:29:46.103 2010 DEBUG bgpspeaker.peer [-] Peer(ip: fd00:10:6b50:600a::2, asn: 65002) trying to connect to ('fd00:10:6b50:600a::2', 179) _connect_loop /usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py:1064
2016-09-26 17:29:46.104 2010 DEBUG bgpspeaker.base [-] Connect TCP called for fd00:10:6b50:600a::2:179 _connect_tcp /usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/base.py:399
2016-09-26 17:29:46.105 2010 DEBUG bgpspeaker.peer [-] Peer fd00:10:6b50:600a::2 BGP FSM went from Connect to Active bgp_state /usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py:222
2016-09-26 17:29:46.106 2010 DEBUG bgpspeaker.peer [-] Socket could not be created in time (30 secs), reason Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py", line 1072, in _connect_loop
    password=password)
  File "/usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/base.py", line 410, in _connect_tcp
    sock.connect(peer_addr)
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 237, in connect
    while not socket_connect(fd, address):
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 39, in socket_connect
    raise socket.error(err, errno.errorcode[err])
error: [Errno 101] ENETUNREACH
============================================

Here is the netstat output.

root@hp3bl2:/etc/neutron# ip netns exec qrouter-ee1287c3-f3d2-498f-82f3-ba7643d1e074 netstat -A inet6 -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
raw6 0 0 :::58 :::* 7

Please let me know if whether more information required.
Thank you,
Vishwanath Shivappa

[Steve Ruan (ruansx)]

Neutron-dynamic-routing run bgp in network node, not in the router namespace.
Could you show about the "bgp-speaker-show" and "bgp-peer-show"?

[vismys (vismys)]

root@hp3bl2:~# neutron bgp-speaker-show bgp1
+-----------------------------------+--------------------------------------+
| Field | Value |
+-----------------------------------+--------------------------------------+
| advertise_floating_ip_host_routes | False |
| advertise_tenant_networks | True |
| id | b9ed49f6-7a32-4049-a74b-9791dcfaea32 |
| ip_version | 6 |
| local_as | 65002 |
| name | bgp1 |
| networks | 220fadaf-47c5-40b0-bc7c-e3c7c5bb00f9 |
| peers | |
| tenant_id | ab18280ba1c0454f95474f31660efaec |
+-----------------------------------+--------------------------------------+

root@hp3bl2:~# neutron bgp-peer-show bgp-peer1
+-----------+--------------------------------------+
| Field | Value |
+-----------+--------------------------------------+
| auth_type | none |
| id | 51afd3f0-53d0-4ee6-b40a-1b54913829bf |
| name | bgp-peer1 |
| peer_ip | FD00:10:6B50:600A::2 |
| remote_as | 65002 |
| tenant_id | ab18280ba1c0454f95474f31660efaec |
+-----------+--------------------------------------+

[vismys (vismys)]

I don't see port 179 listening on network node.

~# netstat -an | grep 179
tcp 0 0 10.54.212.11:9696 10.54.212.11:51796 ESTABLISHED
tcp 0 0 10.54.212.11:51796 10.54.212.11:9696 ESTABLISHED

netstat -A inet6 -an | grep 179

[vikram.choudhary (vikschw)]

Please check with Ryu team..

bgpspeaker.peer [-] Socket could not be created in time (30 secs), reason Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/peer.py", line 1072, in _connect_loop
    password=password)
  File "/usr/lib/python2.7/dist-packages/ryu/services/protocols/bgp/base.py", line 410, in _connect_tcp
    sock.connect(peer_addr)
  File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 237, in connect

[Ryan Tidwell (ryan-tidwell)]

Are you expecting the Cisco router to initiate the session? To avoid having to run the agent with sudo privileges, we aren't opening a port for accepting connections on the neutron end. If you're expecting you peers to initiate the peering session that could be getting you into trouble. Peers should be set in passive mode, essentially forcing them to wait for neutron to initiate peering. Neutron will initiate the peering session with peers. This seems a little odd, but we didn't want to initially force operators to run the BGP agent with sudo privileges. For Mitaka, we didn't want to deal with the implications of that. To keep it simple, we simply didn't allow Ryu to listen on port 179.

If you put your peers in passive mode and allow neutron to initiate the session, does that solve your problem? We can certainly come back to the issue of listening on port 179 and sudo privileges now that BGP is fairly stable. That may make life easier for users.

If neutron is unable to peer at all, we need to dig in to that. A little more clarification on this point would help.

[vismys (vismys)]

we try that. but not seeing BGP syn request from neutron side. attaching router config and tshark capture.

router bgp 65002
 bgp log-neighbor-changes
 neighbor 2001:DB8:4321:42::3 remote-as 65002
 neighbor 2001:DB8:4321:42::3 transport connection-mode passive
 !
 address-family ipv6
  neighbor 2001:DB8:4321:42::3 activate
  network FD00:10:6B50::/48
 exit-address-family

interface GigabitEthernet7/25
 no ip address
 ipv6 address 2001:DB8:4321:42::10/64
 ipv6 enable
end

interface GigabitEthernet7/25
 no ip address
 ipv6 address 2001:DB8:4321:42::10/64
 ipv6 enable
end

 10 5.868807 2001:db8:4321:42::10 -> 2001:db8:4321:42::3 ICMPv6 114 Echo (ping) request id=0x20c0, seq=4, hop limit=0
 11 5.868858 2001:db8:4321:42::3 -> 2001:db8:4321:42::10 ICMPv6 114 Echo (ping) reply id=0x20c0, seq=4, hop limit=0 (request in 10)
11 12 10.878270 fe80::f816:3eff:fe3e:2f06 -> 2001:db8:4321:42::10 ICMPv6 86 Neighbor Solicitation for 2001:db8:4321:42::10 from fa:16:3e:3e:2f:06
 13 10.878819 2001:db8:4321:42::10 -> fe80::f816:3eff:fe3e:2f06 ICMPv6 78 Neighbor Advertisement 2001:db8:4321:42::10 (rtr, sol)
13 14 15.875861 fe80::225:b4ff:fe46:7700 -> fe80::f816:3eff:fe3e:2f06 ICMPv6 86 Neighbor Solicitation for fe80::f816:3eff:fe3e:2f06 from 00:25:b4:46:77:00
 15 15.875976 fe80::f816:3eff:fe3e:2f06 -> fe80::225:b4ff:fe46:7700 ICMPv6 78 Neighbor Advertisement fe80::f816:3eff:fe3e:2f06 (rtr, sol)
15 16 20.878277 fe80::f816:3eff:fe3e:2f06 -> fe80::225:b4ff:fe46:7700 ICMPv6 86 Neighbor Solicitation for fe80::225:b4ff:fe46:7700 from fa:16:3e:3e:2f:06
 17 20.878684 fe80::225:b4ff:fe46:7700 -> fe80::f816:3eff:fe3e:2f06 ICMPv6 78 Neighbor Advertisement fe80::225:b4ff:fe46:7700 (rtr, sol)
17 18 24.999089 HewlettP_6b:ed:26 -> LLDP_Multicast LLDP 338 Chassis Id = 78:48:59:6b:ed:17 Port Id = GigabitEthernet1/0/2 TTL = 120
18 19 40.684792 Cisco_e3:41:08 -> CDP/VTP/DTP/PAgP/UDLD CDP 475 Device ID: ROW-10-Cisco-6509 Port ID: GigabitEthernet7/25

[fumihiko kakuma (kakuma)]

I checked a peer between ryu bgp app and quagga on docker without using openstack.
It seems to work fine.
The following is quagga config file and tcpdump.

A log for dr-agent may be helpful for openstack.

- bgpd.conf for quagga

! -*- bgp -*-
!
! BGPd sample configuratin file
!
! $Id: bgpd.conf.sample,v 1.1 2002/12/13 20:15:29 paul Exp $
!
hostname qg1
password zebra
!enable password please-set-at-here
!
!bgp mulitple-instance
!
router bgp 64522
  bgp router-id 192.168.0.2
  neighbor 2001:10::10 remote-as 64512
  neighbor 2001:10::10 passive
  address-family ipv6
    network 2001:db8:2::/48
    neighbor 2001:10::10 activate
  exit-address-family
!
log file /var/log/quagga/bgpd.log
!
debug bgp fsm
debug bgp updates
debug bgp events

- tcpdump

  1 0.000000 2001:10::20 -> 2001:10::10 ICMPv6 118 Echo (ping) request id=0x0b5a, seq=1, hop limit=0
  2 0.000029 2001:10::10 -> 2001:10::20 ICMPv6 118 Echo (ping) reply id=0x0b5a, seq=1, hop limit=0 (request in 1)
  3 0.999422 2001:10::20 -> 2001:10::10 ICMPv6 118 Echo (ping) request id=0x0b5a, seq=2, hop limit=0
  4 0.999459 2001:10::10 -> 2001:10::20 ICMPv6 118 Echo (ping) reply id=0x0b5a, seq=2, hop limit=0 (request in 3)
  5 1.999430 2001:10::20 -> 2001:10::10 ICMPv6 118 Echo (ping) request id=0x0b5a, seq=3, hop limit=0
  6 1.999468 2001:10::10 -> 2001:10::20 ICMPv6 118 Echo (ping) reply id=0x0b5a, seq=3, hop limit=0 (request in 5)
  7 5.003402 fe80::42:c0ff:fea8:320a -> 2001:10::20 ICMPv6 86 Neighbor Solicitation for 2001:10::20 from 02:42:c0:a8:32:0a
  8 5.003426 2001:10::20 -> fe80::42:c0ff:fea8:320a ICMPv6 78 Neighbor Advertisement 2001:10::20 (sol)
  9 10.011429 fe80::42:c0ff:fea8:3214 -> fe80::42:c0ff:fea8:320a ICMPv6 86 Neighbor Solicitation for fe80::42:c0ff:fea8:320a from 02:42:c0:a8:32:14
 10 10.011510 fe80::42:c0ff:fea8:320a -> fe80::42:c0ff:fea8:3214 ICMPv6 78 Neighbor Advertisement fe80::42:c0ff:fea8:320a (sol)
 11 12.120183 2001:10::10 -> 2001:10::20 TCP 94 43212 > 179 [SYN] Seq=0 Win=28800 Len=0 MSS=1440 SACK_PERM=1 TSval=89659355 TSecr=0 WS=128
 12 12.120205 2001:10::20 -> 2001:10::10 TCP 94 179 > 43212 [SYN, ACK] Seq=0 Ack=1 Win=28560 Len=0 MSS=1440 SACK_PERM=1 TSval=89659355 TSecr=89659355 WS=128
 13 12.120220 2001:10::10 -> 2001:10::20 TCP 86 43212 > 179 [ACK] Seq=1 Ack=1 Win=28800 Len=0 TSval=89659355 TSecr=89659355
 14 12.120325 2001:10::20 -> 2001:10::10 TCP 86 179 > 43212 [FIN, ACK] Seq=1 Ack=1 Win=28672 Len=0 TSval=89659355 TSecr=89659355
 15 12.123350 2001:10::10 -> 2001:10::20 TCP 86 43212 > 179 [ACK] Seq=1 Ack=2 Win=28800 Len=0 TSval=89659356 TSecr=89659355
 16 12.123841 2001:10::10 -> 2001:10::20 BGP 135 OPEN Message
 17 12.123868 2001:10::20 -> 2001:10::10 TCP 74 179 > 43212 [RST] Seq=2 Win=0 Len=0
 18 15.019464 fe80::42:c0ff:fea8:320a -> fe80::42:c0ff:fea8:3214 ICMPv6 86 Neighbor Solicitation for fe80::42:c0ff:fea8:3214 from 02:42:c0:a8:32:0a
 19 15.019521 fe80::42:c0ff:fea8:3214 -> fe80::42:c0ff:fea8:320a ICMPv6 78 Neighbor Advertisement fe80::42:c0ff:fea8:3214 (sol)
 20 43.122908 2001:10::10 -> 2001:10::20 TCP 94 43215 > 179 [SYN] Seq=0 Win=28800 Len=0 MSS=1440 SACK_PERM=1 TSval=89667105 TSecr=0 WS=128
 21 ...