neutron-dynamic-routing bgp ryu hold timer expired but never tried to recover

[Tobias Urdin]

Lost connection to the peer and the hold timer expired but it never tried to recover.

2019-05-09 13:26:24.921 2461284 INFO bgpspeaker.speaker [-] Negotiated hold time 40 expired.
2019-05-09 13:26:24.922 2461284 INFO bgpspeaker.peer [-] Connection to peer <censored peer ip> lost, reason: failed to write to socket Resetting retry connect loop: False
2019-05-09 13:26:24.922 2461284 ERROR ryu.lib.hub [-] hub: uncaught exception: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ryu/lib/hub.py", line 80, in _launch
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/utils/evtlet.py", line 63, in __call__
    self._funct(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/speaker.py", line 542, in _expired
    self.send_notification(code, subcode)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/speaker.py", line 374, in send_notification
    self._send_with_lock(notification)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/speaker.py", line 386, in _send_with_lock
    self.connection_lost('failed to write to socket')
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/speaker.py", line 596, in connection_lost
    self._peer.connection_lost(reason)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/peer.py", line 2323, in connection_lost
    self._protocol.stop()
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/speaker.py", line 405, in stop
    Activity.stop(self)
  File "/usr/lib/python2.7/site-packages/ryu/services/protocols/bgp/base.py", line 314, in stop
    raise ActivityException(desc='Cannot call stop when activity is '
ActivityException: 100.1 - Cannot call stop when activity is not started or has been stopped already.
: ActivityException: 100.1 - Cannot call stop when activity is not started or has been stopped already.

[Lajos Katona (lajos-katona)]

Could you provide some environment details (like the branches you used), and reproduction steps?

[Lajos Katona (lajos-katona)]

If it is ryu I suppose neutron is older than stein as from stein instead of ryu os-ken is used, am I right?

[Tobias Urdin (tobias-urdin)]

Ah sorry, it's a Rocky cloud so yeah it's still Ryu.

Just want to make sure there isn't a bug when it should retry, because it didn't get back once I fixed the network issue.

openstack-neutron-ml2-13.0.3-1.el7.noarch
openstack-neutron-bgp-dragent-13.0.1-1.el7.noarch
python2-neutronclient-6.9.0-1.el7.noarch
python-neutron-13.0.3-1.el7.noarch
openstack-neutron-13.0.3-1.el7.noarch
openstack-neutron-openvswitch-13.0.3-1.el7.noarch
openstack-neutron-dynamic-routing-common-13.0.1-1.el7.noarch
python2-neutron-lib-1.18.0-1.el7.noarch
openstack-neutron-common-13.0.3-1.el7.noarch
python2-neutron-dynamic-routing-13.0.1-1.el7.noarch

python-ryu-common-4.26-1.el7.noarch
python2-ryu-4.26-1.el7.noarch

[Ryan Tidwell (ryan-tidwell)]

This may have been fixed in os-ken on stein https://review.opendev.org/#/c/648993/

[Tobias Urdin (tobias-urdin)]

Thanks Ryan, I'll mark it as invalid and wait until we are on Stein.