neutron-bgp-dragent is missing ryu

[Jan Horstmann]

Upgrading from rocky to stein I noticed that neutron-bgp-dragent kept restarting. The log ([1]) showed that ryu could not be imported.
Comparing rocky and stein containers shows that ryu was required by neutron in version 4.26 ([2]), but not in 4.32 ([3]). Thus ryu never gets pulled in as a dependency.

[1]
2019-08-06 17:29:03.699 6 INFO neutron.common.config [-] Logging enabled!
2019-08-06 17:29:03.699 6 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-bgp-dragent version 14.0.2
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent [-] Error while importing BGP speaker driver neutron_dynamic_routing.services.bgp.agent.driver.ryu.driver.RyuBgpDriver: ImportError: No module named ryu.driver
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent Traceback (most recent call last):
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py", line 73, in initialize_driver
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent self.conf))
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/importutils.py", line 44, in import_object
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent return import_class(import_str)(*args, **kwargs)
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/importutils.py", line 30, in import_class
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent __import__(mod_str)
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent ImportError: No module named ryu.driver
2019-08-06 17:29:03.993 6 ERROR neutron_dynamic_routing.services.bgp.agent.bgp_dragent

[2]
()[neutron@c65641c18af0 /]$ pip show ryu
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Name: ryu
Version: 4.26
Summary: Component-based Software-defined Networking Framework
Home-page: http://osrg.github.io/ryu/
Author: Ryu project team
Author-email: <email address hidden>
License: Apache License 2.0
Location: /var/lib/kolla/venv/lib/python2.7/site-packages
Requires: netaddr, ovs, oslo.config, msgpack, eventlet, six, routes, webob, tinyrpc
Required-by: neutron

[3]
()[neutron@e9077d4f4f93 /]$ pip install --user neutron ryu
[...]
()[neutron@e9077d4f4f93 /]$ pip show ryu
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Name: ryu
Version: 4.32
Summary: Component-based Software-defined Networking Framework
Home-page: http://osrg.github.io/ryu/
Author: Ryu project team
Author-email: <email address hidden>
License: Apache License 2.0
Location: /var/lib/neutron/.local/lib/python2.7/site-packages
Requires: netaddr, ovs, msgpack, eventlet, six, routes, oslo.config, webob, tinyrpc
Required-by:

[Mark Goddard (mgoddard)]

I think the problem here is that neutron-dynamic-routing switched from using ryu to os-ken (a fork of ryu). The config option bgp_speaker_driver is set in kolla-ansible to ryu, but it is not included in requirements.txt. We should just use the default of os-ken.

https://docs.openstack.org/neutron-dynamic-routing/latest/contributor/dragent-drivers.html#bgp-driver

[Mark Goddard (mgoddard)]

I proposed a potential fix here: https://review.opendev.org/#/c/675111

Could you test it out and report back here or in the review?

[Jan Horstmann (janhorstmann)]

Without much debugging it seems that there is no default set for "bgp_speaker_driver", because applying the patch yields the error shown in [1].
However setting

"bgp_speaker_driver = neutron_dynamic_routing.services.bgp.agent.driver.os_ken.driver.OsKenBgpDriver"

explicitly in "bgp_dragent.ini" seems to resolve the issue ([2])

[1]
2019-08-07 15:24:06.302 6 INFO neutron.common.config [-] Logging enabled!
2019-08-07 15:24:06.303 6 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-bgp-dragent version 14.0.2
2019-08-07 15:24:06.605 6 CRITICAL neutron [-] Unhandled error: AttributeError: 'NoneType' object has no attribute 'rpartition'
2019-08-07 15:24:06.605 6 ERROR neutron Traceback (most recent call last):
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/bin/neutron-bgp-dragent", line 10, in <module>
2019-08-07 15:24:06.605 6 ERROR neutron sys.exit(main())
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/cmd/eventlet/agents/bgp_dragent.py", line 20, in main
2019-08-07 15:24:06.605 6 ERROR neutron bgp_dragent.main()
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/services/bgp/agent/entry.py", line 45, in main
2019-08-07 15:24:06.605 6 ERROR neutron manager='neutron_dynamic_routing.services.bgp.agent.bgp_dragent.'
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/service.py", line 414, in create
2019-08-07 15:24:06.605 6 ERROR neutron periodic_fuzzy_delay=periodic_fuzzy_delay)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/service.py", line 346, in __init__
2019-08-07 15:24:06.605 6 ERROR neutron self.manager = manager_class(host=host, *args, **kwargs)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py", line 664, in __init__
2019-08-07 15:24:06.605 6 ERROR neutron self).__init__(host, conf)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py", line 59, in __init__
2019-08-07 15:24:06.605 6 ERROR neutron self.initialize_driver(conf)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 160, in wrapper
2019-08-07 15:24:06.605 6 ERROR neutron result = f(*args, **kwargs)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py", line 73, in initialize_driver
2019-08-07 15:24:06.605 6 ERROR neutron self.conf))
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/importutils.py", line 44, in import_object
2019-08-07 15:24:06.605 6 ERROR neutron return import_class(import_str)(*args, **kwargs)
2019-08-07 15:24:06.605 6 ERROR neutron File "/var/lib/kolla/ve...

[Mark Goddard (mgoddard)]

Well that's annoying - the docs state that the default is os-ken. Thanks for testing though. I'll update the patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675111
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=a13bc5504278346154fa5205683cf2bc64977b14
Submitter: Zuul
Branch: master

commit a13bc5504278346154fa5205683cf2bc64977b14
Author: Mark Goddard <email address hidden>
Date: Wed Aug 7 13:50:21 2019 +0100

    Use default BGP speaker driver in bgp_dragent.ini

    In the Stein release, neutron-dynamic-routing switched from
    using ryu to os-ken (a fork of ryu). The config option
    bgp_speaker_driver [1] is currently set in kolla-ansible to use the ryu
    driver, but this is not included in the image due to it being removed
    from neutron-dynamic-routing's requirements.txt.

    This change uses the os-ken driver for bgp_speaker_driver, since this
    package is included in the image.

    [1] https://docs.openstack.org/neutron-dynamic-routing/latest/contributor/dragent-drivers.html#bgp-driver

    Change-Id: I76ad84c6b963818208f080ebeb054b2553ff40af
    Closes-Bug: #1839165

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/677146

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/677146
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=e95b583b10377c0ae5c43f11614f48aed5df2395
Submitter: Zuul
Branch: stable/stein

commit e95b583b10377c0ae5c43f11614f48aed5df2395
Author: Mark Goddard <email address hidden>
Date: Wed Aug 7 13:50:21 2019 +0100

    Use default BGP speaker driver in bgp_dragent.ini

    In the Stein release, neutron-dynamic-routing switched from
    using ryu to os-ken (a fork of ryu). The config option
    bgp_speaker_driver [1] is currently set in kolla-ansible to use the ryu
    driver, but this is not included in the image due to it being removed
    from neutron-dynamic-routing's requirements.txt.

    This change uses the os-ken driver for bgp_speaker_driver, since this
    package is included in the image.

    [1] https://docs.openstack.org/neutron-dynamic-routing/latest/contributor/dragent-drivers.html#bgp-driver

    Change-Id: I76ad84c6b963818208f080ebeb054b2553ff40af
    Closes-Bug: #1839165
    (cherry picked from commit a13bc5504278346154fa5205683cf2bc64977b14)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 8.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc1 release candidate.