BGP peer is not working

[Vinicius Coelho]

I´m trying to configure dynamic routing, but when I associate provider network with the bgp speaker I start to receive these errors:

2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/api/rpc/handlers/bgp_speaker_rpc.py", line 65, in get_bgp_speakers
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server return self.plugin.get_bgp_speakers_for_agent_host(context, host)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_dragentscheduler_db.py", line 263, in get_bgp_speakers_for_agent_host
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server context, binding['bgp_speaker_id'])
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 165, in get_bgp_speaker_with_advertised_routes
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server bgp_speaker_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 479, in get_routes_by_bgp_speaker_id
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server bgp_speaker_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 673, in _get_central_fip_host_routes_by_bgp_speaker
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server l3_db.Router.id == router_attrs.router_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2259, in outerjoin
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server from_joinpoint=from_joinpoint,
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _join
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/base.py", line 220, in generate
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server fn(self, *args[1:], **kw)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2414, in _join
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server left, right, onclause, prop, create_aliases, outerjoin, full
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2437, in _join_left_to_right
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server ) = self._join_determine_implicit_left_side(left, right, onclause)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2526, in _join_determine_implicit_left_side
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server "Can't determine which FROM clause to join "
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server

I made manual installation, ussuri. Couldn´t find any workaround.

[Dr. Jens Harbott (j-harbott)]

This looks weird, what DB are you using and which OS version? Distro pkgs or installation from source?

[Vinicius Coelho (viniciuscoelho21)]

Centos 8, mariadb. I followed the default guide for a minimal deploy:
https://docs.openstack.org/install-guide/openstack-services.html#minimal-deployment-for-ussuri

[Nate Johnston (nate-johnston)]

This looks like it might be related to https://bugs.launchpad.net/neutron/+bug/1819260 - will look more tomorrow.

[Bernard Cafarelli (bcafarel)]

I had a similar error when trying neutron-dynamic-routing hacked on top of a Red Hat OSP train deployment: recent SQLAlchemy and mariadb behind.
The environment was recycled before I thought to copy the backtraces, but it looked quite similar to this one

[Vinicius Coelho (viniciuscoelho21)]

Hi Nate,
 Just checked my l3_db.py and it is already patched.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/756613

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/756613
Committed: https://git.openstack.org/cgit/openstack/neutron-dynamic-routing/commit/?id=4212b705386153c5ff9986068c78ee9c7b4fbe4c
Submitter: Zuul
Branch: master

commit 4212b705386153c5ff9986068c78ee9c7b4fbe4c
Author: Nate Johnston <email address hidden>
Date: Wed Oct 7 21:56:49 2020 -0400

    Fix outerjoin to comply with SQLAlchemy 1.3 strictness

    The outerjoin in neutron_dynamic_routing/db/bgp_db.py does not meet
    with the stricter new standards in SQLAlchemy 1.3, so this finds a
    more compliant form.

    [1] https://docs.sqlalchemy.org/en/13/changelog/migration_13.html#query-join-handles-ambiguity-in-deciding-the-left-side-more-explicitly

    Change-Id: I603a2febb1755b6db17e6ac9cb22015a1f3336d4
    Closes-Bug: #1898634

[Vinicius Coelho (viniciuscoelho21)]

Hi Hudson,
 I tried to make the change proposed and got the same error.

[Mike Bayer (zzzeek)]

Hi Vinicius -

I've isolated the error condition you are experiencing down to a SQLAlchemy issue in early 1.3.x versions that was fixed in April of 2019, and it appears you are on version 1.3.2 or lower. Please upgrade to the latest SQLAlchemy 1.3 version which is currently 1.3.19, or at the very least version 1.3.3, that will solve the issue here.

[Mike Bayer (zzzeek)]

the fix is https://docs.sqlalchemy.org/en/13/changelog/changelog_13.html#change-0ef2df514d4d40ea01d9b62a9c5fdae1 which was SQLAlchemy issue https://github.com/sqlalchemy/sqlalchemy/issues/4584 and in fact was raised regarding this exact code.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/757178

[Vinicius Coelho (viniciuscoelho21)]

Hi Mike!
 You are completely right. My version was 1.3.2 (the last avaiable for centos 8). I installed the last version using git, and now everything is working.

 Thanks for the help!!

[Tobias Urdin (tobias-urdin)]

This will be solved when CentOS 8.3 is released which ships a later SQLAlchemy version fwiw.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-dynamic-routing 18.0.0.0rc1 release candidate.

[Dr. Jens Harbott (j-harbott)]

Seems this should be resolved by now.