BGP peer is not working

Bug #1898634 reported by Vinicius Coelho
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Unassigned

Bug Description

I´m trying to configure dynamic routing, but when I associate provider network with the bgp speaker I start to receive these errors:

2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/api/rpc/handlers/bgp_speaker_rpc.py", line 65, in get_bgp_speakers
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server return self.plugin.get_bgp_speakers_for_agent_host(context, host)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_dragentscheduler_db.py", line 263, in get_bgp_speakers_for_agent_host
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server context, binding['bgp_speaker_id'])
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 165, in get_bgp_speaker_with_advertised_routes
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server bgp_speaker_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 479, in get_routes_by_bgp_speaker_id
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server bgp_speaker_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/site-packages/neutron_dynamic_routing/db/bgp_db.py", line 673, in _get_central_fip_host_routes_by_bgp_speaker
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server l3_db.Router.id == router_attrs.router_id)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2259, in outerjoin
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server from_joinpoint=from_joinpoint,
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _join
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/base.py", line 220, in generate
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server fn(self, *args[1:], **kw)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2414, in _join
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server left, right, onclause, prop, create_aliases, outerjoin, full
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2437, in _join_left_to_right
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server ) = self._join_determine_implicit_left_side(left, right, onclause)
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2526, in _join_determine_implicit_left_side
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server "Can't determine which FROM clause to join "
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.
2020-10-05 16:56:13.028 2304845 ERROR oslo_messaging.rpc.server

I made manual installation, ussuri. Couldn´t find any workaround.

Revision history for this message
Dr. Jens Harbott (j-harbott) wrote :

This looks weird, what DB are you using and which OS version? Distro pkgs or installation from source?

Changed in neutron:
status: New → Incomplete
tags: added: l3-bgp
Revision history for this message
Vinicius Coelho (viniciuscoelho21) wrote :

Centos 8, mariadb. I followed the default guide for a minimal deploy:
https://docs.openstack.org/install-guide/openstack-services.html#minimal-deployment-for-ussuri

Revision history for this message
Nate Johnston (nate-johnston) wrote :

This looks like it might be related to https://bugs.launchpad.net/neutron/+bug/1819260 - will look more tomorrow.

Revision history for this message
Bernard Cafarelli (bcafarel) wrote :

I had a similar error when trying neutron-dynamic-routing hacked on top of a Red Hat OSP train deployment: recent SQLAlchemy and mariadb behind.
The environment was recycled before I thought to copy the backtraces, but it looked quite similar to this one

Revision history for this message
Vinicius Coelho (viniciuscoelho21) wrote :

Hi Nate,
 Just checked my l3_db.py and it is already patched.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-dynamic-routing (master)

Fix proposed to branch: master
Review: https://review.opendev.org/756613

Changed in neutron:
assignee: nobody → Nate Johnston (nate-johnston)
status: Incomplete → In Progress
Changed in neutron:
importance: Undecided → Medium
Changed in neutron:
assignee: Nate Johnston (nate-johnston) → Dr. Jens Harbott (j-harbott)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron-dynamic-routing (master)

Reviewed: https://review.opendev.org/756613
Committed: https://git.openstack.org/cgit/openstack/neutron-dynamic-routing/commit/?id=4212b705386153c5ff9986068c78ee9c7b4fbe4c
Submitter: Zuul
Branch: master

commit 4212b705386153c5ff9986068c78ee9c7b4fbe4c
Author: Nate Johnston <email address hidden>
Date: Wed Oct 7 21:56:49 2020 -0400

    Fix outerjoin to comply with SQLAlchemy 1.3 strictness

    The outerjoin in neutron_dynamic_routing/db/bgp_db.py does not meet
    with the stricter new standards in SQLAlchemy 1.3, so this finds a
    more compliant form.

    [1] https://docs.sqlalchemy.org/en/13/changelog/migration_13.html#query-join-handles-ambiguity-in-deciding-the-left-side-more-explicitly

    Change-Id: I603a2febb1755b6db17e6ac9cb22015a1f3336d4
    Closes-Bug: #1898634

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
Vinicius Coelho (viniciuscoelho21) wrote :

Hi Hudson,
 I tried to make the change proposed and got the same error.

Revision history for this message
Mike Bayer (zzzeek) wrote :

Hi Vinicius -

I've isolated the error condition you are experiencing down to a SQLAlchemy issue in early 1.3.x versions that was fixed in April of 2019, and it appears you are on version 1.3.2 or lower. Please upgrade to the latest SQLAlchemy 1.3 version which is currently 1.3.19, or at the very least version 1.3.3, that will solve the issue here.

Revision history for this message
Mike Bayer (zzzeek) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-dynamic-routing (master)

Fix proposed to branch: master
Review: https://review.opendev.org/757178

Revision history for this message
Vinicius Coelho (viniciuscoelho21) wrote :

Hi Mike!
 You are completely right. My version was 1.3.2 (the last avaiable for centos 8). I installed the last version using git, and now everything is working.

 Thanks for the help!!

Revision history for this message
Tobias Urdin (tobias-urdin) wrote :

This will be solved when CentOS 8.3 is released which ships a later SQLAlchemy version fwiw.

Changed in neutron:
assignee: Dr. Jens Harbott (j-harbott) → nobody
Changed in neutron:
status: Fix Released → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron-dynamic-routing 18.0.0.0rc1

This issue was fixed in the openstack/neutron-dynamic-routing 18.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.