neutron

Adding floating IP fails with SQLA 1.3.0

Bug #1819260 reported by Thomas Goirand
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Undecided
LIU Yulong

Bug Description

Hi,

Trying to evaluate if we can upgrade Buster to SQLAlchemy 1.3.0, doing this in my PoC:

openstack server add floating ip demo-server 192.168.105.101

leads to this stack dump below. Obviously, there's something wrong that needs fixing. Best would be before Stein is out.

2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource [req-2ca7dd4c-515f-4958-964c-8506811c0b5a a498c39ddde54be4aafa7b3ded5563e6 9e0e0a4c736a4687ade8c5e765353bd7 - default default] update failed: No details.: sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource Traceback (most recent call last):
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/api/v2/resource.py", line 98, in resource
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource result = method(request=request, **args)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/api/v2/base.py", line 626, in update
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return self._update(request, id, body, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 140, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 136, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 154, in wrapper
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 142, in wrapper
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 183, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 179, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/api/v2/base.py", line 682, in _update
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource obj = obj_updater(request.context, id, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/api.py", line 123, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return method(*args, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 140, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 136, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 154, in wrapper
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 142, in wrapper
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*args, **kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 183, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self.force_reraise()
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource raise value
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 179, in wrapped
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_dvr_db.py", line 1113, in update_floatingip
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource context, id, floatingip)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1416, in _update_floatingip
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self._core_plugin.get_port(context.elevated(), fip_port_id))
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1278, in _update_fip_assoc
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource self._check_and_get_fip_assoc(context, fip, floatingip_obj))
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1242, in _check_and_get_fip_assoc
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource floatingip_obj)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1229, in _get_assoc_data
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource internal_subnet_id, floatingip_obj.floating_network_id)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1114, in _get_router_for_floatingip
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource internal_port, subnet, external_network_id)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/neutron/db/l3_db.py", line 1140, in get_router_for_floatingip
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource models_v2.Port, models_v2.IPAllocation).filter(
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 2234, in join
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource from_joinpoint=from_joinpoint,
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "<string>", line 2, in _join
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/sqlalchemy/orm/base.py", line 220, in generate
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource fn(self, *args[1:], **kw)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 2413, in _join
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource left, right, onclause, prop, create_aliases, outerjoin, full
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 2436, in _join_left_to_right
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource ) = self._join_determine_implicit_left_side(left, right, onclause)
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 2567, in _join_determine_implicit_left_side
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource "Can't determine which FROM clause to join "
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.
2019-03-09 10:31:26.785 624233 ERROR neutron.api.v2.resource
[pid: 624233|app: 0|req: 2/13] 192.168.101.2 () {34 vars in 758 bytes} [Sat Mar 9 10:31:26 2019] PUT /v2.0/floatingips/0c403275-0198-42b2-b454-b091a2de42ec => generated 150 bytes in 459 msecs (HTTP/1.1 500) 3 headers in 155 bytes (2 switches on core 0)

Revision history for this message
Mike Bayer (zzzeek) wrote :

the correct coding pattern is in https://review.openstack.org/#/c/642117/ however this is still failing tests.

OpenStack Infra (hudson-openstack)
Changed in neutron:
assignee: nobody → Mike Bayer (zzzeek)
status: New → In Progress
OpenStack Infra (hudson-openstack)
Changed in neutron:
assignee: Mike Bayer (zzzeek) → LIU Yulong (dragon889)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/642117
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c04b8ddd907d786c0ea2fccd31b82561f2cdb8fc
Submitter: Zuul
Branch: master

commit c04b8ddd907d786c0ea2fccd31b82561f2cdb8fc
Author: Mike Bayer <email address hidden>
Date: Fri Mar 8 14:09:14 2019 -0500

    Join on explcit relationship paths

    The join() in get_router_for_floatingip() is joining from entity
    to entity without an explicit ON clause which creates an ambiguous
    situation. SQLAlchemy 1.3 guards against this now, so use the
    real relationship-bound path so that the ORM does not need to guess.

    Closes-bug: #1819260
    Change-Id: Ia377a9d1a32a78abdaee74c79e395acd77e486ef

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 14.0.0.0rc1

This issue was fixed in the openstack/neutron 14.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.