Floating IP association to allowed address pair fails due to transaction in transaction

[Daniel Russell]

The environment is OpenStack Newton, using openvswitch as network plugin. The router in the network used was created as distributed and ha.

I deployed a single Octavia Amphora (which uses allowed address pairs to present the service IP address to the tenant) and then attempted to associate a floating IP address with the allowed address pair. That resulted in the stack trace you see below :

update failed: No details.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 79, in resource
    result = method(request=request, **args)
  File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 604, in update
    return self._update(request, id, body, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
    setattr(e, '_RETRY_EXCEEDED', True)
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
    ectxt.value = e.inner_exc
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
    traceback.format_exc())
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
    return f(*dup_args, **dup_kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/api/v2/base.py", line 652, in _update
    obj = obj_updater(request.context, id, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 159, in wrapped
    return method(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 88, in wrapped
    setattr(e, '_RETRY_EXCEEDED', True)
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 84, in wrapped
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
    ectxt.value = e.inner_exc
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 124, in wrapped
    traceback.format_exc())
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 119, in wrapped
    return f(*dup_args, **dup_kwargs)
  File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 1020, in update_floatingip
    context, id, floatingip)
  File "/usr/lib/python2.7/site-packages/neutron/db/l3_db.py", line 1352, in _update_floatingip
    context.elevated(), fip_port_id))
  File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 284, in _update_fip_assoc
    port)
  File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 297, in _inherit_service_port_and_arp_update
    address_pair_port=allowed_address_port))
  File "/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py", line 1073, in update_unbound_allowed_address_pair_port_binding
    context, address_pair_port['id'], {'port': port_data})
  File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 755, in inner
    "transaction.") % f)
RuntimeError: Method <function update_port at 0x4cbf6e0> cannot be called within a transaction.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422309

[Ann Taraday (akamyshnikova)]

Can you please provide steps how do you get this error?

[James Anziano (janzian)]

Agreed, some exact reproduction steps would be very useful in this bug.

[Daniel Russell (danielr-2)]

Sorry all.

The environment is OpenStack Newton, using openvswitch as network plugin. The router in the network used was created as distributed and ha.

I deployed a single Octavia Amphora (which uses allowed address pairs to present the service IP address to the tenant) and then attempted to associate a floating IP address with it. That resulted in the stack trace you see above.

[James Anziano (janzian)]

No need to apologize :)

If you would please edit the contents of the original report to include the contents of your comment, preferably also with the commands you ran for those who aren't as familiar with this area of neutron, we'll remove the Incomplete status.

[Daniel Russell (danielr-2)]

Looks like the flow for this doesn't go via this path in Ocata and works fine. It's probably easiest just to close this off and recommend upgrading to Ocata

[OpenStack Infra (hudson-openstack)]

Change abandoned by Daniel Russell (<email address hidden>) on branch: master
Review: https://review.openstack.org/422309

[Kevin Benton (kevinbenton)]

It looks like your patch may still be valid for newton. Was it a problem then?