OSTF test fail to remove floating IP: "ClientException: Unexpected API Error. <class 'oslo_db.exception.DBDeadlock'> (HTTP 500)"

[Artem Panchenko]

Sometimes OSTF test "Launch instance with file injection" fails on step #5:

2016-02-16 07:13:44 INFO (test_mixins) STEP:5, verify action: 'removing floating IP'
2016-02-16 07:13:45 DEBUG (test_mixins) Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/fuel_health/common/test_mixins.py", line 177, in verify
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 662, in remove_floating_ip
    self._action('removeFloatingIp', server, {'address': address})
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1279, in _action
    return self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 449, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 424, in _cs_request
    resp, body = self._time_request(url, method, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 397, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 391, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_db.exception.DBDeadlock'> (HTTP 500) (Request-ID: req-ac412e1c-afcf-4ef3-accc-b5463805ca74)

2016-02-16 07:13:45 ERROR (nose_storage_plugin) fuel_health.tests.smoke.test_nova_create_instance_with_connectivity.TestNovaNetwork.test_009_create_server_with_file
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
    yield
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 601, in run
    testMethod()
  File "/usr/lib/python2.7/site-packages/fuel_health/tests/smoke/test_nova_create_instance_with_connectivity.py", line 358, in test_009_create_server_with_file
    "removing floating IP", server, floating_ip)
  File "/usr/lib/python2.7/site-packages/fuel_health/common/test_mixins.py", line 183, in verify
    " Please refer to OpenStack logs for more details.")
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
    raise self.failureException(msg)
AssertionError: Step 5 failed: Floating IP cannot be removed. Please refer to OpenStack logs for more details.

Diagnostic snapshot is attached.

[Roman Podoliaka (rpodolyaka)]

This can only be caused by running Galera in multiwriter mode (probably unintentionally, due to https://bugs.launchpad.net/fuel/+bug/1490595).

Anyway, this should be a really rare case, as usually instance_info_cache_update() is called only from 1 nova-compute at a time, except for the case when it can be updated from nova-api on association / disassociation of a floating IP.

[Tatyanka (tatyana-leontovich)]

ostf do not execute galera run in multiwrite mode- so remove tag ostf

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

version

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Roman Podoliaka (rpodolyaka)]

No longer fixing Medium bugs after SCF in 9.0, closing as Won't Fix.

[Dina Belova (dbelova)]

Added move-to-10.0 tag due to the fact bug was transferred from 9.0 to 10.0

[Roman Podoliaka (rpodolyaka)]

Fixed in https://review.openstack.org/302714