gazillion 'ClientException: The server has either erred or is incapable of performing the requested operation' traces

[Armando Migliaccio]

2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'tag': u'27846c36-9dc4-4822-a2a2-fd3fd0cca019', 'name': 'network-vif-deleted', 'server_uuid': u'f6aab00c-b64e-4054-8f26-4dd232d7bbf7'}]
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova Traceback (most recent call last):
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova File "/opt/stack/new/neutron/neutron/notifiers/nova.py", line 252, in send_events
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova batched_events)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/contrib/server_external_events.py", line 39, in create
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova return_raw=True)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 161, in _create
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova _resp, body = self.api.client.post(url, body=body)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova return self.request(url, 'POST', **kwargs)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 104, in request
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova raise exceptions.from_response(resp, body, url, method)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b385d794-86fc-4b06-a183-a2de985f99d5)
2015-06-03 02:46:50.553 22041 ERROR neutron.notifiers.nova

Most likely introduced by:

https://review.openstack.org/#/c/178666/

Logstash:

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiQ2xpZW50RXhjZXB0aW9uOiBUaGUgc2VydmVyIGhhcyBlaXRoZXIgZXJyZWQgb3IgaXMgaW5jYXBhYmxlIG9mIHBlcmZvcm1pbmcgdGhlIHJlcXVlc3RlZCBvcGVyYXRpb24uXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MzMzMDk3OTk4NTB9

[Armando Migliaccio (armando-migliaccio)]

Looks like this one needs to land:

https://review.openstack.org/#/c/179390/

[Armando Migliaccio (armando-migliaccio)]

Not sure if it's okay for Neutron to barf this bad if there's a misalignment with the server capabilities.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/187868

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/187868
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6575db592c92791a51540134192bc86465940283
Submitter: Jenkins
Branch: master

commit 6575db592c92791a51540134192bc86465940283
Author: Kevin Benton <email address hidden>
Date: Wed Jun 3 05:52:51 2015 +0000

    Revert "Add VIF_DELETED notification event to Nova"

    We need to wait until the nova support is added in
    I998b6bb80cc0a81d665b61b8c4a424d7219c666f. Otherwise
    this generates a ton of error messages in the nova api
    log as well as on the neutron side.

    This reverts commit 0ace88fd4a75ff213dc36fd16c1f8e7080ab7d6d.

    Change-Id: I129c4e4c05cf07d45032fec6f57e0cc17a5a82af
    Closes-Bug: #1461391

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/196701

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kyle Mestery (<email address hidden>) on branch: feature/pecan
Review: https://review.openstack.org/196701
Reason: This is lacking the functional fix [1], so I'll propose a new merge commit which includes that one.

[1] https://review.openstack.org/#/c/196711/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/196920