nova network-vif-plugged event fails with 400 from n-api
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
New
|
Undecided
|
Unassigned |
Bug Description
Running Tempest in our internal CI we hit 3 test failures out of 1,971 tests, one was:
tempest.
This is running x86_64 RHEL 6.5 with the nova libvirt driver and neutron openvswitch.
We have neutron.conf configured for nova events:
[root@rhel62 ~]# cat /etc/neutron/
# being used in conjunction with nova security groups
# allowed_
nova_url = http://
nova_region_name = RegionOne
nova_admin_username = nova
nova_admin_
nova_admin_password = nova
nova_admin_auth_url = http://
For one failure I'm seeing this in the neutron server log:
2014-03-16 05:57:36.619 8462 ERROR neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
2014-03-16 05:57:36.619 8462 TRACE neutron.
I correlated that server uuid to the nova-api log error here:
2014-03-16 05:57:27.770 12207 INFO nova.osapi_
2014-03-16 05:57:36.615 12204 ERROR nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.615 12204 TRACE nova.api.
2014-03-16 05:57:36.616 12204 INFO nova.osapi_
Talking with Dan Smith in IRC it sounds like maybe neutron isn't putting the nova tenant ID into the request for some reason?
(4:15:57 PM) dansmith: mriedem: I think neutron sometimes gets the nova url from the context you pass it, so maybe it's related to that
...
(4:17:20 PM) dansmith: mriedem: right, but arosen did some work to have neutron determine the proper nova endpoint to call back to based on the context, in the case you've got two novas sharing a neutron
This is intermittent, but I do see the neutron server error showing up in community logstash, but only 7% failures:
I'm not seeing the same error in logstash in nova-api logs, but that was only going back the last 48 hours.
tags: | removed: in-stable-icehouse |
Oh and this is nova/neutron code from master as of 3/26, and novaclient is 2.17 and neutronclient is 2.3.4.