server lists return 404 when network api raises InstanceNotFound

[Mark Washenberger]

Sometimes, a problem on the network manager might cause an instance not to exist in the network DB even though it exists in the nova DB. This causes server lists in Nova to return 404 even though the instance was originally found (and also there may be lots of other instances that don't have this problem). Here are some example logs from the nova api.

2012-02-09 16:13:51,845 DEBUG routes.middleware [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Matched GET /658803/servers/detail from (pid=23382) __call__ /usr/lib/pymodules/python2.6/routes/middleware.py:100
2012-02-09 16:13:51,845 DEBUG routes.middleware [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Route path: '/{project_id}/servers/detail', defaults: {'action': u'detail', 'controller': <nova.api.openstack.wsgi.Resource object at 0x19d5c50>} from (pid=23382) __call__ /usr/lib/pymodules/python2.6/routes/middleware.py:102
2012-02-09 16:13:51,846 DEBUG routes.middleware [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Match dict: {'action': u'detail', 'controller': <nova.api.openstack.wsgi.Resource object at 0x19d5c50>, 'project_id': u'658803'} from (pid=23382) __call__ /usr/lib/pymodules/python2.6/routes/middleware.py:103
2012-02-09 16:13:51,846 INFO nova.api.openstack.wsgi [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] GET http://127.0.0.1:8775/v2/658803/servers/detail?changes-since=2011-03-09T18:01:55Z
2012-02-09 16:13:51,847 DEBUG nova.api.openstack.wsgi [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Unrecognized Content-Type provided in request from (pid=23382) get_body /usr/lib/python2.6/dist-packages/nova/api/openstack/wsgi.py:680
2012-02-09 16:13:51,847 DEBUG nova.api.openstack.compute.servers [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Removing options '' from query from (pid=23382) remove_invalid_options /usr/lib/python2.6/dist-packages/nova/api/openstack/compute/servers.py:1195
2012-02-09 16:13:51,848 DEBUG nova.compute.api [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Searching by: {'project_id': '658803', 'local_zone_only': False, 'changes-since': datetime.datetime(2011, 3, 9, 18, 1, 55)} from (pid=23382) get_all /usr/lib/python2.6/dist-packages/nova/compute/api.py:1018
2012-02-09 16:13:51,998 DEBUG nova.api.openstack.common [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Generated ACTIVE from vm_state=active task_state=None. from (pid=23382) status_from_state /usr/lib/python2.6/dist-packages/nova/api/openstack/common.py:96
2012-02-09 16:13:52,002 DEBUG nova.api.openstack.common [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Generated DELETED from vm_state=deleted task_state=None. from (pid=23382) status_from_state /usr/lib/python2.6/dist-packages/nova/api/openstack/common.py:96
2012-02-09 16:13:52,003 DEBUG nova.rpc [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] Making asynchronous call on network ... from (pid=23382) multicall /usr/lib/python2.6/dist-packages/nova/rpc/amqp.py:317
2012-02-09 16:13:52,003 DEBUG nova.rpc [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] MSG_ID is bf93742d13a44ebbb69862a306b8f1cb from (pid=23382) multicall /usr/lib/python2.6/dist-packages/nova/rpc/amqp.py:320
2012-02-09 16:13:52,171 INFO nova.api.openstack.wsgi [req-2323e03b-a587-46df-8e2f-45fa78ee967f 171019 658803] HTTP exception thrown: 404 Not Found

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/4020

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/4020
Committed: http://github.com/openstack/nova/commit/58f8f93b52e7de2d3751dd3291427a65a3c1079b
Submitter: Jenkins
Branch: master

commit 58f8f93b52e7de2d3751dd3291427a65a3c1079b
Author: Mark Washenberger <email address hidden>
Date: Fri Feb 10 13:46:24 2012 -0500

    Handle network api failures more gracefully

    addresses bug 930329

    Change-Id: Idb0934b9dd6ebc0e5e81495fb4454e4e50bc1f57