get_vnc_console returns 500 error if called before host is assigned

Bug #1037809 reported by Johannes Erdfelt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Russell Bryant

Bug Description

There exists a window between a server create is done and when the host is assigned by the scheduler that a request to get the vnc console will result in a 500 error.

An exception similar to this will be seen in the logs:

2012-08-16 21:42:43 TRACE nova.api.openstack Traceback (most recent call last):
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/api/openstack/__init__.py", line 78, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return req.get_response(self.application)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/request.py", line 1053, in get_response
2012-08-16 21:42:43 TRACE nova.api.openstack application, catch_exc_info=False)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/request.py", line 1022, in call_application
2012-08-16 21:42:43 TRACE nova.api.openstack app_iter = application(self.environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 159, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return resp(environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/keystone-2012.2-py2.6.egg/keystone/middleware/auth_token.py", line 213, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return self.app(env, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 159, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return resp(environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 159, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return resp(environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/Routes-1.13-py2.6.egg/routes/middleware.py", line 131, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack response = self.app(environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 159, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack return resp(environ, start_response)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 147, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack resp = self.call_func(req, *args, **self.kwargs)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/virtualenvs/nova/lib/python2.6/site-packages/webob/dec.py", line 208, in call_func
2012-08-16 21:42:43 TRACE nova.api.openstack return self.func(req, *args, **kwargs)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/api/openstack/wsgi.py", line 864, in __call__
2012-08-16 21:42:43 TRACE nova.api.openstack content_type, body, accept)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/api/openstack/wsgi.py", line 912, in _process_stack
2012-08-16 21:42:43 TRACE nova.api.openstack action_result = self.dispatch(meth, request, action_args)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/api/openstack/wsgi.py", line 1000, in dispatch
2012-08-16 21:42:43 TRACE nova.api.openstack return method(req=request, **action_args)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/api/openstack/compute/contrib/consoles.py", line 48, in get_vnc_console
2012-08-16 21:42:43 TRACE nova.api.openstack console_type)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/compute/api.py", line 120, in wrapped
2012-08-16 21:42:43 TRACE nova.api.openstack return func(self, context, target, *args, **kwargs)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/compute/api.py", line 1631, in get_vnc_console
2012-08-16 21:42:43 TRACE nova.api.openstack instance=instance, console_type=console_type)
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/compute/rpcapi.py", line 254, in get_vnc_console
2012-08-16 21:42:43 TRACE nova.api.openstack topic=_compute_topic(self.topic, ctxt, None, instance),
2012-08-16 21:42:43 TRACE nova.api.openstack File "/home/johannes/openstack/nova/trunk/nova/compute/rpcapi.py", line 48, in _compute_topic
2012-08-16 21:42:43 TRACE nova.api.openstack 'Instance %s') % instance['uuid'])
2012-08-16 21:42:43 TRACE nova.api.openstack NovaException: Unable to find host for Instance 1a3f3a81-fd2f-4a99-b446-5e03216fe045

Changed in nova:
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Russell Bryant (russellb)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/11574

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/11574
Committed: http://github.com/openstack/nova/commit/fa16d173938b30af4e1cc6b312a75b12aff6c424
Submitter: Jenkins
Branch: master

commit fa16d173938b30af4e1cc6b312a75b12aff6c424
Author: Russell Bryant <email address hidden>
Date: Fri Aug 17 15:21:36 2012 -0400

    Fix get_vnc_console race.

    There exists a window between a server create is done and when the host
    is assigned by the scheduler that a request to get the vnc console will
    result in a 500 error.

    This patch adds a check in the compute API to ensure that the instance
    has a host assigned before try to execute a rpc on the compute node.
    If not, it raises InstanceNotReady (which translates to a 409 in the
    OS API).

    Fix bug 1037809.

    Change-Id: I3f5af90d57ae84f98f787e14ccb66f1841ac0c6d

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → folsom-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: folsom-rc1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.