get-vnc-console while in BUILD results in traceback

Bug #1154327 reported by Russell Bryant
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Russell Bryant

Bug Description

If you try to get a VNC console while an instance is still in BUILD, it can fail because the instance doesn't actually exist yet on the hypervisor. Here is the exception in the nova-api log from doing this:

2013-03-12 18:49:55.312 INFO nova.api.openstack.wsgi [req-34f3671e-573e-4e63-a35b-868dac563537 demo demo] HTTP exception thrown: Instance instance-00000001 could not be found.
Traceback (most recent call last):

  File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 393, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)

  File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 133, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)

  File "/opt/stack/nova/nova/exception.py", line 117, in wrapped
    temp_level, payload)

  File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()

  File "/opt/stack/nova/nova/exception.py", line 94, in wrapped
    return f(self, context, *args, **kw)

  File "/opt/stack/nova/nova/compute/manager.py", line 225, in decorated_function
    return function(self, context, *args, **kwargs)

  File "/opt/stack/nova/nova/compute/manager.py", line 2604, in get_vnc_console
    connect_info = self.driver.get_vnc_console(instance)

  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1624, in get_vnc_console
    port = get_vnc_port_for_instance(instance['name'])

  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1615, in get_vnc_port_for_instance
    virt_dom = self._lookup_by_name(instance_name)

  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2336, in _lookup_by_name
    raise exception.InstanceNotFound(instance_id=instance_name)

InstanceNotFound: Instance instance-00000001 could not be found.

Changed in nova:
assignee: nobody → Russell Bryant (russellb)
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/24255

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/24353

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/24255
Committed: http://github.com/openstack/nova/commit/dced117ea919b74d9521ff422b4eb8a880474a0d
Submitter: Jenkins
Branch: master

commit dced117ea919b74d9521ff422b4eb8a880474a0d
Author: Russell Bryant <email address hidden>
Date: Wed Mar 13 14:45:49 2013 -0400

    Generalize console error handling during build.

    There is a race condition where you can create an instance and then
    quickly request a VNC or Spice console before the instance is ready.
    The way this went down was different depending on the virt driver in
    use. The libvirt driver would raise InstanceNotFound. The xenapi
    driver would raise InstanceNotReady.

    This patch moves the handling of this race that was in the xenapi driver
    up to the compute manager. Now, all of the virt drivers that support
    this method (libvirt, xenapi, vmware) will all raise InstanceNotFound in
    this case, and the compute manager will convert it into InstanceNotReady
    IFF the vm_state is BUILDING.

    Related to bug 1154327.

    Change-Id: I68f4a6db8aac26c6f731c985d97299ee38c34448

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/24353
Committed: http://github.com/openstack/nova/commit/a02b2d77d3bd40954c9c8dda680e2f7ce3e0327e
Submitter: Jenkins
Branch: master

commit a02b2d77d3bd40954c9c8dda680e2f7ce3e0327e
Author: Russell Bryant <email address hidden>
Date: Wed Mar 13 15:24:05 2013 -0400

    Don't log traceback on expected console error.

    When requesting a console before an instance is read (while still
    building) we will get an InstanceNotReady exception. This exception is
    raised in the compute manager. The explanation in the exception will
    include the traceback from the manager side. If we use this as the
    explanation, we end up with a traceback in the API log, even though it's
    just an INFO log message.

    Use a more basic explanation, which is accuately more consistent with
    other cases of raising HTTP exceptions, which doesn't include a
    traceback.

    Fix bug 1154327.

    Change-Id: Idca443165983289a0df40970f68f95d5550ce72a

Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-rc1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.