Image deletion causes instance hard-reboot failure

[Darren Sanders]

When an instance is using an image that gets deleted, the instance will fail to launch if it is hard-rebooted.

Steps to reproduce:

    1. Launch new instance using cirros 0.3.2 image

    2. After instance is done booting, delete the cirros 0.3.2 image files

    3. Return to 'instances' page and select "hard reboot" for instance

Expected behavior:

    Image should successfully reboot

Actual behvior:

    UI reports "Error: Failed to launch instance ..." (image could not be found),
    and the instance goes into error state

[Darren Sanders (darren-s)]

Additional detail and steps to reproduce using the CLI.

CLI commands to reproduce
=========================
$ nova boot --availability-zone nova --image cirros-0.3.2-x86_64-uec --flavor m1.tiny inst1
$ glance image-delete cirros-0.3.2-x86_64-uec cirros-0.3.2-x86_64-uec-kernel cirros-0.3.2-x86_64-uec-ramdisk
$ nova reboot --hard bcfc5d83-7d22-42c3-b7be-cf83c63c5f10

$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------+
| bcfc5d83-7d22-42c3-b7be-cf83c63c5f10 | inst1 | ERROR | - | Running | private=10.0.0.3 |
+--------------------------------------+-------+--------+------------+-------------+------------------+

Stacktrace from Nova Compute screen log:
========================================

2015-03-11 12:53:17.318 ERROR oslo_messaging.rpc.dispatcher [req-c28bee61-2e10-4363-8f6a-fc83ee0fe48f admin admin] Exception during message handling: Image 68a7d315-5ef6-436f-bd39-739d0076f99c could not be found.
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher payload)
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/nova/nova/compute/manager.py", line 302, in decorated_function
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher pass
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 82, in __exit__
2015-03-11 12:53:17.318 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_...

[Darren Sanders (darren-s)]

This test was done using Nova Master branch

Nova commit 4103a86f30bf7da3e4616d9d769213b128f0946d

I've also confirmed the bug in Icehouse, so presumably it also exists in Juno.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/163661