when creating instance root image file,nova does not check backing file is ready.file is

Bug #953831 reported by Yaguang Tang
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Anthony Young

Bug Description

when starting an instance to a compute node which does not has the system image file,
nova first download the file through glance api,and then try to use qemu-img resize to create a backing file which used by instances;when creating this backing file,a create instance request comes ,nova does not check whether the backing file is ready.
so the new instane root image file is not complete, and can not be booted .
the following log shows the situation.

2012-03-13 14:47:28 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /var/lib/nova/instances/instance-00000009/ from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:28 INFO nova.virt.libvirt.connection [-] [instance: 4de2f829-c846-4f87-b06a-ed3515f1ef21] Creating image
2012-03-13 14:47:28 DEBUG nova.utils [-] Attempting to grab semaphore "e10ff21f-f330-444f-8175-01246feed704" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:834
2012-03-13 14:47:28 DEBUG nova.utils [-] Got semaphore "e10ff21f-f330-444f-8175-01246feed704" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:838
2012-03-13 14:47:28 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/nova/instances/_base/e10ff21f-f330-444f-8175-01246feed704 /var/lib/nova/instances/instance-00000009/kernel from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:28 DEBUG nova.virt.libvirt.connection [-] block_device_list [] from (pid=31646) _volume_in_mapping /usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py:1235
2012-03-13 14:47:28 DEBUG nova.utils [-] Attempting to grab semaphore "2bd6e387adad362fedd4a477fe9d7c6590cddf57" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:834
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57.part from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57 /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:40 DEBUG nova.utils [-] Got semaphore "2bd6e387adad362fedd4a477fe9d7c6590cddf57" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:838
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img create -f qcow2 -o cluster_size=2M,backing_file=/var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 /var/lib/nova/instances/instance-00000009/disk from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:42 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img resize /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 10737418240 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:42 DEBUG nova.utils [-] Running cmd (subprocess): e2fsck -fp /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213

Changed in nova:
assignee: nobody → Anthony Young (sleepsonthefloor)
Changed in nova:
milestone: none → essex-rc1
importance: Undecided → High
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/5296

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/5296
Committed: http://github.com/openstack/nova/commit/aa204ea41b88e6896f5505df2d35cdb275cf6187
Submitter: Jenkins
Branch: master

commit aa204ea41b88e6896f5505df2d35cdb275cf6187
Author: Anthony Young <email address hidden>
Date: Tue Mar 13 14:44:31 2012 -0700

    Fix backing file cp/resize race condition.

     * Fixes bug 953831

    Change-Id: I39950b16c9b76159b16203f7b8b504cae5475516

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: essex-rc1 → 2012.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.