OpenStack Compute (Nova)

when creating instance root image file,nova does not check backing file is ready.file is

Reported by Yaguang Tang on 2012-03-13
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
High
Anthony Young

Bug Description

when starting an instance to a compute node which does not has the system image file,
nova first download the file through glance api,and then try to use qemu-img resize to create a backing file which used by instances;when creating this backing file,a create instance request comes ,nova does not check whether the backing file is ready.
so the new instane root image file is not complete, and can not be booted .
the following log shows the situation.

2012-03-13 14:47:28 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /var/lib/nova/instances/instance-00000009/ from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:28 INFO nova.virt.libvirt.connection [-] [instance: 4de2f829-c846-4f87-b06a-ed3515f1ef21] Creating image
2012-03-13 14:47:28 DEBUG nova.utils [-] Attempting to grab semaphore "e10ff21f-f330-444f-8175-01246feed704" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:834
2012-03-13 14:47:28 DEBUG nova.utils [-] Got semaphore "e10ff21f-f330-444f-8175-01246feed704" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:838
2012-03-13 14:47:28 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/nova/instances/_base/e10ff21f-f330-444f-8175-01246feed704 /var/lib/nova/instances/instance-00000009/kernel from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:28 DEBUG nova.virt.libvirt.connection [-] block_device_list [] from (pid=31646) _volume_in_mapping /usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py:1235
2012-03-13 14:47:28 DEBUG nova.utils [-] Attempting to grab semaphore "2bd6e387adad362fedd4a477fe9d7c6590cddf57" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:834
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57.part from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57 /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:40 DEBUG nova.utils [-] Got semaphore "2bd6e387adad362fedd4a477fe9d7c6590cddf57" for method "call_if_not_exists"... from (pid=31646) inner /usr/lib/python2.7/dist-packages/nova/utils.py:838
2012-03-13 14:47:40 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img create -f qcow2 -o cluster_size=2M,backing_file=/var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 /var/lib/nova/instances/instance-00000009/disk from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:42 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img resize /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 10737418240 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213
2012-03-13 14:47:42 DEBUG nova.utils [-] Running cmd (subprocess): e2fsck -fp /var/lib/nova/instances/_base/2bd6e387adad362fedd4a477fe9d7c6590cddf57_10 from (pid=31646) execute /usr/lib/python2.7/dist-packages/nova/utils.py:213

Changed in nova:
assignee: nobody → Anthony Young (sleepsonthefloor)
Changed in nova:
milestone: none → essex-rc1
importance: Undecided → High
status: New → In Progress

Reviewed: https://review.openstack.org/5296
Committed: http://github.com/openstack/nova/commit/aa204ea41b88e6896f5505df2d35cdb275cf6187
Submitter: Jenkins
Branch: master

commit aa204ea41b88e6896f5505df2d35cdb275cf6187
Author: Anthony Young <email address hidden>
Date: Tue Mar 13 14:44:31 2012 -0700

    Fix backing file cp/resize race condition.

     * Fixes bug 953831

    Change-Id: I39950b16c9b76159b16203f7b8b504cae5475516

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-03-20
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-rc1 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers