RunInstances fail due to state change of _poll_instance_state()
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
High
|
justinsb |
Bug Description
run_instance sometimes fails.
tested: r894, qemu.
If _poll_instance_
I think that following state changes should not be applied to just spawning instances:
2011-03-28 15:18:18,737 INFO nova.compute.
2011-03-28 15:18:18,738 INFO nova.compute.
Here are nova-compute's log output:
2011-03-28 15:17:33,337 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /home/iida/
2011-03-28 15:17:33,375 INFO nova.virt.
2011-03-28 15:17:33,438 DEBUG nova.utils [-] Attempting to grab semaphore "481bc4e3" for method "call_if_
2011-03-28 15:17:33,616 DEBUG nova.utils [-] Running cmd (subprocess): cp /home/iida/
2011-03-28 15:17:33,743 DEBUG nova.utils [-] Attempting to grab semaphore "395a3690" for method "call_if_
2011-03-28 15:17:34,087 DEBUG nova.utils [-] Running cmd (subprocess): cp /home/iida/
2011-03-28 15:17:34,201 DEBUG nova.utils [-] Attempting to grab semaphore "6a4902cb_sm" for method "call_if_
2011-03-28 15:18:18,200 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img create -f qcow2 -o cluster_
2011-03-28 15:18:18,737 INFO nova.compute.
2011-03-28 15:18:18,738 INFO nova.compute.
2011-03-28 15:18:19,167 INFO nova.virt.
2011-03-28 15:18:19,167 INFO nova.virt.
2011-03-28 15:18:19,177 DEBUG nova.utils [-] Running cmd (subprocess): sudo qemu-nbd -c /dev/nbd15 /home/iida/
2011-03-28 15:18:20,799 DEBUG nova.utils [-] Running cmd (subprocess): sudo tune2fs -c 0 -i 0 /dev/nbd15 from (pid=14228) execute /home/iida/
2011-03-28 15:18:21,964 DEBUG nova.utils [-] Running cmd (subprocess): sudo mount /dev/nbd15 /tmp/tmpkRFDUp from (pid=14228) execute /home/iida/
2011-03-28 15:18:22,181 DEBUG nova.utils [-] Running cmd (subprocess): sudo mkdir -p /tmp/tmpkRFDUp/
2011-03-28 15:18:22,341 DEBUG nova.utils [-] Running cmd (subprocess): sudo chown root /tmp/tmpkRFDUp/
2011-03-28 15:18:22,490 DEBUG nova.utils [-] Running cmd (subprocess): sudo chmod 700 /tmp/tmpkRFDUp/
2011-03-28 15:18:22,590 DEBUG nova.utils [-] Running cmd (subprocess): sudo tee -a /tmp/tmpkRFDUp/
2011-03-28 15:18:22,689 DEBUG nova.utils [-] Running cmd (subprocess): sudo mkdir -p /tmp/tmpkRFDUp/
2011-03-28 15:18:22,790 DEBUG nova.utils [-] Running cmd (subprocess): sudo chown root:root /tmp/tmpkRFDUp/
2011-03-28 15:18:22,873 DEBUG nova.utils [-] Running cmd (subprocess): sudo chmod 755 /tmp/tmpkRFDUp/
2011-03-28 15:18:22,950 DEBUG nova.utils [-] Running cmd (subprocess): sudo tee /tmp/tmpkRFDUp/
2011-03-28 15:18:23,060 DEBUG nova.utils [-] Running cmd (subprocess): sudo umount /dev/nbd15 from (pid=14228) execute /home/iida/
2011-03-28 15:18:24,190 DEBUG nova.utils [-] Running cmd (subprocess): rmdir /tmp/tmpkRFDUp from (pid=14228) execute /home/iida/
2011-03-28 15:18:24,258 DEBUG nova.utils [-] Running cmd (subprocess): sudo qemu-nbd -d /dev/nbd15 from (pid=14228) execute /home/iida/
2011-03-28 15:18:28,116 DEBUG nova.virt.
2011-03-28 15:18:28,243 ERROR nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
(nova.compute.
2011-03-28 15:18:28,345 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/home/
(nova): TRACE: rval = node_func(
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kw)
(nova): TRACE: File "/home/
(nova): TRACE: power_state.
(nova): TRACE: File "/home/
(nova): TRACE: return IMPL.instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: 'state_
(nova): TRACE: File "/home/
(nova): TRACE: return IMPL.instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: instance_ref = instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: instance_id)
(nova): TRACE: InstanceNotFound: Instance 1 not found
(nova): TRACE:
2011-03-28 15:18:28,613 ERROR nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
(nova.virt.
2011-03-28 15:18:28,700 ERROR nova [-] in looping call
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/home/
(nova): TRACE: self.f(*self.args, **self.kw)
(nova): TRACE: File "/home/
(nova): TRACE: power_state.
(nova): TRACE: File "/home/
(nova): TRACE: return IMPL.instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: 'state_
(nova): TRACE: File "/home/
(nova): TRACE: return IMPL.instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: instance_ref = instance_
(nova): TRACE: File "/home/
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/home/
(nova): TRACE: instance_id)
(nova): TRACE: InstanceNotFound: Instance 1 not found
(nova): TRACE:
Related branches
- Vish Ishaya (community): Approve
- Jay Pipes (community): Approve
-
Diff: 41 lines (+21/-4)1 file modifiednova/compute/manager.py (+21/-4)
Changed in nova: | |
importance: | Undecided → High |
status: | New → In Progress |
tags: | added: regression |
Changed in nova: | |
status: | In Progress → Fix Committed |
Changed in nova: | |
milestone: | none → 2011.2 |
status: | Fix Committed → Fix Released |
Thanks for the great bug report. This sounds like a bug I introduced, so I will work on fixing it.