I should scroll down more in the future, here is the actual stacktrace that is causing the failure (it's may be related to the above messages)
http://logs.openstack.org/83/47283/1/gate/gate-tempest-devstack-vm-full/cdc49f9/logs/screen-n-cpu.txt.gz#_2013-09-23_14_18_41_867
2013-09-23 14:18:41.867 ERROR nova.compute.manager [req-25a33c89-f4ba-4250-96e0-a122bb16b3c2 tempest.scenario.manager348477648-user tempest.scenario.manager1015858706-tenant] [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] Instance failed to spawn 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] Traceback (most recent call last): 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/compute/manager.py", line 1408, in _spawn 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] block_device_info) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2068, in spawn 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] write_to_disk=True) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3026, in to_xml 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] disk_info, rescue, block_device_info) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2912, in get_guest_config 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] inst_type): 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2738, in get_guest_storage_config 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] info) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1030, in volume_driver_method 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] return method(connection_info, *args, **kwargs) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/openstack/common/lockutils.py", line 246, in inner 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] return f(*args, **kwargs) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] File "/opt/stack/new/nova/nova/virt/libvirt/volume.py", line 277, in connect_volume 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] % (host_device)) 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7] NovaException: iSCSI device not found at /dev/disk/by-path/ip-127.0.0.1:3260-iscsi-iqn.2010-10.org.openstack:volume-b31ec3e4-74f0-4aa8-aa25-2fcdf70e8d1b-lun-1 2013-09-23 14:18:41.867 23218 TRACE nova.compute.manager [instance: 196feb6a-fdd7-4ff6-9fb9-863b92c834a7]
I should scroll down more in the future, here is the actual stacktrace that is causing the failure (it's may be related to the above messages)
http:// logs.openstack. org/83/ 47283/1/ gate/gate- tempest- devstack- vm-full/ cdc49f9/ logs/screen- n-cpu.txt. gz#_2013- 09-23_14_ 18_41_867
2013-09-23 14:18:41.867 ERROR nova.compute. manager [req-25a33c89- f4ba-4250- 96e0-a122bb16b3 c2 tempest. scenario. manager34847764 8-user tempest. scenario. manager10158587 06-tenant] [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] Instance failed to spawn manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] Traceback (most recent call last): manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/compute/ manager. py", line 1408, in _spawn manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] block_device_info) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 2068, in spawn manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] write_to_disk=True) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 3026, in to_xml manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] disk_info, rescue, block_device_info) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 2912, in get_guest_config manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] inst_type): manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 2738, in get_guest_ storage_ config manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] info) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 1030, in volume_ driver_ method manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] return method( connection_ info, *args, **kwargs) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/openstack/ common/ lockutils. py", line 246, in inner manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] return f(*args, **kwargs) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ volume. py", line 277, in connect_volume manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] % (host_device)) manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7] NovaException: iSCSI device not found at /dev/disk/ by-path/ ip-127. 0.0.1:3260- iscsi-iqn. 2010-10. org.openstack: volume- b31ec3e4- 74f0-4aa8- aa25-2fcdf70e8d 1b-lun- 1 manager [instance: 196feb6a- fdd7-4ff6- 9fb9-863b92c834 a7]
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.
2013-09-23 14:18:41.867 23218 TRACE nova.compute.