Hello guys! We faced with the same issues today on gates: 2015-02-19 11:26:02.215 5467 ERROR nova.compute.manager [-] [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Instance failed to spawn 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Traceback (most recent call last): 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/compute/manager.py", line 2316, in _build_resources 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] yield resources 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/compute/manager.py", line 2186, in _build_and_run_instance 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] flavor=flavor) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2358, in spawn 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] admin_pass=admin_password) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2762, in _create_image 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] project_id=instance.project_id) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 230, in cache 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] *args, **kwargs) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 477, in create_image 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] prepare_template(target=base, max_size=size, *args, **kwargs) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 431, in inner 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] return f(*args, **kwargs) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 220, in fetch_func_sync 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] fetch_func(target=target, *args, **kwargs) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/utils.py", line 488, in fetch_image 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] max_size=max_size) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/images.py", line 121, in fetch_to_raw 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] convert_image(path_tmp, staged, 'raw') 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/images.py", line 68, in convert_image 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] utils.execute(*cmd, run_as_root=run_as_root) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/utils.py", line 197, in execute 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] return processutils.execute(*cmd, **kwargs) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 228, in execute 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] cmd=sanitized_cmd) 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] ProcessExecutionError: Unexpected error while running command. 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Command: qemu-img convert -O raw /opt/stack/data/nova/instances/_base/9916110004678060463128e2f14a55ac921f479c.part /opt/stack/data/nova/instances/_base/9916110004678060463128e2f14a55ac921f479c.converted 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Exit code: 1 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Stdout: u'' 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Stderr: u'qemu-img: error while reading sector 941568: Input/output error\n' 2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Logs: http://logs.openstack.org/85/156985/5/gate/check-heat-dsvm-functional-mysql/b21de47/logs/ http://logs.openstack.org/32/155732/3/check/check-heat-dsvm-functional-mysql/8fd1ae5/logs/ The image that we are using for this test is Fedora-x86_64-20-20140618-sda that is downloading during test-job from official repo. Do you have any ideas how to fix this? Is it possible that image has been corrupted during glance->nova transfer?
Hello guys! manager [-] [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Instance failed to spawn manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Traceback (most recent call last): manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/compute/ manager. py", line 2316, in _build_resources manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] yield resources manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/compute/ manager. py", line 2186, in _build_ and_run_ instance manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] flavor=flavor) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 2358, in spawn manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] admin_pass= admin_password) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ driver. py", line 2762, in _create_image manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] project_ id=instance. project_ id) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ imagebackend. py", line 230, in cache manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] *args, **kwargs) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ imagebackend. py", line 477, in create_image manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] prepare_ template( target= base, max_size=size, *args, **kwargs) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/lockutils. py", line 431, in inner manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] return f(*args, **kwargs) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ imagebackend. py", line 220, in fetch_func_sync manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] fetch_func( target= target, *args, **kwargs) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ libvirt/ utils.py" , line 488, in fetch_image manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] max_size=max_size) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ images. py", line 121, in fetch_to_raw manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] convert_ image(path_ tmp, staged, 'raw') manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/virt/ images. py", line 68, in convert_image manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] utils.execute(*cmd, run_as_ root=run_ as_root) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/opt/stack/ new/nova/ nova/utils. py", line 197, in execute manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] return processutils. execute( *cmd, **kwargs) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] File "/usr/local/ lib/python2. 7/dist- packages/ oslo_concurrenc y/processutils. py", line 228, in execute manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] cmd=sanitized_cmd) manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] ProcessExecutio nError: Unexpected error while running command. manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Command: qemu-img convert -O raw /opt/stack/ data/nova/ instances/ _base/991611000 4678060463128e2 f14a55ac921f479 c.part /opt/stack/ data/nova/ instances/ _base/991611000 4678060463128e2 f14a55ac921f479 c.converted manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Exit code: 1 manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Stdout: u'' manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] Stderr: u'qemu-img: error while reading sector 941568: Input/output error\n' manager [instance: 11e30652- 70a9-480a- 9b16-9026be2e35 92] logs.openstack. org/85/ 156985/ 5/gate/ check-heat- dsvm-functional -mysql/ b21de47/ logs/ logs.openstack. org/32/ 155732/ 3/check/ check-heat- dsvm-functional -mysql/ 8fd1ae5/ logs/ x86_64- 20-20140618- sda that is downloading during test-job from official repo.
We faced with the same issues today on gates:
2015-02-19 11:26:02.215 5467 ERROR nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.
Logs:
http://
http://
The image that we are using for this test is Fedora-
Do you have any ideas how to fix this?
Is it possible that image has been corrupted during glance->nova transfer?