Comment 7 for bug 1303802

Revision history for this message
Kairat Kushaev (kkushaev) wrote :

Hello guys!
We faced with the same issues today on gates:
2015-02-19 11:26:02.215 5467 ERROR nova.compute.manager [-] [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Instance failed to spawn
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Traceback (most recent call last):
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/compute/manager.py", line 2316, in _build_resources
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] yield resources
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/compute/manager.py", line 2186, in _build_and_run_instance
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] flavor=flavor)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2358, in spawn
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] admin_pass=admin_password)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2762, in _create_image
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] project_id=instance.project_id)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 230, in cache
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] *args, **kwargs)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 477, in create_image
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] prepare_template(target=base, max_size=size, *args, **kwargs)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 431, in inner
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] return f(*args, **kwargs)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 220, in fetch_func_sync
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] fetch_func(target=target, *args, **kwargs)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/libvirt/utils.py", line 488, in fetch_image
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] max_size=max_size)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/images.py", line 121, in fetch_to_raw
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] convert_image(path_tmp, staged, 'raw')
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/virt/images.py", line 68, in convert_image
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] utils.execute(*cmd, run_as_root=run_as_root)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/opt/stack/new/nova/nova/utils.py", line 197, in execute
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] return processutils.execute(*cmd, **kwargs)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 228, in execute
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] cmd=sanitized_cmd)
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] ProcessExecutionError: Unexpected error while running command.
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Command: qemu-img convert -O raw /opt/stack/data/nova/instances/_base/9916110004678060463128e2f14a55ac921f479c.part /opt/stack/data/nova/instances/_base/9916110004678060463128e2f14a55ac921f479c.converted
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Exit code: 1
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Stdout: u''
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592] Stderr: u'qemu-img: error while reading sector 941568: Input/output error\n'
2015-02-19 11:26:02.215 5467 TRACE nova.compute.manager [instance: 11e30652-70a9-480a-9b16-9026be2e3592]
Logs:
http://logs.openstack.org/85/156985/5/gate/check-heat-dsvm-functional-mysql/b21de47/logs/
http://logs.openstack.org/32/155732/3/check/check-heat-dsvm-functional-mysql/8fd1ae5/logs/
The image that we are using for this test is Fedora-x86_64-20-20140618-sda that is downloading during test-job from official repo.
Do you have any ideas how to fix this?
Is it possible that image has been corrupted during glance->nova transfer?