test_attach_cloned_encrypted_volume fails in nova-ceph-multistore job permanently

Bug #1912607 reported by Balazs Gibizer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Critical
Lee Yarwood

Bug Description

The job is red since 2021.01.20 20:18 UTC[1]. The failure is:

tempest.exceptions.BuildErrorException: Server 79ce2743-1b98-4867-8396-5c68c25f2cf3 failed to build and is in ERROR status
Details: {'code': 500, 'created': '2021-01-20T22:28:59Z', 'message': 'Build of instance 79ce2743-1b98-4867-8396-5c68c25f2cf3 aborted: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format'}

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [None req-3a8ebd8e-b22a-446f-ad35-16b12e7a1182 tempest-TestEncryptedCinderVolumes-88193216 tempest-TestEncryptedCinderVolumes-88193216] [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Instance failed to spawn: nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Traceback (most recent call last):

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] yield resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/compute/manager.py", line 2384, in _build_and_run_instance

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.driver.spawn(context, instance, image_meta,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4061, in spawn

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] created_instance_dir, created_disks = self._create_image(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4457, in _create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] created_disks = self._create_and_inject_local_root(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4585, in _create_and_inject_local_root

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self._try_fetch_image_cache(backend, fetch_func, context,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 10164, in _try_fetch_image_cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] image.cache(fetch_func=fetch_func,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 273, in cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.create_image(fetch_func_sync, base, size,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 939, in create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] prepare_template(target=base, *args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py", line 360, in inner

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] return f(*args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 270, in fetch_func_sync

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] fetch_func(target=target, *args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4572, in clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] LOG.warning(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in __exit__

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.force_reraise()

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] six.reraise(self.type_, self.value, self.tb)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] raise value

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4564, in clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] backend.clone(context, disk_images['image_id'])

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 1039, in clone

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] raise exception.ImageUnacceptable(image_id=image_id_or_uri,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3]

Logstash signature: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20unacceptable%3A%20Image%20is%20not%20raw%20format'%5C%22

[1] https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova

Tags: gate-failure
Changed in nova:
importance: Undecided → Critical
tags: added: gate-failure
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :
Revision history for this message
Lee Yarwood (lyarwood) wrote :

61427 Jan 21 08:10:57.185061 ubuntu-focal-ovh-gra1-0022643495 nova-compute[94297]: WARNING nova.virt.libvirt.driver [None req-5443ffbb-803e-4e7e-86d0-e77a4eb650cf tempest-TestEncryptedCinderVolumes-1602703716 tempest-TestEncryptedCinderVolumes-1602703716] Image 6eb6b21b-cb90-47c7-81eb-6d2071c0f62c is not on my ceph and [workarounds]/never_download_image_if_on_rbd=True; refusing to fetch and upload.: nova.exception.ImageUnacceptable: Image 6eb6b21b-cb90-47c7-81eb-6d2071c0f62c is unacceptable: Image is not raw format

https://review.opendev.org/c/openstack/nova/+/771777 posted to skip it in this job

Changed in nova:
status: New → In Progress
assignee: nobody → Lee Yarwood (lyarwood)
Lee Yarwood (lyarwood)
Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 23.0.0.0rc1

This issue was fixed in the openstack/nova 23.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers