Gate failure: failing glance and cinder gate

[Rajat Dhasmana]

Glance and cinder gate is failing with the following error trace in cinder-api

https://ab2eec104c374432eb2d-c80779e78a1a20b721125eb9333742c5.ssl.cf1.rackcdn.com/802653/2/check/grenade/7ca0949/controller/logs/screen-c-api.txt

Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: WARNING cinder.volume.api [None req-bc37d60e-ba94-4ebc-94c4-17b448829303 cinder_grenade cinder_grenade] Task 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create' (83435913-f461-43fe-9bb7-b66641ba30d9) transitioned into state 'FAILURE' from state 'RUNNING'
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: 1 predecessors (most recent first):
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: Flow 'volume_create_api': cinder.exception.InvalidVolumeType: Invalid encryption spec
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api Traceback (most recent call last):
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api result = task.execute(**arguments)
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/flows/api/create_volume.py", line 422, in execute
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api encryption_key_id = self._get_encryption_key_id(
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/flows/api/create_volume.py", line 344, in _get_encryption_key_id
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api encryption_key_id = volume_utils.create_encryption_key(
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/volume_utils.py", line 937, in create_encryption_key
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api raise exception.InvalidVolumeType(
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api cinder.exception.InvalidVolumeType: Invalid encryption spec
Aug 03 03:47:14.160162 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: ERROR cinder.volume.api
Aug 03 03:47:14.162983 ubuntu-focal-inap-mtl01-0025737114 <email address hidden>[87348]: DEBUG cinder.volume.api [None req-bc37d60e-ba94-4ebc-94c4-17b448829303 cinder_grenade cinder_grenade] Task 'cinder.volume.flows.api.create_volume.ExtractVolumeRequestTask;volume:create' (83435913-f461-43fe-9bb7-b66641ba30d9) transitioned into state 'REVERTING' from state 'FAILURE' {{(pid=87348) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}

[Lee Yarwood (lyarwood)]

Yeah also seeing this in nova-grenade-multinode, appears introduced by Ia4b1c303a057fe70cf88bdbbebc1d4f62474f011?

https://zuul.opendev.org/t/openstack/build/a566aad8ef1146c598ea5b7b06de4644/log/controller/logs/grenade.sh_log.txt#1294

2021-08-03 10:03:02.287 | ++ /opt/stack/new/grenade/projects/70_cinder/resources.sh:create:167 : openstack volume create --size 1 cinder_grenade_vol3 --type cinder_grenade_encrypted_type -f shell
2021-08-03 10:03:03.803 | Invalid encryption spec (HTTP 400) (Request-ID: req-4ea9c9eb-8c06-45d9-b3a8-bc63dab4f688)

https://zuul.opendev.org/t/openstack/build/a566aad8ef1146c598ea5b7b06de4644/log/controller/logs/screen-c-api.txt

Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: Flow 'volume_create_api': cinder.exception.InvalidVolumeType: Invalid encryption spec
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api Traceback (most recent call last):
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api result = task.execute(**arguments)
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/flows/api/create_volume.py", line 422, in execute
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api encryption_key_id = self._get_encryption_key_id(
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/flows/api/create_volume.py", line 344, in _get_encryption_key_id
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api encryption_key_id = volume_utils.create_encryption_key(
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api File "/opt/stack/old/cinder/cinder/volume/volume_utils.py", line 937, in create_encryption_key
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api raise exception.InvalidVolumeType(
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api cinder.exception.InvalidVolumeType: Invalid encryption spec
Aug 03 10:03:03.793591 ubuntu-focal-ovh-gra1-0025742596 <email address hidden>[87790]: ERROR cinder.volume.api

[Lee Yarwood (lyarwood)]

https://review.opendev.org/c/openstack/grenade/+/803317

[Ghanshyam Mann (ghanshyammann)]

the cinder change bug ref https://bugs.launchpad.net/cinder/+bug/1926630

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/grenade/+/803317
Committed: https://opendev.org/openstack/grenade/commit/d61a95325b3d052216aa13931c7a6d4a132a5843
Submitter: "Zuul (22348)"
Branch: master

commit d61a95325b3d052216aa13931c7a6d4a132a5843
Author: Rajat Dhasmana <email address hidden>
Date: Fri Jul 30 06:32:26 2021 -0400

    Unblock Gate: create encryption type with right params

    Recently we added a check in cinder for cipher algorithm[1]
    Currently grenade is not passing cipher when creating an
    encrypted type, hence grenade job is failing in glance and cinder gate.
    This patch addresses the same by passing the cipher and other
    properties.

    [1] https://review.opendev.org/c/openstack/cinder/+/800268

    Closes-Bug: #1938763

    Change-Id: I1eb352deca11916f4db1010a50d15a83e33a61bb

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/grenade/+/803431

[OpenStack Infra (hudson-openstack)]

Change abandoned by "yatin <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/grenade/+/803431
Reason: Not needed, if cinder team plans backport for 800268 then it can be re picked.