When creating an encrypted volume from an encrypted volume the cinder-volume manager code calls the driver to clone a volume multiple times if rekeying fails.
In method "_create_from_source_volume" from "cinder/volume/flows/manager/create_volume.py" we have:
model_update = self.driver.create_cloned_volume(volume, srcvol_ref)
if model_update is None:
model_update = {}
if volume.encryption_key_id is not None:
volume.update(model_update)
rekey_model_update = self._rekey_volume(context, volume)
model_update.update(rekey_model_update)
Which means that if "create_cloned_volume" succeeds and "_rekey_volume" fails, there will be no record that the volume cloning actually worked, and the flow will retry and will call "create_cloned_volume" again.
This is problematic because many drivers use the "volume.name" value (which uses the template and the ID) to name the volume in the backend, so they will fail on the second cloning because the volume to be created already exists.
For RBD this will show as the exception:
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager Traceback (most recent call last):
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 783, in create_cloned_volume
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager src_volume.create_snap(clone_snap)
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 193, in doit
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager result = proxy_call(self._autowrap, f, *args, **kwargs)
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 151, in proxy_call
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager rv = execute(f, *args, **kwargs)
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 132, in execute
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager six.reraise(c, e, tb)
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/usr/local/lib/python3.8/dist-packages/six.py", line 719, in reraise
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager raise value
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 86, in tworker
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager rv = meth(*args, **kwargs)
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager File "rbd.pyx", line 3514, in rbd.Image.create_snap
Jul 26 12:15:13 stack cinder-volume[1449504]: ERROR cinder.volume.manager rbd.ImageExists: [errno 17] RBD image already exists (error creating snapshot b'volume-a417e1d6-f538-4aba-abbf-69f751608507.clone_snap' from b'volume-12bdc0b3-4b3f-4471-97f8-b0ac93809cf4')
I've only seen this happen on an RBD deployment without the fix for https://bugs.launchpad.net/os-brick/+bug/1981455
