Cinder

Volume retype fails from volume type with encryption to volume type with different encryption

Bug #1689711 reported by Dinesh Gavand on 2017-05-10

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Cinder	Invalid	High	Vishakha Agarwal

Bug Description

Env:
One Devstack with stable/ocata build which has lvm as backend.

Test steps:
1. Create a volume type "LUKS" with encription using below command:
cinder encryption-type-create --cipher aes-xts-plain64 --key_size 256 --control_location front-end LUKS nova.volume.encryptors.luks.LuksEncryptor

2. Create another volume type "LUKS2" with encription using below command:
cinder encryption-type-create --cipher aes-cbc-essiv --key_size 256 --control_location front-end LUKS2 nova.volume.encryptors.luks.LuksEncryptor

3. Both the volume type must be configured with the same (lvmdriver-1) volume_backend_name in extra specs.

3. Create a volume <volume1> with volume type LUKS.

4. Retype above volume from LUKS to LUKS2.

Expect result:
The retype should succeed.

Actual result:
Retype failed. Observed below traces in c-vol.log

2017-05-05 00:54:59.756 TRACE cinder.volume.manager
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 2396, in retype
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server new_reservations, status_update)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self.force_reraise()
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 2392, in retype
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server new_type_id=new_type_id)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 2034, in migrate_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server volume.save()
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self.force_reraise()
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 2027, in migrate_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self._migrate_volume_generic(ctxt, volume, host, new_type_id)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1846, in _migrate_volume_generic
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server new_volume)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self.force_reraise()
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1824, in _migrate_volume_generic
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1705, in _copy_volume_data
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server attach_encryptor=attach_encryptor)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1655, in _attach_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self._detach_volume(ctxt, attach_info, volume, properties)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self.force_reraise()
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 1650, in _attach_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server encryption)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/utils.py", line 514, in brick_attach_volume_encryptor
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server encryptor.attach_volume(context, **encryption)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/os_brick/encryptors/luks.py", line 163, in attach_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server self._format_volume(passphrase, **kwargs)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/os_brick/encryptors/luks.py", line 88, in _format_volume
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server attempts=3)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/os_brick/executor.py", line 49, in _execute
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server result = self.__execute(*args, **kwargs)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/os_brick/privileged/rootwrap.py", line 57, in execute
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server return execute_root(*cmd, **kwargs)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 205, in _wrap
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server return self.channel.remote_call(name, args, kwargs)
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 187, in remote_call
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server raise exc_type(*result[2])
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command.
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Command: cryptsetup --batch-mode luksFormat --key-file=- --cipher aes-cbc-essiv --key-size 256 /dev/sdb
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Exit code: 1
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Stdout: u''
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Stderr: u'device-mapper: reload ioctl on failed: Invalid argument\nFailed to setup dm-crypt key mapping for device /dev/sdb.\nCheck that kernel supports aes-cbc-essiv cipher (check syslog for more info).\n'
2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server
stack@cld6b14:~/devstack$

See original description

Tags:

Revision history for this message

Dinesh Gavand (dinesh-gavand) wrote on 2017-05-10:

Retype commands output Edit (5.0 KiB, text/plain)

Dinesh Gavand (dinesh-gavand) on 2017-05-10

description:

updated

Jay Bryant (jsbryant) on 2017-05-11

Changed in cinder:
importance:	Undecided → High
tags:	added: bugsmash

wanghao (wanghao749) on 2017-05-17

Changed in cinder:
assignee:	nobody → wanghao (wanghao749)

Revision history for this message

Alan Bishop (alan-bishop) wrote on 2018-01-04:

This message suggests an unrelated root problem:

2017-05-05 00:54:59.931 TRACE oslo_messaging.rpc.server Stderr: u'device-mapper: reload ioctl on failed: Invalid argument\nFailed to setup dm-crypt key mapping for device /dev/sdb.\nCheck that kernel supports aes-cbc-essiv cipher (check syslog for more info).\n'

Can you create a LUKS2 volume?

Eric Harney (eharney) on 2018-03-06

Changed in cinder:
assignee:	wanghao (wanghao749) → Eric Harney (eharney)

Vishakha Agarwal (vishakha.agarwal) on 2018-06-08

Changed in cinder:
assignee:	Eric Harney (eharney) → Vishakha Agarwal (vishakha.agarwal)

Revision history for this message

Vishakha Agarwal (vishakha.agarwal) wrote on 2018-06-18:

Tried with creating the volume with type LUKS2 and migrated to LUKS which worked fine. The bugs looks invalid as no problem from cinder side because cipher 'aes-cbc-essiv sha256' is sometimes not compatible with kernel.

Some of the open issues for this cipher in linux-
https://github.com/raspberrypi/linux/issues/1312
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=344867