Cannot attach an encrypted volume to an instance

[Yaroslav Lobankov]

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  api: "1.0"
  build_number: "112"
  build_id: "2015-02-10_22-54-44"
  nailgun_sha: "e6d6cd143b3c52713e301b721807ac199482680b"
  python-fuelclient_sha: "e2e556f6654f9ba280ae43e1cdd0c12d1825bb43"
  astute_sha: "214604ee6ba164a112cef6775f428de68372eff3"
  fuellib_sha: "6435016559edf22bd790c6d2f627272de447e8d7"
  ostf_sha: "6c046b69d29021524906109f18092363505ee222"
  fuelmain_sha: "a6e7cc1b8998dd86f627ec4b556310ae1fc1d31e"

ENVIRONMENT:
HA mode, Ubuntu, Neutron with GRE segmentation, Cinder LVM, 1 controller, 1 (compute + Cinder)

HOW TO REPRODUCE:
Log into the controller node.
Execute the following commands:
1. Source your admin credentials.
$ . openrc

2. Create a volume type.
$ cinder type-create some-type

3. Mark the volume type as encrypted and provide the necessary details.
$ cinder encryption-type-create --cipher aes-xts-plain64 --key_size 512 --control_location front-end some-type nova.volume.encryptors.luks.LuksEncryptor

4. Create an encrypted 1 GB test volume.
$ cinder create --display-name 'encrypted volume' --volume-type some-type 1

5. Create a VM.
$ nova boot --flavor m1.tiny --image TestVM --nic net-id=<private_net_id> vm-test

6. Attach the created volume to your VM.
$ nova volume-attach vm-test <volume_id> /dev/vdb

EXPECTED RESULT:
The volume has been successfully attached.

ACTUAL RESULT:
The volume has not been attached.

In nova-compute.log I can see as follows:
2015-02-11T15:20:38.065465+00:00 err: Failed to attach volume at mountpoint: /dev/vdb
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] Traceback (most recent call last):
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1398, in attach_volume
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryption)
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1345, in _get_volume_encryptor
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] **encryption)
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/__init__.py", line 32, in get_volume_encryptor
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryptor = nop.NoOpEncryptor(connection_info, **kwargs)
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/nop.py", line 33, in __init__
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] super(NoOpEncryptor, self).__init__(connection_info, **kwargs)
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/base.py", line 38, in __init__
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] self._key_manager = keymgr.API()
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/__init__.py", line 34, in API
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] return cls()
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/conf_key_mgr.py", line 58, in __init__
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] raise ValueError(_('keymgr.fixed_key not defined'))
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] ValueError: keymgr.fixed_key not defined
2015-02-11 15:20:38.016 22407 TRACE nova.virt.libvirt.driver [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097]
2015-02-11T15:20:38.065465+00:00 debug: Created new semaphore "connect_volume"
2015-02-11T15:20:38.065465+00:00 debug: Acquired semaphore "connect_volume"
2015-02-11T15:20:38.065465+00:00 debug: Got semaphore / lock "disconnect_volume"
2015-02-11T15:20:38.065465+00:00 debug: Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-a303365a-b2d4-4308-bfab-6562964839d0 -p 192.168.1.3:3260 --op update -n node.startup -v manual
2015-02-11T15:20:38.065465+00:00 debug: Result was 0
2015-02-11T15:20:38.065465+00:00 debug: iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr=
2015-02-11T15:20:38.066099+00:00 debug: Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-a303365a-b2d4-4308-bfab-6562964839d0 -p 192.168.1.3:3260 --logout
2015-02-11T15:20:38.602980+00:00 debug: Result was 0
2015-02-11T15:20:38.603478+00:00 debug: iscsiadm ('--logout',): stdout=Logging out of session [sid: 1, target: iqn.2010-10.org.openstack:volume-a303365a-b2d4-4308-bfab-6562964839d0, portal: 192.168.1.3,3260]
Logout of [sid: 1, target: iqn.2010-10.org.openstack:volume-a303365a-b2d4-4308-bfab-6562964839d0, portal: 192.168.1.3,3260]: successful
 stderr=
2015-02-11T15:20:38.604057+00:00 debug: Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-a303365a-b2d4-4308-bfab-6562964839d0 -p 192.168.1.3:3260 --op delete
2015-02-11T15:20:38.638221+00:00 debug: Result was 0
2015-02-11T15:20:38.638739+00:00 debug: iscsiadm ('--op', 'delete'): stdout= stderr=
2015-02-11T15:20:38.639259+00:00 debug: Releasing semaphore "connect_volume"
2015-02-11T15:20:38.640325+00:00 debug: Semaphore / lock released "disconnect_volume"
2015-02-11T15:20:38.640325+00:00 err: Driver failed to attach volume a303365a-b2d4-4308-bfab-6562964839d0 at /dev/vdb
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] Traceback (most recent call last):
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 252, in attach
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] device_type=self['device_type'], encryption=encryption)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1412, in attach_volume
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] self._disconnect_volume(connection_info, disk_dev)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1398, in attach_volume
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryption)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1345, in _get_volume_encryptor
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] **encryption)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/__init__.py", line 32, in get_volume_encryptor
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryptor = nop.NoOpEncryptor(connection_info, **kwargs)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/nop.py", line 33, in __init__
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] super(NoOpEncryptor, self).__init__(connection_info, **kwargs)
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/base.py", line 38, in __init__
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] self._key_manager = keymgr.API()
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/__init__.py", line 34, in API
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] return cls()
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/conf_key_mgr.py", line 58, in __init__
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] raise ValueError(_('keymgr.fixed_key not defined'))
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] ValueError: keymgr.fixed_key not defined
2015-02-11 15:20:38.632 22407 TRACE nova.virt.block_device [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097]
2015-02-11T15:20:38.640703+00:00 debug: Cinderclient connection created using URL: http://192.168.0.2:8776/v1/a57de80ea04f430b9483b764dac75bcd
2015-02-11T15:20:38.866546+00:00 err: Failed to attach a303365a-b2d4-4308-bfab-6562964839d0 at /dev/vdb
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] Traceback (most recent call last):
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4547, in _attach_volume
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] do_check_attach=False, do_driver_attach=True)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 46, in wrapped
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] ret_val = method(obj, context, *args, **kwargs)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 261, in attach
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] connector)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 252, in attach
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] device_type=self['device_type'], encryption=encryption)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1412, in attach_volume
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] self._disconnect_volume(connection_info, disk_dev)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1398, in attach_volume
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryption)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1345, in _get_volume_encryptor
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] **encryption)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/__init__.py", line 32, in get_volume_encryptor
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] encryptor = nop.NoOpEncryptor(connection_info, **kwargs)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/nop.py", line 33, in __init__
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] super(NoOpEncryptor, self).__init__(connection_info, **kwargs)
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/base.py", line 38, in __init__
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] self._key_manager = keymgr.API()
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/__init__.py", line 34, in API
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] return cls()
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] File "/usr/lib/python2.7/dist-packages/nova/keymgr/conf_key_mgr.py", line 58, in __init__
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] raise ValueError(_('keymgr.fixed_key not defined'))
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097] ValueError: keymgr.fixed_key not defined
2015-02-11 15:20:38.754 22407 TRACE nova.compute.manager [instance: 0405da0c-94fa-49c9-b0fa-7cd2d53eb097]
2015-02-11T15:20:38.866546+00:00 debug: Cinderclient connection created using URL: http://192.168.0.2:8776/v1/a57de80ea04f430b9483b764dac75bcd
2015-02-11T15:20:38.881867+00:00 debug: Releasing semaphore "0405da0c-94fa-49c9-b0fa-7cd2d53eb097"
2015-02-11T15:20:38.882320+00:00 debug: Semaphore / lock released "do_attach_volume"
2015-02-11T15:20:39.013953+00:00 debug: Created new semaphore "compute_resources"
2015-02-11T15:20:39.014490+00:00 debug: Acquired semaphore "compute_resources"
2015-02-11T15:20:39.014911+00:00 debug: Got semaphore / lock "update_usage"
2015-02-11T15:20:39.056990+00:00 info: Compute_service record updated for ('node-2', 'node-2.domain.tld')
2015-02-11T15:20:39.057507+00:00 debug: Releasing semaphore "compute_resources"
2015-02-11T15:20:39.057907+00:00 debug: Semaphore / lock released "update_usage"
2015-02-11T15:20:39.060509+00:00 err: Exception during message handling: keymgr.fixed_key not defined
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 137, in _dispatch_and_reply
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 180, in _dispatch
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 126, in _do_dispatch
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 414, in decorated_function
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher payload)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 298, in decorated_function
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher pass
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 284, in decorated_function
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 326, in decorated_function
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 314, in decorated_function
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4537, in attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher do_attach_volume(context, instance, driver_bdm)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 272, in inner
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return f(*args, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4535, in do_attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher bdm.destroy(context)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4532, in do_attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return self._attach_volume(context, instance, driver_bdm)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4555, in _attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, bdm.volume_id)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4547, in _attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher do_check_attach=False, do_driver_attach=True)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 46, in wrapped
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher ret_val = method(obj, context, *args, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 261, in attach
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher connector)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/block_device.py", line 252, in attach
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher device_type=self['device_type'], encryption=encryption)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1412, in attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher self._disconnect_volume(connection_info, disk_dev)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1398, in attach_volume
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher encryption)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1345, in _get_volume_encryptor
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher **encryption)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/__init__.py", line 32, in get_volume_encryptor
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher encryptor = nop.NoOpEncryptor(connection_info, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/nop.py", line 33, in __init__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher super(NoOpEncryptor, self).__init__(connection_info, **kwargs)
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/volume/encryptors/base.py", line 38, in __init__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher self._key_manager = keymgr.API()
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/keymgr/__init__.py", line 34, in API
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher return cls()
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/keymgr/conf_key_mgr.py", line 58, in __init__
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher raise ValueError(_('keymgr.fixed_key not defined'))
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher ValueError: keymgr.fixed_key not defined
2015-02-11 15:20:39.051 22407 TRACE oslo.messaging.rpc.dispatcher

The diagnostic snapshot is attached.

[Eugeniya Kudryashova (ekudryashova)]

Looks like this needs additional configuration:

1) Install cryptsetup

2) Add fixed_keys to nova and cinder configuration files, as mentioned here: http://docs.openstack.org/trunk/config-reference/content/section_initial-configuration.html

3) Restart nova-compure and cinder-volume services

On my env after performing this steps everything works OK

[Roman Podoliaka (rpodolyaka)]

This provides very limited security, so this is probably Medium rather than High.

I'm not sure we want to fix this in 6.0.1 either, as this requires changes to Nailgun/fuel-library/cinder-volume packages. Eugeniya provided a workaround, which we could document in release notes.

[Yaroslav Lobankov (ylobankov)]

Roman, will it take much effort to make corresponding changes in fuel-library? This issue affects two scenario Tempest tests. It would be nice to have this fix in 6.0.1 to avoid Tempest fails.

[Roman Podoliaka (rpodolyaka)]

Yaroslav, it must not, but the fact it requires changes to 3 different projects (make Nailgun generate the fixed key, make fuel-library puppet manifests set the fixed_key value in cinder.conf and nova.conf, add dependency on cryptsetup to cinder-volume) make me think it's not something we want to do in a maintenance release, given the fact that it doesn't provide 'real' security value for end users.

At the same time, 6.1 is a feature release and we still have time to fix such bugs.

[Yaroslav Lobankov (ylobankov)]

Thanks, Roman. Got it.

[Yaroslav Lobankov (ylobankov)]

The issue was not reproduced on

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  api: "1.0"
  build_number: "157"
  build_id: "2015-03-03_10-40-31"
  nailgun_sha: "1b0210e35de6acc9b7c8b9b39d0d477692cd8c36"
  python-fuelclient_sha: "3ebfa9c14a192d0298ff787526bf990055a23694"
  astute_sha: "1be5b9b827f512d740fe907c7ff72486d4030938"
  fuellib_sha: "31b6b2c2a2ea4daa0c8b8a6fe7457a478a08b008"
  ostf_sha: "b47bb2ad09e60a25040e21f250018e256f3bebdd"
  fuelmain_sha: "baf24424a4e056c6753913de5f8c94851903f718"

Ivan, could you please provide corresponding links to patches that fix the issue?

[OSCI Robot (oscirobot)]

package cinder has been built for project openstack/cinder
Package version == 2014.2.1, package release == fuel6.1.mira1.git.10db3a5.3db1f4a

Changeset: https://review.fuel-infra.org/3638
project: openstack-build/cinder-build
branch: openstack-ci/fuel-6.1/2014.2
author: Max Yatsenko
committer: Max Yatsenko
subject: Add \"cryptsetup\" requirements to deb, rpm specs.
status: patchset-created

Files placed on repository:

NOTE: Changeset is not merged, created temporary package repository.
 repository URL: /centos-fuel-6.1-stable-3638/

[OSCI Robot (oscirobot)]

package cinder has been built for project openstack/cinder
Package version == 2014.2.2, package release == fuel6.1~mira2+git.76b9ff4.020ca1f

Changeset: https://review.fuel-infra.org/4707
project: openstack-build/cinder-build
branch: openstack-ci/fuel-6.1/2014.2
author: Max Yatsenko
committer: Max Yatsenko
subject: Add \"cryptsetup\" requirements to /trusty/debian/control .
status: patchset-created

Files placed on repository:

NOTE: Changeset is not merged, created temporary package repository.
 repository URL: /trusty-fuel-6.1-stable-4707/

[OSCI Robot (oscirobot)]

package cinder has been built for project openstack/cinder
Package version == 2014.2.2, package release == fuel6.1~mira2+git.76b9ff4.5ecf076

Changeset: https://review.fuel-infra.org/4707
project: openstack-build/cinder-build
branch: openstack-ci/fuel-6.1/2014.2
author: Max Yatsenko
committer: Max Yatsenko
subject: Add \"cryptsetup\" requirements to /trusty/debian/control.
status: patchset-created

Files placed on repository:

NOTE: Changeset is not merged, created temporary package repository.
 repository URL: /trusty-fuel-6.1-stable-4707/

[OSCI Robot (oscirobot)]

package cinder has been built for project openstack/cinder
Package version == 2014.2.2, package release == fuel6.1~mira2

Changeset: https://review.fuel-infra.org/4707
project: openstack-build/cinder-build
branch: openstack-ci/fuel-6.1/2014.2
author: Max Yatsenko
committer: Max Yatsenko
subject: Add \"cryptsetup\" requirements to /trusty/debian/control.
status: change-merged

Files placed on repository:

Changeset merged. Package placed on primary repository
 repository URL: /trusty-fuel-6.1-stable/