when using ceph as a backend for cinder I am running into an issue where volume creation based on an image source fails.
empty volume creation works fine, it's only volume creation based on an image source that fails.
Here is the relevant excerpt from /var/log/cinder/cinder-volume.log (within the cinder-volume container):
2016-05-10 14:38:30.636 10357 INFO cinder.volume.flows.manager.create_volume [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Volume 2ffb89d3-4048-4927-8795-4d13c825cf84: being created as image with specification: {'status': u'creating', 'image_location': (u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', None), 'volume_size': 20, 'volume_name': 'volume-2ffb89d3-4048-4927-8795-4d13c825cf84', 'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f648f89a890>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'trusty64', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2016, 5, 5, 21, 47, 13, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2016, 5, 5, 21, 47, 20, tzinfo=<iso8601.Utc>), u'visibility': u'private', 'properties': {}, u'owner': u'a0adde92c1b74a67ad8c640168c28632', u'protected': False, u'id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'file': u'/v2/images/28bf4043-d449-416a-a994-234aa613ed7c/file', u'checksum': u'e05b2f067604a1b4d0e9aa430b4e8721', u'min_disk': 0, u'direct_url': u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', u'min_ram': 0, u'size': 259457536}}
2016-05-10 14:38:33.648 10357 INFO cinder.image.image_utils [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Image download 247.00 MB at 82.89 MB/s
2016-05-10 14:38:33.769 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1e8040cc-80f2-40a4-ad4f-a15ab329ba35) transitioned into state 'FAILURE' from state 'RUNNING'
5 predecessors (most recent first):
Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume_ref': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,consistencygroup=<?>,consistencygroup_id=None,created_at=2016-05-10T14:38:25Z,deleted=False,deleted_at=None,display_description='',display_name='test2',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,host='hostname-cinder-volumes-container-16ac4806@rbd#rbd',id=2ffb89d3-4048-4927-8795-4d13c825cf84,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id=a0adde92c1b74a67ad8c640168c28632,provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2016-05-10T14:38:29Z,size=20,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2016-05-10T14:38:30Z,user_id=b8e11a21c5f84dd2b629d480c00b9953,volume_attachment=<?>,volume_type=VolumeType(3aa4c5f4-720b-4566-af2e-71858c355504),volume_type_id=3aa4c5f4-720b-4566-af2e-71858c355504), 'context': <cinder.context.RequestContext object at 0x7f6493d02d10>}, 'provides': None}
|__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume_ref': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,consistencygroup=<?>,consistencygroup_id=None,created_at=2016-05-10T14:38:25Z,deleted=False,deleted_at=None,display_description='',display_name='test2',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,host='hostname-cinder-volumes-container-16ac4806@rbd#rbd',id=2ffb89d3-4048-4927-8795-4d13c825cf84,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id=a0adde92c1b74a67ad8c640168c28632,provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2016-05-10T14:38:29Z,size=20,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2016-05-10T14:38:30Z,user_id=b8e11a21c5f84dd2b629d480c00b9953,volume_attachment=<?>,volume_type=VolumeType(3aa4c5f4-720b-4566-af2e-71858c355504),volume_type_id=3aa4c5f4-720b-4566-af2e-71858c355504), 'request_spec': {u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'snapshot_id': None, u'display_name': u'test2', u'availability_zone': u'nova', u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'multiattach': False, u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'source_replicaid': None, u'source_volid': None, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20, u'metadata': {}}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'ec2_id': None, u'replication_status': u'disabled', u'snapshot_id': None, u'replication_extended_status': None, u'deleted_at': None, u'id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'size': 20, u'display_name': u'test2', u'display_description': u'', u'name_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'volume_admin_metadata': [], u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'launched_at': None, u'metadata': {}, u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'name': u'volume-2ffb89d3-4048-4927-8795-4d13c825cf84', u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'bootable': False, u'created_at': u'2016-05-10T14:38:25.231113', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'scheduled_at': None}, u'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': {u'name': u'rbd', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2016-05-05T21:15:45.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'rbd'}, u'is_public': True, u'deleted_at': None, u'id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'description': None}, u'volume_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'resource_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'snapshot_id': None, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'availability_zone': u'nova', u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'multiattach': False, u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'source_replicaid': None, u'source_volid': None, u'display_name': u'test2', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20, u'metadata': {}}}, 'context': <cinder.context.RequestContext object at 0x7f6493d02d10>}, 'provides': {'status': u'creating', 'image_location': (u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', None), 'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', 'volume_id': '2ffb89d3-4048-4927-8795-4d13c825cf84', 'volume_size': 20, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f648f89a890>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'trusty64', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2016, 5, 5, 21, 47, 13, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2016, 5, 5, 21, 47, 20, tzinfo=<iso8601.Utc>), u'visibility': u'private', 'properties': {}, u'owner': u'a0adde92c1b74a67ad8c640168c28632', u'protected': False, u'id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'file': u'/v2/images/28bf4043-d449-416a-a994-234aa613ed7c/file', u'checksum': u'e05b2f067604a1b4d0e9aa430b4e8721', u'min_disk': 0, u'direct_url': u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', u'min_ram': 0, u'size': 259457536}, 'volume_name': 'volume-2ffb89d3-4048-4927-8795-4d13c825cf84', 'type': 'image'}}
|__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume_ref': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,consistencygroup=<?>,consistencygroup_id=None,created_at=2016-05-10T14:38:25Z,deleted=False,deleted_at=None,display_description='',display_name='test2',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,host='hostname-cinder-volumes-container-16ac4806@rbd#rbd',id=2ffb89d3-4048-4927-8795-4d13c825cf84,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id=a0adde92c1b74a67ad8c640168c28632,provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2016-05-10T14:38:29Z,size=20,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2016-05-10T14:38:30Z,user_id=b8e11a21c5f84dd2b629d480c00b9953,volume_attachment=<?>,volume_type=VolumeType(3aa4c5f4-720b-4566-af2e-71858c355504),volume_type_id=3aa4c5f4-720b-4566-af2e-71858c355504), 'filter_properties': {u'config_options': {}, u'request_spec': {u'volume_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'display_name': u'test2', u'multiattach': False, u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'availability_zone': u'nova', u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'source_replicaid': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'snapshot_id': None, u'source_volid': None, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'ec2_id': None, u'replication_status': u'disabled', u'snapshot_id': None, u'replication_extended_status': None, u'deleted_at': None, u'id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'size': 20, u'display_name': u'test2', u'display_description': u'', u'name_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'volume_admin_metadata': [], u'encryption_key_id': None, u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'launched_at': None, u'metadata': {}, u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'name': u'volume-2ffb89d3-4048-4927-8795-4d13c825cf84', u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'bootable': False, u'created_at': u'2016-05-10T14:38:25.231113', u'attach_status': u'detached', u'_name_id': None, u'volume_metadata': [], u'replication_driver_data': None, u'scheduled_at': None}, u'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'consistencygroup_id': None, u'volume_type': {u'name': u'rbd', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2016-05-05T21:15:45.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'rbd'}, u'is_public': True, u'deleted_at': None, u'id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'description': None}, u'snapshot_id': None, u'resource_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'multiattach': False, u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'availability_zone': u'nova', u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'source_replicaid': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'snapshot_id': None, u'source_volid': None, u'display_name': u'test2', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20}}, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'availability_zone': u'nova', u'volume_type': {u'name': u'rbd', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2016-05-05T21:15:45.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'rbd'}, u'is_public': True, u'deleted_at': None, u'id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'description': None}, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'hosts': [u'hostname@rbd#rbd', u'hostname-cinder-volumes-container-16ac4806@rbd#rbd']}, u'metadata': {}, u'resource_type': {u'name': u'rbd', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2016-05-05T21:15:45.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'rbd'}, u'is_public': True, u'deleted_at': None, u'id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'description': None}, u'size': 20}, 'context': <cinder.context.RequestContext object at 0x7f6493d02d10>, 'request_spec': {u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'snapshot_id': None, u'display_name': u'test2', u'availability_zone': u'nova', u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'multiattach': False, u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'source_replicaid': None, u'source_volid': None, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20, u'metadata': {}}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'ec2_id': None, u'replication_status': u'disabled', u'snapshot_id': None, u'replication_extended_status': None, u'deleted_at': None, u'id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'size': 20, u'display_name': u'test2', u'display_description': u'', u'name_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'volume_admin_metadata': [], u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'launched_at': None, u'metadata': {}, u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'name': u'volume-2ffb89d3-4048-4927-8795-4d13c825cf84', u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'bootable': False, u'created_at': u'2016-05-10T14:38:25.231113', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'scheduled_at': None}, u'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': {u'name': u'rbd', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2016-05-05T21:15:45.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'rbd'}, u'is_public': True, u'deleted_at': None, u'id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'description': None}, u'volume_id': u'2ffb89d3-4048-4927-8795-4d13c825cf84', u'resource_properties': {u'status': u'creating', u'volume_type_id': u'3aa4c5f4-720b-4566-af2e-71858c355504', u'snapshot_id': None, u'user_id': u'b8e11a21c5f84dd2b629d480c00b9953', u'availability_zone': u'nova', u'reservations': [u'296cb2cb-3aad-4bee-87ad-8d0faf1b0ce8', u'8da597f1-adad-4ca5-b63a-d7e513ab4834', u'e29c90da-a941-4ab5-9dff-a420005ba86d', u'2ac1317f-f6fc-4948-8358-54d3db86503b'], u'multiattach': False, u'attach_status': u'detached', u'display_description': u'', u'cgsnapshot_id': None, u'qos_specs': None, u'encryption_key_id': None, u'replication_status': u'disabled', u'source_replicaid': None, u'source_volid': None, u'display_name': u'test2', u'project_id': u'a0adde92c1b74a67ad8c640168c28632', u'consistencygroup_id': None, u'size': 20, u'metadata': {}}}}, 'provides': None}
|__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f6493d02d10>, 'volume_id': '2ffb89d3-4048-4927-8795-4d13c825cf84'}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,consistencygroup=<?>,consistencygroup_id=None,created_at=2016-05-10T14:38:25Z,deleted=False,deleted_at=None,display_description='',display_name='test2',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,host='hostname-cinder-volumes-container-16ac4806@rbd#rbd',id=2ffb89d3-4048-4927-8795-4d13c825cf84,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id=a0adde92c1b74a67ad8c640168c28632,provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status='disabled',scheduled_at=2016-05-10T14:38:29Z,size=20,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2016-05-10T14:38:30Z,user_id=b8e11a21c5f84dd2b629d480c00b9953,volume_attachment=<?>,volume_type=VolumeType(3aa4c5f4-720b-4566-af2e-71858c355504),volume_type_id=3aa4c5f4-720b-4566-af2e-71858c355504)}
|__Flow 'volume_create_manager'
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager Traceback (most recent call last):
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager result = task.execute(**arguments)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 833, in execute
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager **volume_spec)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 741, in _create_from_image
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager image_service, context, image_id) as tmp_image:
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager return self.gen.next()
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 533, in fetch
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager fetch_verify_image(context, image_service, image_id, tmp)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 200, in fetch_verify_image
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager data = qemu_img_info(dest, run_as_root=run_as_root)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 63, in qemu_img_info
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager out, _err = utils.execute(*cmd, run_as_root=run_as_root)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/utils.py", line 148, in execute
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 371, in execute
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager cmd=sanitized_cmd)
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager ProcessExecutionError: Unexpected error while running command.
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/conversion/tmpUXabEL
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager Exit code: 96
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager Stdout: u''
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager Stderr: u'/openstack/venvs/cinder-13.0.1/bin/cinder-rootwrap: Executable not found: qemu-img (filter match = qemu-img)\n'
2016-05-10 14:38:33.769 10357 ERROR cinder.volume.manager
2016-05-10 14:38:33.777 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1e8040cc-80f2-40a4-ad4f-a15ab329ba35) transitioned into state 'REVERTED' from state 'REVERTING'
2016-05-10 14:38:33.780 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f9afed83-510a-48a5-8e7e-04cdad871558) transitioned into state 'REVERTED' from state 'REVERTING'
2016-05-10 14:38:33.783 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aa307174-5ff2-465e-88e3-0bc177547115) transitioned into state 'REVERTED' from state 'REVERTING'
2016-05-10 14:38:33.854 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (536bf26f-e4a7-4445-ba92-505729069c90) transitioned into state 'REVERTED' from state 'REVERTING'
2016-05-10 14:38:33.859 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7ccb801-b137-4a9a-9d2f-faa8980c2571) transitioned into state 'REVERTED' from state 'REVERTING'
2016-05-10 14:38:33.863 10357 WARNING cinder.volume.manager [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Flow 'volume_create_manager' (6493c410-18c7-46c1-9840-af701b2ff554) transitioned into state 'REVERTED' from state 'RUNNING'
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher [req-a13d56d2-ca85-4fba-a164-b1e5ce422e6f b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Exception during message handling: Unexpected error while running command.
Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/conversion/tmpUXabEL
Exit code: 96
Stdout: u''
Stderr: u'/openstack/venvs/cinder-13.0.1/bin/cinder-rootwrap: Executable not found: qemu-img (filter match = qemu-img)\n'
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/manager.py", line 630, in create_volume
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher _run_flow()
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/manager.py", line 617, in _run_flow
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher flow_engine.run()
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 230, in run
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher for _state in self.run_iter(timeout=timeout):
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 308, in run_iter
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher failure.Failure.reraise_if_any(fails)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher failures[0].reraise()
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher six.reraise(*self._exc_info)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher result = task.execute(**arguments)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 833, in execute
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher **volume_spec)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 741, in _create_from_image
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher image_service, context, image_id) as tmp_image:
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher return self.gen.next()
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 533, in fetch
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher fetch_verify_image(context, image_service, image_id, tmp)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 200, in fetch_verify_image
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher data = qemu_img_info(dest, run_as_root=run_as_root)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/image/image_utils.py", line 63, in qemu_img_info
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher out, _err = utils.execute(*cmd, run_as_root=run_as_root)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/cinder/utils.py", line 148, in execute
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher return processutils.execute(*cmd, **kwargs)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher File "/openstack/venvs/cinder-13.0.1/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 371, in execute
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher cmd=sanitized_cmd)
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher ProcessExecutionError: Unexpected error while running command.
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/conversion/tmpUXabEL
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher Exit code: 96
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher Stdout: u''
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher Stderr: u'/openstack/venvs/cinder-13.0.1/bin/cinder-rootwrap: Executable not found: qemu-img (filter match = qemu-img)\n'
2016-05-10 14:38:33.865 10357 ERROR oslo_messaging.rpc.dispatcher
A workaround is to manually install the qemu-utils package in the cinder-volume container. After performing the workaround I am able to successfully create cinder volumes backed by ceph, based on image source:
2016-05-10 14:49:28.317 10357 INFO cinder.volume.flows.manager.create_volume [req-fbc6fb79-cbc2-4fc7-97f8-d7b886f5b66c b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Volume af72d5a3-de07-4b7c-927c-f62ff2b18b97: being created as image with specification: {'status': u'creating', 'image_location': (u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', None), 'volume_size': 20, 'volume_name': 'volume-af72d5a3-de07-4b7c-927c-f62ff2b18b97', 'image_id': u'28bf4043-d449-416a-a994-234aa613ed7c', 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f648f70da90>, 'image_meta': {u'status': u'active', u'virtual_size': None, u'name': u'trusty64', u'tags': [], u'container_format': u'bare', u'created_at': datetime.datetime(2016, 5, 5, 21, 47, 13, tzinfo=<iso8601.Utc>), u'disk_format': u'qcow2', u'updated_at': datetime.datetime(2016, 5, 5, 21, 47, 20, tzinfo=<iso8601.Utc>), u'visibility': u'private', 'properties': {}, u'owner': u'a0adde92c1b74a67ad8c640168c28632', u'protected': False, u'id': u'28bf4043-d449-416a-a994-234aa613ed7c', u'file': u'/v2/images/28bf4043-d449-416a-a994-234aa613ed7c/file', u'checksum': u'e05b2f067604a1b4d0e9aa430b4e8721', u'min_disk': 0, u'direct_url': u'rbd://1a99be1b-fd46-4b67-b2f1-3ea3efdd000f/glance/28bf4043-d449-416a-a994-234aa613ed7c/snap', u'min_ram': 0, u'size': 259457536}}
2016-05-10 14:49:30.851 10357 INFO cinder.image.image_utils [req-fbc6fb79-cbc2-4fc7-97f8-d7b886f5b66c b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Image download 247.00 MB at 98.78 MB/s
2016-05-10 14:49:38.552 10357 INFO cinder.image.image_utils [req-fbc6fb79-cbc2-4fc7-97f8-d7b886f5b66c b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Converted 2252.00 MB image at 335.28 MB/s
2016-05-10 14:49:48.518 10357 INFO cinder.volume.flows.manager.create_volume [req-fbc6fb79-cbc2-4fc7-97f8-d7b886f5b66c b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Volume volume-af72d5a3-de07-4b7c-927c-f62ff2b18b97 (af72d5a3-de07-4b7c-927c-f62ff2b18b97): created successfully
2016-05-10 14:49:48.524 10357 INFO cinder.volume.manager [req-fbc6fb79-cbc2-4fc7-97f8-d7b886f5b66c b8e11a21c5f84dd2b629d480c00b9953 a0adde92c1b74a67ad8c640168c28632 - - -] Created volume successfully.
The right package is installed, but only if the cinder-volume service is also running on a host that's also used to provide an LVM backend.
https:/
https:/
We'll need to ensure that the qemu-img package is installed wherever the cinder-volume service is installed, regardless of the back-end. Alternatively we need to implement back-end specific package lists.