OpenStack Liberty with CEPH as a backend for Glance and Cinder.
root@node22:~# dpkg -l | grep nova
ii nova-common 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - common files
ii nova-compute 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node libvirt support
ii python-nova 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute Python libraries
ii python-novaclient 2:2.30.1-1~cloud0 all client library for OpenStack Compute API
dpkg -l | grep ceph
ii ceph 0.94.5-0ubuntu0.15.10.1~cloud0 amd64 distributed storage and file system
ii ceph-common 0.94.5-0ubuntu0.15.10.1~cloud0 amd64 common utilities to mount and interact with a ceph storage cluster
ii libcephfs1 0.94.5-0ubuntu0.15.10.1~cloud0 amd64 Ceph distributed file system client library
ii python-cephfs 0.94.5-0ubuntu0.15.10.1~cloud0 amd64 Python libraries for the Ceph libcephfs library
kris@node23:~$ dpkg -l | grep python | grep rbd
ii python-rbd 0.94.5-0ubuntu0.15.10.1~cloud0 amd64 Python libraries for the Ceph librbd library
QCOW2 image worked fine, but content of image was copied form image volume to vms volume in Ceph storage.
RAW images are uploaded to Glance. Instantiate it from image faile with follwing error on Compute node.
2016-01-24 20:26:29.640 1801 INFO nova.compute.manager [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Starting instance...
2016-01-24 20:26:29.834 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Attempting claim: memory 128 MB, disk 1 GB
2016-01-24 20:26:29.835 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Total memory: 16038 MB, used: 640.00 MB
2016-01-24 20:26:29.835 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] memory limit: 24057.00 MB, free: 23417.00 MB
2016-01-24 20:26:29.836 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Total disk: 1342 GB, used: 1.00 GB
2016-01-24 20:26:29.836 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] disk limit: 1342.00 GB, free: 1341.00 GB
2016-01-24 20:26:29.917 1801 INFO nova.compute.claims [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Claim successful
2016-01-24 20:26:31.023 1801 INFO nova.virt.libvirt.driver [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Creating image
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Instance failed to spawn
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Traceback (most recent call last):
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2155, in _build_resources
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] yield resources
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] block_device_info=block_device_info)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2437, in spawn
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] admin_pass=admin_password)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2861, in _create_image
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] instance, size, fallback_from_host)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 6282, in _try_fetch_image_cache
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] size=size)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 240, in cache
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] *args, **kwargs)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 777, in create_image
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] prepare_template(target=base, max_size=size, *args, **kwargs)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 254, in inner
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] return f(*args, **kwargs)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 230, in fetch_func_sync
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] fetch_func(target=target, *args, **kwargs)
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2853, in clone_fallback_to_fetch
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] backend.clone(context, disk_images['image_id'])
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 808, in clone
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] if self.driver.is_cloneable(location, image_meta):
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py", line 201, in is_cloneable
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] if image_meta.disk_format != 'raw':
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] AttributeError: 'dict' object has no attribute 'disk_format'
2016-01-24 20:26:31.328 1801 ERROR nova.compute.manager [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db]
2016-01-24 20:26:32.131 1801 INFO nova.compute.manager [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Terminating instance
2016-01-24 20:26:32.137 1801 INFO nova.virt.libvirt.driver [-] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] During wait destroy, instance disappeared.
2016-01-24 20:26:32.203 1801 INFO nova.virt.libvirt.driver [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Deleting instance files /var/lib/nova/instances/768e2495-5af2-4b67-a82f-ef65bc3514db_del
2016-01-24 20:26:32.204 1801 INFO nova.virt.libvirt.driver [req-7ec83a5e-ede5-4ff6-8753-3b57bdbcea31 295c57a2867b47e0a1fab756bf44477f 43abd7de61794e61830d21c23b87f798 - - -] [instance: 768e2495-5af2-4b67-a82f-ef65bc3514db] Deletion of /var/lib/nova/instances/768e2495-5af2-4b67-a82f-ef65bc3514db_del complete
I figure out temorary fix by changing /usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/rbd_utils.py around line 193 form
if image_meta.disk_format != 'raw':
reason = ("rbd image clone requires image format to be "
"'raw' but image {0} is '{1}'").format(
url, image_meta.disk_format)
to:
if image_meta['disk_format'] != 'raw':
reason = ("rbd image clone requires image format to be "
"'raw' but image {0} is '{1}'").format(
url, image_meta['disk_format'])
This fix solved problem in my setup. RAW images use properly snapshot and clone function of CEPH.
Best regards,
KK
@Krzysztof Kwiecień:
Bug skimming
------------
This should be a duplicate to bug 1508230
Please double-check that and if you think it's not the same remove the duplication notice so that we can check that again.