My first thought on this is that the cloned image is flattened as part of direct_snapshot, so I'm interested in finding out why that's not sufficient for the image-delete to work. To investigate, I tried to repro this myself in devstack and am hitting multiple other bugs. I can't get it to complete the snapshot successfully and the instance does not get shelved. The first trace I get is: 2017-01-05 01:14:50.499 ERROR nova.virt.libvirt.driver [req-a00f11f9-344f-40b7-b2ef-27b46825d9d6 admin admin] Failed to snapshot image 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver Traceback (most recent call last): 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1559, in snapshot 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver instance.image_ref) 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 990, in direct_snapshot 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver self.driver.clone(location, image_id, dest_pool=parent_pool) 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/storage/rbd_utils.py", line 234, in clone 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver with RADOSClient(self, dest_pool) as dest_client: 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/storage/rbd_utils.py", line 105, in __init__ 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver self.cluster, self.ioctx = driver._connect_to_rados(pool) 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/storage/rbd_utils.py", line 138, in _connect_to_rados 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver ioctx = client.open_ioctx(pool_to_open) 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/rados.py", line 662, in open_ioctx 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver raise TypeError('the name of the pool must be a string') 2017-01-05 01:14:50.499 TRACE nova.virt.libvirt.driver TypeError: the name of the pool must be a string which I remedied locally by s/dest_pool/str(dest_pool)/ in the clone function in rbd_utils.py. In my environment, dest_pool is of type unicode. After that, I hit the next trace: 2017-01-05 01:20:31.675 ERROR nova.virt.libvirt.driver [req-152c9895-39cc-464a-ae38-0e6f613278b5 admin admin] Failed to snapshot image 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver Traceback (most recent call last): 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1559, in snapshot 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver instance.image_ref) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 995, in direct_snapshot 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver self.cleanup_direct_snapshot(location) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 1017, in cleanup_direct_snapshot 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver ignore_errors=ignore_errors) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/opt/stack/nova/nova/virt/libvirt/storage/rbd_utils.py", line 404, in remove_snap 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver vol.unprotect_snap(name) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver rv = execute(f, *args, **kwargs) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver six.reraise(c, e, tb) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver rv = meth(*args, **kwargs) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/rbd.py", line 654, in unprotect_snap 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver raise make_ex(ret, 'error unprotecting snapshot %s@%s' % (self.name, name)) 2017-01-05 01:20:31.675 TRACE nova.virt.libvirt.driver ImageBusy: error unprotecting snapshot 782834b3-f1a7-4b35-9309-796b3562ee49_disk@0a8c11db6e1349648f752dc6c0f81e88 The instance ends up in SHUTOFF state and not shelved. $ pwd /opt/stack/nova $ git rev-parse HEAD 1bcf3b553ae8665dc6308d1bd11898efb75d3a41 and local.conf: [[local|localrc]] DEST=/opt/stack/ LOGDIR=$DEST/logs LOGFILE=$LOGDIR/stack.sh.log enable_plugin devstack-plugin-ceph git://git.openstack.org/openstack/devstack-plugin-ceph HOST_IP=127.0.0.1