Ceph backup-restore fails if original volume got deleted

Bug #1199661 reported by Marc Koderer on 2013-07-10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Edward Hope-Morley

Bug Description

Backup driver set to "ceph". The swift driver works with this scenario.


1. cinder create 1 --display-name=QA_acceptance # note down the id from output
2. cinder backup-create <volume ID from QA_acceptance volume>
3. watch -n 0 -d cinder backup-list # wait for backup to complete
4. cinder delete <volume ID from QA_acceptance volume>
5. cinder backup-restore <backup ID of the created backup - taken from step 3>
6. cinder list | grep restore_backup_<backup ID> # check status. Actual: error_restoring Expected: available


2013-07-10 09:44:28 ERROR [cinder.openstack.common.rpc.amqp] Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 430, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/dispatcher.py", line 133, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 217, in restore_backup
    {'status': 'available'})
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 211, in restore_backup
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/rbd.py", line 596, in restore_backup
    backup_service.restore(backup, volume['id'], rbd_fd)
  File "/usr/lib/python2.7/dist-packages/cinder/backup/services/ceph.py", line 191, in restore
    volume = self.db.volume_get(self.context, volume_id)
  File "/usr/lib/python2.7/dist-packages/cinder/db/api.py", line 232, in volume_get
    return IMPL.volume_get(context, volume_id)
  File "/usr/lib/python2.7/dist-packages/cinder/db/sqlalchemy/api.py", line 123, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/db/sqlalchemy/api.py", line 1023, in volume_get
    raise exception.VolumeNotFound(volume_id=volume_id)
VolumeNotFound: Volume 31ecd14f-ef84-4e1a-acc2-b98b29261a56 could not be found.

cinder list status: error_restoring

Expected result:
Restore should always work even if the original volume got deleted.

Edward Hope-Morley (hopem) wrote :

The issue here is that the restore is using the wrong volume_id i.e. always that of the original volume. This has been fixed in https://review.openstack.org/#/c/35216/

Not sure whether to backport fix or wait for that review to be finished?

Changed in cinder:
assignee: nobody → Edward Hope-Morley (hopem)
Mike Perez (thingee) on 2013-07-10
tags: added: backup-service ceph
Changed in cinder:
status: New → In Progress

Reviewed: https://review.openstack.org/36719
Committed: http://github.com/openstack/cinder/commit/f6a8b28dbbb86f97458b80cb5f583725fec4b62f
Submitter: Jenkins
Branch: master

commit f6a8b28dbbb86f97458b80cb5f583725fec4b62f
Author: Edward Hope-Morley <email address hidden>
Date: Thu Jul 11 19:38:15 2013 +0100

    Fixes ceph-backup failure if original volume deleted

    Fixes: bug #1199661

    Change-Id: I21f979bf2c0ca6910a6437cb6277193f71fed664

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2013-07-17
Changed in cinder:
milestone: none → havana-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2013-10-17
Changed in cinder:
milestone: havana-2 → 2013.2
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers