Ceph backup-restore fails if original volume got deleted

Bug #1199661 reported by Marc Koderer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Undecided
Edward Hope-Morley

Bug Description

Setup:
Backup driver set to "ceph". The swift driver works with this scenario.

Steps:

1. cinder create 1 --display-name=QA_acceptance # note down the id from output
2. cinder backup-create <volume ID from QA_acceptance volume>
3. watch -n 0 -d cinder backup-list # wait for backup to complete
4. cinder delete <volume ID from QA_acceptance volume>
5. cinder backup-restore <backup ID of the created backup - taken from step 3>
6. cinder list | grep restore_backup_<backup ID> # check status. Actual: error_restoring Expected: available

Result:

2013-07-10 09:44:28 ERROR [cinder.openstack.common.rpc.amqp] Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 430, in _process_data
    rval = self.proxy.dispatch(ctxt, version, method, **args)
  File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/dispatcher.py", line 133, in dispatch
    return getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 217, in restore_backup
    {'status': 'available'})
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/usr/lib/python2.7/dist-packages/cinder/backup/manager.py", line 211, in restore_backup
    backup_service)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/rbd.py", line 596, in restore_backup
    backup_service.restore(backup, volume['id'], rbd_fd)
  File "/usr/lib/python2.7/dist-packages/cinder/backup/services/ceph.py", line 191, in restore
    volume = self.db.volume_get(self.context, volume_id)
  File "/usr/lib/python2.7/dist-packages/cinder/db/api.py", line 232, in volume_get
    return IMPL.volume_get(context, volume_id)
  File "/usr/lib/python2.7/dist-packages/cinder/db/sqlalchemy/api.py", line 123, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/db/sqlalchemy/api.py", line 1023, in volume_get
    raise exception.VolumeNotFound(volume_id=volume_id)
VolumeNotFound: Volume 31ecd14f-ef84-4e1a-acc2-b98b29261a56 could not be found.

cinder list status: error_restoring

Expected result:
Restore should always work even if the original volume got deleted.

Revision history for this message
Edward Hope-Morley (hopem) wrote :

The issue here is that the restore is using the wrong volume_id i.e. always that of the original volume. This has been fixed in https://review.openstack.org/#/c/35216/

Not sure whether to backport fix or wait for that review to be finished?

Changed in cinder:
assignee: nobody → Edward Hope-Morley (hopem)
Mike Perez (thingee)
tags: added: backup-service ceph
Changed in cinder:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/36719

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/36719
Committed: http://github.com/openstack/cinder/commit/f6a8b28dbbb86f97458b80cb5f583725fec4b62f
Submitter: Jenkins
Branch: master

commit f6a8b28dbbb86f97458b80cb5f583725fec4b62f
Author: Edward Hope-Morley <email address hidden>
Date: Thu Jul 11 19:38:15 2013 +0100

    Fixes ceph-backup failure if original volume deleted

    Fixes: bug #1199661

    Change-Id: I21f979bf2c0ca6910a6437cb6277193f71fed664

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in cinder:
milestone: none → havana-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in cinder:
milestone: havana-2 → 2013.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.