backup service startup fails when backup's original volume has been deleted

[Tom Barron]

The non-disruptive backup commit e78018bd05a51e33d5e1cd7c001ee6e5116a6370 introduced a new method to cleanup temporary snapshots and volumes left by interrupted live-backup operations. This method generates an exception that prevents the backup service from starting successfully when the original volume for a backup has been removed.

Steps to reproduce:

reate a volume:

tbarron@ubuntu4:/opt/stack/cinder$ cinder create 1
+---------------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2015-07-26T11:42:35.000000 |
| description | None |
| encrypted | False |
| id | 7513ff6f-8d9f-493a-8f71-48b682f07fcf |
| metadata | {} |
| multiattach | False |
| name | None |
| os-vol-host-attr:host | None |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 118accf365134a42947950eff2b6a1f9 |
| os-volume-replication:driver_data | None |
| os-volume-replication:extended_status | None |
| replication_status | disabled |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| user_id | 003cba7ded5b42bc9b2b13a1120442ae |
| volume_type | lvmdriver-1 |
+---------------------------------------+--------------------------------------+

Create a backup from the volume:

tbarron@ubuntu4:/opt/stack/cinder$ cinder backup-create 7513ff6f-8d9f-493a-8f71-48b682f07fcf
+-----------+--------------------------------------+
| Property | Value |
+-----------+--------------------------------------+
| id | 11cf1734-20d9-4cda-bfef-71a071af51f3 |
| name | None |
| volume_id | 7513ff6f-8d9f-493a-8f71-48b682f07fcf |
+-----------+--------------------------------------+

Delete the volume:
-----------------
tbarron@ubuntu4:/opt/stack/cinder$ cinder delete 7513ff6f-8d9f-493a-8f71-48b682f07fcf

Restart c-vol and observe the exception:
----------------------------------------

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
    result = function(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py", line 623, in run_service
    service.start()
  File "/opt/stack/cinder/cinder/service.py", line 145, in start
    self.manager.init_host()
  File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
    return f(*args, **kwargs)
  File "/opt/stack/cinder/cinder/backup/manager.py", line 259, in init_host
    self._cleanup_temp_volumes_snapshots(backups)
  File "/opt/stack/cinder/cinder/backup/manager.py", line 268, in _cleanup_temp_volumes_snapshots
    volume = self.db.volume_get(ctxt, backup.volume_id)
  File "/opt/stack/cinder/cinder/db/api.py", line 205, in volume_get
    return IMPL.volume_get(context, volume_id)
  File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 182, in wrapper
    return f(*args, **kwargs)
  File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 1337, in volume_get
    return _volume_get(context, volume_id)
  File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 182, in wrapper
    return f(*args, **kwargs)
  File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 1282, in _volume_get
    raise exception.VolumeNotFound(volume_id=volume_id)
VolumeNotFound: Volume 7513ff6f-8d9f-493a-8f71-48b682f07fcf could not be found.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/206131

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/206131
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=54f97d28b04b996ed1c8e7fa15f1f4d8f3d58c50
Submitter: Jenkins
Branch: master

commit 54f97d28b04b996ed1c8e7fa15f1f4d8f3d58c50
Author: Tom Barron <email address hidden>
Date: Sat Jul 25 17:11:29 2015 -0400

    Fix cleanup_temp_volume_snapshots for missing vol

    The cleanup_temp_volume_snapshots method raises an exception that
    causes the backup service to exit if there is a backup without a
    corresponding volume.

    This commit catches these exceptions so that the backup service is
    not prevented from starting in this circumstance.

    Change-Id: Ia1aac2fe78048df5fce595d5de181cb4930ea78d
    Closes-bug: 1478599