Corey,
My apologies for the previous comment please disregard, I forgot that I needed to run upgrade rather than install.
after running
sudo apt-get upgrade os-brick-common/focal-proposed python3-os-brick/focal-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:20.04/focal-proposed [all]) for 'os-brick-common'
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:20.04/focal-proposed [all]) for 'python3-os-brick'
Calculating upgrade... Done
The following packages will be upgraded:
os-brick-common python3-os-brick
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst python3-os-brick [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all]) []
Inst os-brick-common [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])
Conf python3-os-brick (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])
Conf os-brick-common (3.0.1-0ubuntu1.3 Ubuntu:20.04/focal-proposed [all])
and restarting cinder services
I am able to perform cinder backups and restore cinder backups, but it looks like there may be another issue with creating a snapshot and then creating a backup from the snapshot and then restoring that backup.
As a side note spawning a new instance from a snapshot, which creates a new volume works just fine.
Here are the logs from a failed restore, when restoring a backup that was created from a volume snapshot
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager [req-ff639558-c9ae-4dd3-923d-ec158b08f3dc 78af964f6db708aebd4103b6ee7fbdfe2d30f6e775a0a007ee04cd4633ccb87d aea00f20ec77483d963d2e8f54a1132a - e5bf6979cc0e42fa86ce5be4f12241f1 e5bf6979cc0e42fa86ce5be4f12241f1] Restoring backup 9574fcbf-fe7c-41f9-8d28-dc46e3643aae to volume a0fd2623-9ab0-402e-a9da-d7bcc4bd3247 failed.: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager Traceback (most recent call last):
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 657, in _run_restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager backup_service.restore(backup, volume.id,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1264, in restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager self._restore_volume(backup, target_volume, volume_file)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1230, in _restore_volume
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager self._full_restore(backup, volume_file, volume.name,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1023, in _full_restore
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager src_rbd = eventlet.tpool.Proxy(self.rbd.Image(client.ioctx,
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager File "rbd.pyx", line 3376, in rbd.Image.__init__
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.002 951 ERROR cinder.backup.manager
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server [req-ff639558-c9ae-4dd3-923d-ec158b08f3dc 78af964f6db708aebd4103b6ee7fbdfe2d30f6e775a0a007ee04cd4633ccb87d aea00f20ec77483d963d2e8f54a1132a - e5bf6979cc0e42fa86ce5be4f12241f1 e5bf6979cc0e42fa86ce5be4f12241f1] Exception during message handling: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 196, in _do_dispatch
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 597, in restore_backup
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server backup.save()
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server self.force_reraise()
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server raise value
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 586, in restore_backup
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server self._run_restore(context, backup, volume)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/manager.py", line 657, in _run_restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server backup_service.restore(backup, volume.id,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1264, in restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server self._restore_volume(backup, target_volume, volume_file)
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1230, in _restore_volume
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server self._full_restore(backup, volume_file, volume.name,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/backup/drivers/ceph.py", line 1023, in _full_restore
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server src_rbd = eventlet.tpool.Proxy(self.rbd.Image(client.ioctx,
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server File "rbd.pyx", line 3376, in rbd.Image.__init__
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.rpc.server rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume-a97b9aaa-bcce-4ac2-ab84-60a6f07f4be2.backup.base' at snapshot None)
Corey,
My apologies for the previous comment please disregard, I forgot that I needed to run upgrade rather than install.
after running common/ focal-proposed python3- os-brick/ focal-proposed 20.04/focal- proposed [all]) for 'os-brick-common' 20.04/focal- proposed [all]) for 'python3-os-brick' 20.04/focal- proposed [all]) [] 20.04/focal- proposed [all]) 20.04/focal- proposed [all]) 20.04/focal- proposed [all])
sudo apt-get upgrade os-brick-
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:
Selected version '3.0.1-0ubuntu1.3' (Ubuntu:
Calculating upgrade... Done
The following packages will be upgraded:
os-brick-common python3-os-brick
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst python3-os-brick [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:
Inst os-brick-common [3.0.1-0ubuntu1.2] (3.0.1-0ubuntu1.3 Ubuntu:
Conf python3-os-brick (3.0.1-0ubuntu1.3 Ubuntu:
Conf os-brick-common (3.0.1-0ubuntu1.3 Ubuntu:
and restarting cinder services
I am able to perform cinder backups and restore cinder backups, but it looks like there may be another issue with creating a snapshot and then creating a backup from the snapshot and then restoring that backup.
As a side note spawning a new instance from a snapshot, which creates a new volume works just fine.
Here are the logs from a failed restore, when restoring a backup that was created from a volume snapshot
2020-10-06 18:38:17.002 951 ERROR cinder. backup. manager [req-ff639558- c9ae-4dd3- 923d-ec158b08f3 dc 78af964f6db708a ebd4103b6ee7fbd fe2d30f6e775a0a 007ee04cd4633cc b87d aea00f20ec77483 d963d2e8f54a113 2a - e5bf6979cc0e42f a86ce5be4f12241 f1 e5bf6979cc0e42f a86ce5be4f12241 f1] Restoring backup 9574fcbf- fe7c-41f9- 8d28-dc46e3643a ae to volume a0fd2623- 9ab0-402e- a9da-d7bcc4bd32 47 failed.: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume- a97b9aaa- bcce-4ac2- ab84-60a6f07f4b e2.backup. base' at snapshot None) backup. manager Traceback (most recent call last): backup. manager File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ manager. py", line 657, in _run_restore backup. manager backup_ service. restore( backup, volume.id, backup. manager File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1264, in restore backup. manager self._restore_ volume( backup, target_volume, volume_file) backup. manager File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1230, in _restore_volume backup. manager self._full_ restore( backup, volume_file, volume.name, backup. manager File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1023, in _full_restore backup. manager src_rbd = eventlet. tpool.Proxy( self.rbd. Image(client. ioctx, backup. manager File "rbd.pyx", line 3376, in rbd.Image.__init__ backup. manager rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume- a97b9aaa- bcce-4ac2- ab84-60a6f07f4b e2.backup. base' at snapshot None) backup. manager rpc.server [req-ff639558- c9ae-4dd3- 923d-ec158b08f3 dc 78af964f6db708a ebd4103b6ee7fbd fe2d30f6e775a0a 007ee04cd4633cc b87d aea00f20ec77483 d963d2e8f54a113 2a - e5bf6979cc0e42f a86ce5be4f12241 f1 e5bf6979cc0e42f a86ce5be4f12241 f1] Exception during message handling: rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume- a97b9aaa- bcce-4ac2- ab84-60a6f07f4b e2.backup. base' at snapshot None) rpc.server Traceback (most recent call last): rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/server. py", line 165, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/dispatcher. py", line 276, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/dispatcher. py", line 196, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ manager. py", line 597, in restore_backup rpc.server backup.save() rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 220, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 196, in force_reraise rpc.server six.reraise( self.type_ , self.value, self.tb) rpc.server File "/usr/lib/ python3/ dist-packages/ six.py" , line 703, in reraise rpc.server raise value rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ manager. py", line 586, in restore_backup rpc.server self._run_ restore( context, backup, volume) rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ manager. py", line 657, in _run_restore rpc.server backup_ service. restore( backup, volume.id, rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1264, in restore rpc.server self._restore_ volume( backup, target_volume, volume_file) rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1230, in _restore_volume rpc.server self._full_ restore( backup, volume_file, volume.name, rpc.server File "/usr/lib/ python3/ dist-packages/ cinder/ backup/ drivers/ ceph.py" , line 1023, in _full_restore rpc.server src_rbd = eventlet. tpool.Proxy( self.rbd. Image(client. ioctx, rpc.server File "rbd.pyx", line 3376, in rbd.Image.__init__ rpc.server rbd.ImageNotFound: [errno 2] RBD image not found (error opening image b'volume- a97b9aaa- bcce-4ac2- ab84-60a6f07f4b e2.backup. base' at snapshot None)
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.002 951 ERROR cinder.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.
2020-10-06 18:38:17.081 951 ERROR oslo_messaging.