Activity log for bug #1952998

Date Who What changed Old value New value Message
2021-12-02 06:47:00 Edin S bug added bug
2021-12-02 21:49:05 Andrea Ieri bug added subscriber Canonical IS BootStack
2021-12-03 01:26:05 Edin S description The volume backup is an error state which prevents its removal. Output: $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 | fail_reason | Backup should only have one snapshot but instead has 4 | Volume backup maps to a volume_id of 07bb983c... $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 <SNIP> | volume_id | 07bb983c-0ab4-48ae-822d-890433cf1359 | Below, we can see that 4 snapshots exist in ceph (for the volume_id 07bb983c-0ab4-48ae-822d-890433cf1359): # rbd --pool cinder-ceph snap list cinder-ceph/volume-07bb983c-0ab4-48ae-822d-890433cf1359 SNAPID NAME SIZE PROTECTED TIMESTAMP 12935 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.88 100 GiB Tue Jun 15 02:32:06 2021 12936 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.92 100 GiB Tue Jun 15 02:32:06 2021 12941 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.09 100 GiB Tue Jun 15 02:32:06 2021 12944 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.11 100 GiB Tue Jun 15 02:32:06 2021 Looking at the timestamps of the snapshots, it looks like it tried creating 4 snapshots at exactly the same time. Database state for the backup: mysql> select id,deleted,deleted_at,status from backups where id in ("718ad411-20ce-4f15-a4f6-03ddde5d0ee9"); +--------------------------------------+---------+------------+-----------+ | id | deleted | deleted_at | status | +--------------------------------------+---------+------------+-----------+ <SNIP> | 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 | 0 | NULL | error | <SNIP> I was unable to determine what triggered this issue. The volume backup has multiple snapshots associated with it, which prevents its removal. Output:  $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9     | fail_reason | Backup should only have one snapshot but instead has 4 | Volume backup maps to a volume_id of 07bb983c...    $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 <SNIP>     | volume_id | 07bb983c-0ab4-48ae-822d-890433cf1359 | Below, we can see that 4 snapshots exist in ceph (for the volume_id 07bb983c-0ab4-48ae-822d-890433cf1359):     # rbd --pool cinder-ceph snap list cinder-ceph/volume-07bb983c-0ab4-48ae-822d-890433cf1359     SNAPID NAME SIZE PROTECTED TIMESTAMP      12935 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.88 100 GiB Tue Jun 15 02:32:06 2021      12936 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.92 100 GiB Tue Jun 15 02:32:06 2021      12941 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.09 100 GiB Tue Jun 15 02:32:06 2021      12944 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.11 100 GiB Tue Jun 15 02:32:06 2021 Looking at the timestamps of the snapshots, it looks like it tried creating 4 snapshots at exactly the same time. Database state for the backup:     mysql> select id,deleted,deleted_at,status from backups where id in ("718ad411-20ce-4f15-a4f6-03ddde5d0ee9");     +--------------------------------------+---------+------------+-----------+     | id | deleted | deleted_at | status |     +--------------------------------------+---------+------------+-----------+ <SNIP>     | 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 | 0 | NULL | error | <SNIP> I was unable to determine what triggered this issue.
2021-12-04 09:13:50 Edin S description The volume backup has multiple snapshots associated with it, which prevents its removal. Output:  $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9     | fail_reason | Backup should only have one snapshot but instead has 4 | Volume backup maps to a volume_id of 07bb983c...    $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 <SNIP>     | volume_id | 07bb983c-0ab4-48ae-822d-890433cf1359 | Below, we can see that 4 snapshots exist in ceph (for the volume_id 07bb983c-0ab4-48ae-822d-890433cf1359):     # rbd --pool cinder-ceph snap list cinder-ceph/volume-07bb983c-0ab4-48ae-822d-890433cf1359     SNAPID NAME SIZE PROTECTED TIMESTAMP      12935 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.88 100 GiB Tue Jun 15 02:32:06 2021      12936 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.92 100 GiB Tue Jun 15 02:32:06 2021      12941 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.09 100 GiB Tue Jun 15 02:32:06 2021      12944 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.11 100 GiB Tue Jun 15 02:32:06 2021 Looking at the timestamps of the snapshots, it looks like it tried creating 4 snapshots at exactly the same time. Database state for the backup:     mysql> select id,deleted,deleted_at,status from backups where id in ("718ad411-20ce-4f15-a4f6-03ddde5d0ee9");     +--------------------------------------+---------+------------+-----------+     | id | deleted | deleted_at | status |     +--------------------------------------+---------+------------+-----------+ <SNIP>     | 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 | 0 | NULL | error | <SNIP> I was unable to determine what triggered this issue. The volume backup has multiple snapshots associated with it, which prevents its removal. Ubuntu 20.04 Openstack Ussuri Output:  $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9     | fail_reason | Backup should only have one snapshot but instead has 4 | Volume backup maps to a volume_id of 07bb983c...    $ os volume backup show 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 <SNIP>     | volume_id | 07bb983c-0ab4-48ae-822d-890433cf1359 | Below, we can see that 4 snapshots exist in ceph (for the volume_id 07bb983c-0ab4-48ae-822d-890433cf1359):     # rbd --pool cinder-ceph snap list cinder-ceph/volume-07bb983c-0ab4-48ae-822d-890433cf1359     SNAPID NAME SIZE PROTECTED TIMESTAMP      12935 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.88 100 GiB Tue Jun 15 02:32:06 2021      12936 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724325.92 100 GiB Tue Jun 15 02:32:06 2021      12941 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.09 100 GiB Tue Jun 15 02:32:06 2021      12944 backup.718ad411-20ce-4f15-a4f6-03ddde5d0ee9.snap.1623724326.11 100 GiB Tue Jun 15 02:32:06 2021 Looking at the timestamps of the snapshots, it looks like it tried creating 4 snapshots at exactly the same time. Database state for the backup:     mysql> select id,deleted,deleted_at,status from backups where id in ("718ad411-20ce-4f15-a4f6-03ddde5d0ee9");     +--------------------------------------+---------+------------+-----------+     | id | deleted | deleted_at | status |     +--------------------------------------+---------+------------+-----------+ <SNIP>     | 718ad411-20ce-4f15-a4f6-03ddde5d0ee9 | 0 | NULL | error | <SNIP> I was unable to determine what triggered this issue.
2021-12-07 21:50:50 Sofia Enriquez summary Volume backup created with multiple snapshots preventing removal [Ceph] Volume backup created with multiple snapshots preventing removal
2021-12-07 21:53:38 Sofia Enriquez cinder: importance Undecided Low
2021-12-07 21:53:41 Sofia Enriquez cinder: status New Incomplete
2021-12-07 21:55:21 Sofia Enriquez tags backup-service