Another situation where the "generated object list" returns empty is when a large number of volume backup deletes are performed. For example (this is an extreme, but possible, example that I tested), when 1,000 commands such as:
openstack volume backup delete <uuid> &
are run in a bash shell, with valid UUIDs of non-deleted backups (those that have objects reported by Swift), many of the backups are marked as deleted in the Cinder database, but the objects are orphaned on the object storage.
One of the volume backups that was deleted and has orphaned objects produced these two cinder-backup.log entries:
showing that it wasn't able to retrieve the object list (but did not error in any way) - even though I can list the objects successfully from this user's account using:
swift list volumebackups -p volume_77bd1522-68d6-43d3-8178-04a096c6db95/20190529071528/az_us-central-1a_backup_6efd5baf-4914-411c-bc20-d9fb53504c6d
Another situation where the "generated object list" returns empty is when a large number of volume backup deletes are performed. For example (this is an extreme, but possible, example that I tested), when 1,000 commands such as:
openstack volume backup delete <uuid> &
are run in a bash shell, with valid UUIDs of non-deleted backups (those that have objects reported by Swift), many of the backups are marked as deleted in the Cinder database, but the objects are orphaned on the object storage.
One of the volume backups that was deleted and has orphaned objects produced these two cinder-backup.log entries:
/var/lib/ docker/ volumes/ kolla_logs/ _data/cinder/ cinder- backup. log:2019- 09-30 16:20:33.367 6 DEBUG cinder. backup. chunkeddriver [req-9434f1e1- c382-4d86- 8f81-55602dc281 85 720763d70a094bf 3b11ebabd31eee8 96 8acb2072057b45a b9da245880af92c 93 - default default] delete started, backup: 6efd5baf- 4914-411c- bc20-d9fb53504c 6d, container: volumebackups, prefix: volume_ 77bd1522- 68d6-43d3- 8178-04a096c6db 95/201905290715 28/az_us- central- 1a_backup_ 6efd5baf- 4914-411c- bc20-d9fb53504c 6d. delete_backup /var/lib/ kolla/venv/ lib/python2. 7/site- packages/ cinder/ backup/ chunkeddriver. py:807
/var/lib/ docker/ volumes/ kolla_logs/ _data/cinder/ cinder- backup. log:2019- 09-30 16:20:33.418 6 DEBUG cinder. backup. chunkeddriver [req-9434f1e1- c382-4d86- 8f81-55602dc281 85 720763d70a094bf 3b11ebabd31eee8 96 8acb2072057b45a b9da245880af92c 93 - default default] generated object list: []. _generate_ object_ names /var/lib/ kolla/venv/ lib/python2. 7/site- packages/ cinder/ backup/ chunkeddriver. py:232
showing that it wasn't able to retrieve the object list (but did not error in any way) - even though I can list the objects successfully from this user's account using:
swift list volumebackups -p volume_ 77bd1522- 68d6-43d3- 8178-04a096c6db 95/201905290715 28/az_us- central- 1a_backup_ 6efd5baf- 4914-411c- bc20-d9fb53504c 6d
Eric