Glance cannot delete incomplete chunked upload to Swift backend if token expires

[Dmitry Sutyagin]

This bug is related to https://bugs.launchpad.net/glance-store/+bug/1471814, which has not been properly fixed upstream (please reopen, see latest updates in https://bugs.launchpad.net/mos/+bug/1584070)

Downstream bug - https://bugs.launchpad.net/mos/+bug/1732033

Originally reported as part of https://bugs.launchpad.net/mos/+bug/1731054

There are 2 issues with deletion of stale chunks if a large upload failed (for whatever reason):
1. there is no retry on exception https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L872
2. the auth retry in swiftclient's code fails to get auth because (for some reason) Glance initializes connection with (implicit) auth_version = 1 - here https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L1142 - no auth_version is passed, swiftclient defaults to "1", and swiftclient then uses get_auth_1_0 function instead of using v2 or v3 function get_auth_keystone, and fails (because self.session.authurl = None which is assumed mandatory for this function).

Here is a log extract - http://paste.openstack.org/show/626200/
Note how I first forced the upload to fail (by Ctrl+C on client side), which caused the 1st trace

Then I forced the token to expire by setting expiration to 120 in keystone, and adding a sleep inside _delete_stale_chunks loop (imitating slow deletion / many-many chunks). As long as execution of _delete_stale_chunks takes around or longer than swift_store_expire_soon_interval (default = 60s), this issue will appear by itself.

The 2nd trace is auth error, and then you can see that the deletion is not retried for this chunk - this is the first issue
the 3rd and subsequent traces are caused by auth_version = 1, so deletion of all subsequent chunks also fails - this is the second issue

[Brian Rosmaita (brian-rosmaita)]

Looks like a problem when using swift backend in multi-tenant-store configuration (see the related bug mentioned in the description).

[Erno Kuvaja (jokke)]

Is this still valid bug?

No actions for over a year to address it.