Can't delete an image in deleted status

[Sam Morrison]

I'm trying to delete an image that has a status of "deleted"

It's not deleted as I can do an image-show and it returns plus I can see it in image_locations and it exists in the backend which for us is swift

glance image-show 17c6077c-99f0-41c7-9bd2-175216330990
+-------------------------------------------+--------------------------------------+
| Property | Value |
+-------------------------------------------+--------------------------------------+
| checksum | c9ef771d317595fd3654ca69a4be5f31 |
| container_format | bare |
| created_at | 2014-05-22T07:58:23 |
| deleted | True |
| deleted_at | 2014-05-23T02:16:53 |
| disk_format | raw |
| id | 17c6077c-99f0-41c7-9bd2-175216330990 |
| is_public | True |
| min_disk | 10 |
| min_ram | 0 |
| name | XXXXXX |
| owner | XXXXX |
| protected | False |
| size | 10737418240 |
| status | deleted |
| updated_at | 2014-05-23T02:16:53 |
+-------------------------------------------+--------------------------------------+

glance image-delete 17c6077c-99f0-41c7-9bd2-175216330990
Request returned failure status.
404 Not Found
Image 17c6077c-99f0-41c7-9bd2-175216330990 not found.
    (HTTP 404): Unable to delete image 17c6077c-99f0-41c7-9bd2-175216330990

[Alexander Tivelkov (ativelkov)]

The image-show in v1 does indeed show deleted images (i.e. having 'status' equal to 'deleted' - this is a "by design" behaviour for v1). An attempt to delete such an image should also cause 404 error.

The strange thing is that your images do not get removed from the backend storage though: this looks like a bug indeed.

[Hemanth Makkapati (hemanth-makkapati)]

If Glance hits an error while deleting image segments, it leaves the image data behind in the store. Could this be the reason for seeing data in the backend?

[Sam Morrison (sorrison)]

Yes I think that is the cause here, still should be able to retry the delete though.

[Abhishek Talwar (abhishek-talwar)]

Hey,

So what kind of solution are we looking for this bug ? As v1 permits us to see deleted images and flavors. So what could be the probable solution ?

Please suggest.

[Sam Morrison (sorrison)]

There is a couple of issues here. An image can have a status of deleted but not actually be deleted from the store.
Maybe it should have a different status. Cinder has error_deleting. The api could then allow delete on that resource to retry the delete.

In V1 an image is considered deleted if the DB column deleted is 1. In V2 API this is the same with the addition of taking the status=deleted into account.

This is a bigger issue in the V2 API as an image is essentially gone but it could still be consuming swift resources which is bad.

[Louis Taylor (kragniz)]

Marking as a duplicate, since that appears to be the root cause.

[Sam Morrison (sorrison)]

Unmarking this as a duplicate, there are many ways where glance can get into this issue. Yes that is one but it doesn't matter how it gets into this state there should be a way for an operator to get it out of this state.

For example if while deleting an image you get a failure in swift when deleting a chunk this will also make it end up in this state.

[Steve Lewis (steve-lewis)]

Related blueprint
https://blueprints.launchpad.net/glance/+spec/make-delete-image-repeatable

[Nikhil Komawar (nikhil-komawar)]

Unfortunately, we cannot re-delete Image via the API by design.

Another approach for this real issue might be to use the scrubber to keep track of such leftover data and retry deletes. Please provide your suggestions here: https://review.openstack.org/#/c/125156/