Physical image can not be deleted again if deletion is failed with OSError
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Glance |
Fix Released
|
Medium
|
Abhishek Kekane |
Bug Description
If the deletion of physical image failed with OSError for some reason, it is not possible to delete the image using image-delete api call.
In the longevity test we have encountered this issue, IMO there is some issue with OS and it is throwing "Forbidden" exception and somehow its not deleting that file, so in order to reproduce this issue please follow the below mentioned steps:
Steps to reproduce:
1. image creation, upload
openstack@
+--
| Property | Value |
+--
| checksum | None |
| container_format | None |
| created_at | 2014-01-29T02:27:07 |
| deleted | False |
| deleted_at | None |
| disk_format | None |
| id | 675a82ab-
| is_public | False |
| min_disk | 0 |
| min_ram | 0 |
| name | None |
| owner | de4a6631051a4df
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2014-01-29T02:27:07 |
+--
openstack@
+--
| Property | Value |
+--
| checksum | a5282e9259f822c
| container_format | bare |
| created_at | 2014-01-29T02:27:07 |
| deleted | False |
| deleted_at | None |
| disk_format | qcow2 |
| id | 675a82ab-
| is_public | False |
| min_disk | 0 |
| min_ram | 0 |
| name | None |
| owner | de4a6631051a4df
| protected | False |
| size | 43450368 |
| status | active |
| updated_at | 2014-01-29T02:27:20 |
+--
2. write protect the image file
openstack@
3. delete the image using "glance image-delete"
openstack@
Request returned failure status.
HTTPForbidden (HTTP 403)
4. Image is logically deleted, but physically remaining
openstack@
+--
| Property | Value |
+--
| checksum | a5282e9259f822c
| container_format | bare |
| created_at | 2014-01-29T02:27:07 |
| deleted | True |
| deleted_at | 2014-01-29T02:30:49 |
| disk_format | qcow2 |
| id | 675a82ab-
| is_public | False |
| min_disk | 0 |
| min_ram | 0 |
| owner | de4a6631051a4df
| protected | False |
| size | 43450368 |
| status | deleted |
| updated_at | 2014-01-29T02:30:49 |
+--
openstack@
total 33088
42432 -rw-r----- 1 glance glance 43450368 Jan 29 11:27 675a82ab-
5. Remove the write-protect from image file
openstack@
6. Try to delete the image again using "glance image-delete"
openstack@
Request returned failure status.
HTTPForbidden (HTTP 403)
7. Image file will not get deleted
openstack@
total 33088
42432 -rw-r----- 1 glance glance 43450368 Jan 29 11:27 675a82ab-
This issue is reproducible only in v1 version. In case of v2 version, it doesn't delete the meta data associated with the image before actually deleting it from the backend store.
In glance, if you enable delayed_delete = True and run glance-scrubber service, glance api will put the image in the queue when the image is deleted by the user and this image will be deleted asynchronously. If it encounters the above reported issue while deleting the image in the glance-scrubber, then this service will simply log error message and keep on retrying deleting the image until it is deleted finally.
Possible solution:
1. When delayed_delete is enabled.
delete_
2. When delayed_delete is not enabled.
delete_
i.e. after upload_
Also the status of the image should be set back to the previous state in the forbidden exception block.
Changed in glance: | |
assignee: | nobody → Abhishek Kekane (abhishek-kekane) |
description: | updated |
description: | updated |
description: | updated |
summary: |
Physical image can not be deleted again if deletion is failed with - OSError for some reason + OSError |
Changed in glance: | |
importance: | Undecided → Medium |
Changed in glance: | |
milestone: | none → icehouse-3 |
status: | Fix Committed → Fix Released |
Changed in glance: | |
milestone: | icehouse-3 → 2014.1 |
Why use want to touch the image repository manually in step 2? Is there any specific requirement?