Image not deleted after upload when exceeding image_size_cap

[Benard Okolie]

When specifying a maximum size cap in glance-api.conf using 'image_size_cap', and then trying to upload an image that exceeds that size cap a warning is returned that the image is too large and it is not actually uploaded into the backend, however the entry for the image will stay in the glance database as queued until it is manually deleted

+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2017-10-23T15:33:54Z |
| disk_format | qcow2 |
| id | 83ffbe5a-667b-42ff-a742-fc88bf3132e3 |
| locations | [] |
| min_disk | 0 |
| min_ram | 0 |
| name | cap_test |
| owner | db03a56c279b4c9d83bc897a2221725a |
| protected | False |
| size | None |
| status | queued |
| tags | [] |
| updated_at | 2017-10-23T15:33:54Z |
| virtual_size | None |
| visibility | private |
+------------------+--------------------------------------+
413 Request Entity Too Large: Image exceeds the storage quota: The size of the data None will exceed the limit. None bytes remaining. (HTTP 413)

=================================================================================

+------------------+------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------+
| checksum | None |
| container_format | bare |
| created_at | 2017-10-23T15:33:54Z |
| disk_format | qcow2 |
| file | /v2/images/83ffbe5a-667b-42ff-a742-fc88bf3132e3/file |
| id | 83ffbe5a-667b-42ff-a742-fc88bf3132e3 |
| min_disk | 0 |
| min_ram | 0 |
| name | cap_test |
| owner | db03a56c279b4c9d83bc897a2221725a |
| properties | locations='[]' |
| protected | False |
| schema | /v2/schemas/image |
| size | None |
| status | queued |
| tags | |
| updated_at | 2017-10-23T15:34:10Z |
| virtual_size | None |
| visibility | private |
+------------------+------------------------------------------------------+

This behaviour is undesirable. When a user attempts to upload an image that exceeds the size cap the entry should not be added to the glance image database, and should not appear in 'glance image-list'.

[Brian Rosmaita (brian-rosmaita)]

Could you please clarify this bug? I may be misreading this, but the introductory paragraph and the concluding paragraph seem to contradict each other:

Intro:
> a warning is returned that the image is too large and it is not actually uploaded into the backend, however the entry for the image will stay in the glance database as queued until it is manually deleted

Conclusion:
> when a user attempts to upload an image that exceeds the size cap the entry should not be added/autoremoved without manual intervention from the user

[Benard Okolie (benokolie)]

Apologies, the conclusion should read:

when a user attempts to upload an image that exceeds the size cap the entry should not be added to the glance image database. I will amend the bug description to remove any confusion.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/516401

[Brian Rosmaita (brian-rosmaita)]

@Bernard: Thanks for the clarification.

I have some concerns about whether this is a bug. Here's why.

Image creation in the v2 API is a two step process: (1) you create an image record, which is created in status == queued and gives you an image_id to work with, and (2) you upload the image data to v2/images/{image_id}/file. Under the interoperable image import process, it's three steps: (1) create image record, (2) upload data to v2/images/{image_id}/stage, (3) post an import call.

(The multiple step process is hidden in the CLI, where there is a one-liner to do everything in a single call.)

If we look at your proposed fix with respect to its impact on the REST API, you have a call to a sub-resource (PUT v2/images/{image_id}/file) whose failure deletes the entire resource (that is, v2/images/{image_id} is now gone). This does not seem to be correct.

Additionally, there's the possibility of users in a project collaborating to upload images, where one user creates the image record and sets a bunch of metadata, and another user does the data upload. It's not obvious to me that if the person who's doing the upload fails, the person who created the record should have all his work undone.

Now, you may say: this is a non-recoverable case because image_size_cap has been exceeded. But suppose the reason for exceeding the size cap is that the second user forgot to compress the image before uploading. If the image stays in 'queued' status, the second user has another chance to upload the compressed data and possibly succeed without undoing all the first user's work.

My conclusion is that we shouldn't make this change at the API level. It's possible that it could be handled in the CLI (maybe the single-step image-create could clean up after itself), but I'd need to think about that some more.

I suggest that you put discussion of this issue on the agenda for the weekly Glance meeting so that we can get wider community input: https://etherpad.openstack.org/p/glance-team-meeting-agenda

Thanks!

[Benard Okolie (benokolie)]

Thank you for the detailed explanation Brian. I have added this bug as a topic of discussion to the next Glance meeting as suggested.

[Brian Rosmaita (brian-rosmaita)]

We discussed this at today's weekly Glance meeting and concluded that this is not something that should be changed at the API layer. See the meeting log for more details: http://eavesdrop.openstack.org/meetings/glance/2017/glance.2017-11-02-14.01.log.html

[OpenStack Infra (hudson-openstack)]

Change abandoned by Brian Rosmaita (<email address hidden>) on branch: master
Review: https://review.openstack.org/516401
Reason: After discussion at the Glance meeting on 2017-11-02, the bug this fixes has been classified as invalid.