[Glance] Glance user storage quota bypass #2

[Alexey Galkin]

Based on https://bugs.launchpad.net/mos/+bug/1414685 , but in fact is a another issue. Only Glance setups configured with user_storage_quota are affected.

Affected:
 All version v1 and v2+registry of glance. Only Glance setups configured with user_storage_quota are affected.

Steps to reproduce:

1) Login to controller node by ssh.
2) Change token expiration time in 'keystone.conf' from 3600 seconds at 120.
3) Set 'user_storage_quota = 603979780' in 'glance-api.conf'.
4) Run one of the attached scripts (depending on the using api version).
5) When after token expiration time spending, we need to get a list of images of glance and storage backend, and compare them.

Expected result:

Responses from the glance and storage backend service must be identical.

Actual result:

Glance returns an empty list, while storage backend shows that it has some elements.

[Mike Fedosin (mfedosin)]

Fix for MOS 7.0 on review: https://review.fuel-infra.org/#/c/11914/

[Alexey Galkin (agalkin)]

Thanks, Mike!
Tested on iso #301 (RC4).
Your fix works correctly, good job!

[Vitaly Sedelnik (vsedelnik)]

Per upstream bug the issue affects juno and kilo, so nominating for 6.0-updates and 6.1-updates

[Kairat Kushaev (kkushaev)]

I have added liberty into list of nominations.
Also I have corrected the nomination list: added 7.0 into list of nominations. Unfortunately Mirantis Openstack 7.0 is not available anymore in Milestone targets so I leave it empty because the fix has been applied to 7.0 and 7.0 has been released.

[Kairat Kushaev (kkushaev)]

8.0 already contains the bug fix cause the bug has been merged to upstream before liberty release. So I marked it as won't fix.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexey Khivin <email address hidden> on branch: openstack-ci/fuel-6.0-updates/2014.2
Review: https://review.fuel-infra.org/12665
Reason: merge #8490 before this

[Fuel Devops McRobotson (fuel-devops-robot)]

Change restored by Alexey Khivin <email address hidden> on branch: openstack-ci/fuel-6.0-updates/2014.2
Review: https://review.fuel-infra.org/12665

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/12665
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-6.0-updates/2014.2

Commit: f00f61c1431c793342f31d70d61d2dba648a26e4
Author: Mike Fedosin <email address hidden>
Date: Tue Nov 10 16:04:16 2015

Cleanup chunks for deleted image if token expired.

In patch I47229b366c25367ec1bd48aec684e0880f3dfe60 it was
introduced the logic that if image was deleted during file
upload when we want to update image status from 'saving'
to 'active' it's expected to get Duplicate error and delete
stale chunks after that. But if user's token is expired
there will be Unathorized exception and chunks will stay
in store and clog it.
And when, the upload operation for such an image is
completed the operator configured quota can be exceeded.

This patch fixes the issue of left over chunks for an image
which was deleted from saving status, by correcly handle
auth exceptions from registry server

Closes-bug: #1497984

Change-Id: I17a66eca55bfb83107046910e69c4da01415deec
(cherry picked from commit ec89aaa933e4960ea053a6ad57e397574506a362)

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-5.1.1-updates/2014.1.1
Change author: Mike Fedosin <email address hidden>
Review: https://review.fuel-infra.org/13818

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/13818
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-5.1.1-updates/2014.1.1

Commit: 50c44da2ef20444e51a39f90fc2a801f03d929de
Author: Mike Fedosin <email address hidden>
Date: Wed Nov 11 16:49:03 2015

Cleanup chunks for deleted image if token expired.

In patch I47229b366c25367ec1bd48aec684e0880f3dfe60 it was
introduced the logic that if image was deleted during file
upload when we want to update image status from 'saving'
to 'active' it's expected to get Duplicate error and delete
stale chunks after that. But if user's token is expired
there will be Unathorized exception and chunks will stay
in store and clog it.
And when, the upload operation for such an image is
completed the operator configured quota can be exceeded.

This patch fixes the issue of left over chunks for an image
which was deleted from saving status, by correcly handle
auth exceptions from registry server

Closes-bug: #1497984

Change-Id: I17a66eca55bfb83107046910e69c4da01415deec
(cherry picked from commit ec89aaa933e4960ea053a6ad57e397574506a362)

[Vadim Rovachev (vrovachev)]

Verified on 5.1.1

[Vadim Rovachev (vrovachev)]

Verified on 6.0:
packages:
glance-api,glance-common,glance-registry,python-glance
version:
1:2014.2-fuel6.0~mira20

[TatyanaGladysheva (tgladysheva)]

This scenario is not automatable for MOS 8.0 and above.