Quota not properly enforced during unshelve when [quota]count_usage_from_placement = True

[melanie witt]

When nova is configured to count quota usage from placement [1], there are some behaviors that are different from the legacy quota resource counting.

With legacy quotas, all of an instance's resources remained consumed from a quota perspective while the instance was SHELVED_OFFLOADED. Because of this, there was no need to check quota when doing an unshelve and an unshelve request could not be blocked for quota related reasons. The quota usage remained the same whether the instance was SHELVED_OFFLOADED or not.

With counting quota usage from placement, cores and ram resource usage is counted from placement while instances are counted from the API database. And when an instance is SHELVED_OFFLOADED, it does not have any resource allocations in placement for cores and ram during that time. Because of this, it is possible to go over cores and ram quota after unshelving an instance as new resources will be allocated in placement for the unshelved instance.

The unshelve quota scenario is currently not being properly enforced because there are no quota checks in the scheduling code path, so when the unshelving instance goes through the scheduling process, it is not validated against quota. There needs to be a dedicated quota check for unshelve.

[1] https://docs.openstack.org/nova/latest/admin/quotas.html#quota-usage-from-placement

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/872470

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/872471

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/872470
Committed: https://opendev.org/openstack/nova/commit/427b2cb4d61cdfaf18b2467eb50b3772dffd3def
Submitter: "Zuul (22348)"
Branch: master

commit 427b2cb4d61cdfaf18b2467eb50b3772dffd3def
Author: melanie witt <email address hidden>
Date: Fri Jan 27 00:23:06 2023 +0000

    Reproducer for bug 2003991 unshelving offloaded instance

    This adds test coverage for:

      * Shelve/unshelve offloaded with legacy quota usage
      * Shelve/unshelve offloaded with quota usage from placement
      * Shelve/unshelve offloaded with unified limits
      * Shelve/unshelve with legacy quota usage
      * Shelve/unshelve with quota usage from placement
      * Shelve/unshelve with unified limits

    Related-Bug: #2003991

    Change-Id: Icc9b6366aebba2f8468e2127da7b7e099098513a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/872471
Committed: https://opendev.org/openstack/nova/commit/6f79d6321e7c3edaab2eb911198b7b7f851371b3
Submitter: "Zuul (22348)"
Branch: master

commit 6f79d6321e7c3edaab2eb911198b7b7f851371b3
Author: melanie witt <email address hidden>
Date: Fri Jan 27 03:08:09 2023 +0000

    Enforce quota usage from placement when unshelving

    When [quota]count_usage_from_placement = true or
    [quota]driver = nova.quota.UnifiedLimitsDriver, cores and ram quota
    usage are counted from placement. When an instance is SHELVED_OFFLOADED,
    it will not have allocations in placement, so its cores and ram should
    not count against quota during that time.

    This means however that when an instance is unshelved, there is a
    possibility of going over quota if the cores and ram it needs were
    allocated by some other instance(s) while it was SHELVED_OFFLOADED.

    This fixes a bug where quota was not being properly enforced during
    unshelve of a SHELVED_OFFLOADED instance when quota usage is counted
    from placement. Test coverage is also added for the "recheck" quota
    cases.

    Closes-Bug: #2003991

    Change-Id: I4ab97626c10052c7af9934a80ff8db9ddab82738

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/888508
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/888509
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 28.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 27.2.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/888096
Reason: stable/yoga branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/yoga if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/888095
Reason: stable/yoga branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/yoga if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/888206
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/888205
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

[sean mooney (sean-k-mooney)]

https://review.opendev.org/q/topic:%22bug/2003991%22

note there were bugs backport filed back to train but those branches are now unsupproted.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/nova/+/910009

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/nova/+/910010

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/888204
Reason: stable/victoria branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/888203
Reason: stable/victoria branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/888104
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/888103
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/888100
Reason: stable/xena branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/xena if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/888099
Reason: stable/xena branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/xena if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/888093
Committed: https://opendev.org/openstack/nova/commit/710116f4beadb1553c9ad7991ea69b0e286657d6
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 710116f4beadb1553c9ad7991ea69b0e286657d6
Author: melanie witt <email address hidden>
Date: Fri Jan 27 00:23:06 2023 +0000

    Reproducer for bug 2003991 unshelving offloaded instance

    This adds test coverage for:

      * Shelve/unshelve offloaded with legacy quota usage
      * Shelve/unshelve offloaded with quota usage from placement
      * Shelve/unshelve offloaded with unified limits
      * Shelve/unshelve with legacy quota usage
      * Shelve/unshelve with quota usage from placement
      * Shelve/unshelve with unified limits

    Related-Bug: #2003991

    Change-Id: Icc9b6366aebba2f8468e2127da7b7e099098513a
    (cherry picked from commit 427b2cb4d61cdfaf18b2467eb50b3772dffd3def)
    (cherry picked from commit 004a773a3a286f39889519bb5b2009fb9bf44fb1)

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/zed
Review: https://review.opendev.org/c/openstack/nova/+/888094
Reason: stable/zed branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/zed if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/nova/+/919724