Quota code does not respect [api]/instance_list_per_project_cells

[Mohammed Naser]

The function which counts resources using the legacy method involves getting a list of all cell mappings assigned to a specific project:

https://github.com/openstack/nova/blob/575a91ff5be79ac35aef4b61d84c78c693693304/nova/quota.py#L1170-L1209

This code can be very heavy on a database which contains a lot of instances (but not a lot of mappings), potentially scanning millions of rows to gather 1-2 cell mappings. In a single cell environment, it is just extra CPU usage with exactly the same outcome.

The [api]/instance_list_per_project_cells was introduced to workaround this:

https://github.com/openstack/nova/blob/575a91ff5be79ac35aef4b61d84c78c693693304/nova/compute/instance_list.py#L146-L153

However, the quota code does not implement it which means quota count take a big toll on the database server. We should ideally mirror the same behaviour in the quota code.

[melanie witt (melwitt)]

Patch proposed here, not sure why the bot didn't add it:

https://review.opendev.org/728575

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/728575
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ab16946885e68ccdef7222a71cc0ad6f92b10de7
Submitter: Zuul
Branch: master

commit ab16946885e68ccdef7222a71cc0ad6f92b10de7
Author: Mohammed Naser <email address hidden>
Date: Fri May 15 16:49:12 2020 -0400

    Make quotas respect instance_list_per_project_cells

    This option was introduced in order to limit queries to cells which only
    belong to the project we're interacting with. However, The nova quota
    code was not respecting it and therefore it was always trying to get the
    cells assigned to a project even with that option disabled.

    This patch makes the quota code respect that option and adds testing to
    ensure that enabling the option does make sure it doesn't search all
    cells but only the ones for this project.

    Closes-Bug: #1878979
    Change-Id: I2e0d48e799e70d550f912ad8a424c86df3ade3a2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/729086

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/729087

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/729088

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/729086
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=fa2bfac862e9e1db5a98c64a56a933987f857903
Submitter: Zuul
Branch: stable/ussuri

commit fa2bfac862e9e1db5a98c64a56a933987f857903
Author: Mohammed Naser <email address hidden>
Date: Fri May 15 16:49:12 2020 -0400

    Make quotas respect instance_list_per_project_cells

    This option was introduced in order to limit queries to cells which only
    belong to the project we're interacting with. However, The nova quota
    code was not respecting it and therefore it was always trying to get the
    cells assigned to a project even with that option disabled.

    This patch makes the quota code respect that option and adds testing to
    ensure that enabling the option does make sure it doesn't search all
    cells but only the ones for this project.

    Closes-Bug: #1878979
    Change-Id: I2e0d48e799e70d550f912ad8a424c86df3ade3a2
    (cherry picked from commit ab16946885e68ccdef7222a71cc0ad6f92b10de7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/729087
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=91160c423839fc8a385dd28c96f94c2b2cdb02cd
Submitter: Zuul
Branch: stable/train

commit 91160c423839fc8a385dd28c96f94c2b2cdb02cd
Author: Mohammed Naser <email address hidden>
Date: Fri May 15 16:49:12 2020 -0400

    Make quotas respect instance_list_per_project_cells

    This option was introduced in order to limit queries to cells which only
    belong to the project we're interacting with. However, The nova quota
    code was not respecting it and therefore it was always trying to get the
    cells assigned to a project even with that option disabled.

    This patch makes the quota code respect that option and adds testing to
    ensure that enabling the option does make sure it doesn't search all
    cells but only the ones for this project.

    Closes-Bug: #1878979
    Change-Id: I2e0d48e799e70d550f912ad8a424c86df3ade3a2
    (cherry picked from commit ab16946885e68ccdef7222a71cc0ad6f92b10de7)
    (cherry picked from commit fa2bfac862e9e1db5a98c64a56a933987f857903)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/729088
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4413e9bf569c46c1dadd8a729f7fa7f47a948697
Submitter: Zuul
Branch: stable/stein

commit 4413e9bf569c46c1dadd8a729f7fa7f47a948697
Author: Mohammed Naser <email address hidden>
Date: Fri May 15 16:49:12 2020 -0400

    Make quotas respect instance_list_per_project_cells

    This option was introduced in order to limit queries to cells which only
    belong to the project we're interacting with. However, The nova quota
    code was not respecting it and therefore it was always trying to get the
    cells assigned to a project even with that option disabled.

    This patch makes the quota code respect that option and adds testing to
    ensure that enabling the option does make sure it doesn't search all
    cells but only the ones for this project.

    Closes-Bug: #1878979

     Conflicts:
        nova/tests/functional/db/test_quota.py

    NOTE(melwitt): The conflict and difference (import mock) from the
    cherry-picked change are because of the following changes not in Stein:

      Iaffb27bd8c562ba120047c04bb62619c0864f594
      I3ff39d5ed99a68ad8678e5ff62b343f3018b4768

    Change-Id: I2e0d48e799e70d550f912ad8a424c86df3ade3a2
    (cherry picked from commit ab16946885e68ccdef7222a71cc0ad6f92b10de7)
    (cherry picked from commit fa2bfac862e9e1db5a98c64a56a933987f857903)
    (cherry picked from commit 91160c423839fc8a385dd28c96f94c2b2cdb02cd)