Allow use of Placement Aggregates to limit tenant instance placement

[Vern Hart]

A customer requires using filter_tenant_id in host aggregates to limit where a customer can deploy instances. To do this, we require adding limit_tenants_to_placement_aggregate to the [scheduler] section of nova.conf. Additionally, to enable this config directive, we also need enable_isolated_aggregate_filtering to be True (also in [scheduler] section).

In short:

  [scheduler]
  enable_isolated_aggregate_filtering = True
  limit_tenants_to_placement_aggregate = True

This could be implemented as a config directive to the charm (maybe limit_tenants_to_placement_aggregate, boolean, default=False) and, if True, set the above two lines in the [scheduler] section of nova.conf.

[Alex Kavanagh (ajkavanagh)]

Triaged to wishlist as this is a feature request.

> This could be implemented as a config directive to the charm (maybe limit_tenants_to_placement_aggregate, boolean, default=False) and, if True, set the above two lines in the [scheduler] section of nova.conf.

Yes, this is a very reasonable approach. This would be a good "first bug/feature" type piece of work.

[Andre Ruiz (andre-ruiz)]

Just FYI https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1908740 was marked as a duplicate of this one.

[Billy Olsen (billy-olsen)]

Patch proposed at https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/794699

[James Vaughn (jmcvaughn)]

From my testing with Ussuri, `enable_isolated_aggregate_filtering` does not need to be enabled in order for `limit_tenants_to_placement_aggregate` to work. These are two entirely separate settings (explained succinctly below):

- `limit_tenants_to_placement_aggregate` is used to place instances of particular tenants onto particular aggregates, and is detailed in [1] and [2]. The filtering is performed using project IDs.

- `enable_isolated_aggregate_filtering` is used to ensure that *only* instances matching particular traits (whether by flavour or image metadata) can be scheduled to hosts in a particular aggregate. With this set to False, instances without these traits can also be scheduled to hosts in the aggregate. The filtering is done with traits. See [3] and [4].

Note that `placement_aggregate_required_for_tenants` [5] requires that *all* tenants be confined to an aggregate (i.e. they will need to go through the steps detailed in [1] for every tenant). Presumptuously, this is probably not the behaviour that most users want or expect; they probably want to limit a single host or set of hosts to a particular tenant but otherwise allow instances to be freely scheduled. In this case, using `enable_isolated_aggregate_filtering` and following [3] (i.e. using traits) I think is the best approach.

Despite all the above, I see that both this bug and [6] mention `enable_isolated_aggregate_filtering` and `limit_tenants_to_placement_aggregate` together. Hopefully I'm not missing something here. If I haven't, the patch needs to be rejected in order to separate these two settings out.

[1] https://docs.openstack.org/nova/ussuri/admin/aggregates.html#tenant-isolation-with-placement
[2] https://docs.openstack.org/nova/latest/configuration/config.html#scheduler.limit_tenants_to_placement_aggregate
[3] https://docs.openstack.org/nova/latest/reference/isolate-aggregates.html
[4] https://docs.openstack.org/nova/latest/configuration/config.html#scheduler.enable_isolated_aggregate_filtering
[5] https://docs.openstack.org/nova/ussuri/configuration/config.html#scheduler.placement_aggregate_required_for_tenants
[6] https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1908740

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/806126

[Nobuto Murata (nobuto)]

Subscribing ~field-medium.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/806126
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/6c1efee2d085334cb00d0a4fbf69008b99af73f4
Submitter: "Zuul (22348)"
Branch: master

commit 6c1efee2d085334cb00d0a4fbf69008b99af73f4
Author: James Vaughn <email address hidden>
Date: Mon Aug 9 14:09:19 2021 +0100

    Add aggregate filtering and isolation options

    Add limit-tenants-to-placement-aggregate,
    placement-aggregate-required-for-tenants and
    enable-isolated-aggregate-filtering options.

    Closes-Bug: #1920982
    Change-Id: Ibbb0e412d1a821cdb56e447ca88b91afca50c8c6

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-guide/+/837653

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-guide/+/837653
Committed: https://opendev.org/openstack/charm-guide/commit/48d83f19042bcd2cf2d86102652c8ec50669f63e
Submitter: "Zuul (22348)"
Branch: master

commit 48d83f19042bcd2cf2d86102652c8ec50669f63e
Author: Nobuto Murata <email address hidden>
Date: Wed Apr 13 11:55:52 2022 +0900

    release-notes: isolating aggregates

    Closes-Bug: #1920982
    Change-Id: I4fc5b5619b2ffbad407e19c9b79df89f0b7a1ae0

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Felipe Reyes <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/794699
Reason: in favor of https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/806126

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902781

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902678

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902679

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902680

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902781
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/f3596029461fd9a07e06f1d8a50277371c83b1b4
Submitter: "Zuul (22348)"
Branch: stable/xena

commit f3596029461fd9a07e06f1d8a50277371c83b1b4
Author: James Vaughn <email address hidden>
Date: Mon Aug 9 14:09:19 2021 +0100

    Add aggregate filtering and isolation options

    Add limit-tenants-to-placement-aggregate,
    placement-aggregate-required-for-tenants and
    enable-isolated-aggregate-filtering options.

    Resolved Conflicts:
            config.yaml

    Closes-Bug: #1920982
    Change-Id: Ibbb0e412d1a821cdb56e447ca88b91afca50c8c6
    (cherry picked from commit 6c1efee2d085334cb00d0a4fbf69008b99af73f4)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902678
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/a9e0fa82f52d92e2012a69f2f1f2e6026e740abe
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit a9e0fa82f52d92e2012a69f2f1f2e6026e740abe
Author: James Vaughn <email address hidden>
Date: Mon Aug 9 14:09:19 2021 +0100

    Add aggregate filtering and isolation options

    Add limit-tenants-to-placement-aggregate,
    placement-aggregate-required-for-tenants and
    enable-isolated-aggregate-filtering options.

    Resolved Conflicts:
            config.yaml

    Closes-Bug: #1920982
    Change-Id: Ibbb0e412d1a821cdb56e447ca88b91afca50c8c6
    (cherry picked from commit 6c1efee2d085334cb00d0a4fbf69008b99af73f4)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902679
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/eae72a4267f98f52805615cce5dee048f76a95af
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit eae72a4267f98f52805615cce5dee048f76a95af
Author: James Vaughn <email address hidden>
Date: Mon Aug 9 14:09:19 2021 +0100

    Add aggregate filtering and isolation options

    Add limit-tenants-to-placement-aggregate,
    placement-aggregate-required-for-tenants and
    enable-isolated-aggregate-filtering options.

    Resolved Conflicts:
            config.yaml

    Closes-Bug: #1920982
    Change-Id: Ibbb0e412d1a821cdb56e447ca88b91afca50c8c6
    (cherry picked from commit 6c1efee2d085334cb00d0a4fbf69008b99af73f4)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/902680
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/69bfc321533a3db252914e6decdf778b3e04ee5a
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 69bfc321533a3db252914e6decdf778b3e04ee5a
Author: James Vaughn <email address hidden>
Date: Mon Aug 9 14:09:19 2021 +0100

    Add aggregate filtering and isolation options

    Add limit-tenants-to-placement-aggregate,
    placement-aggregate-required-for-tenants and
    enable-isolated-aggregate-filtering options.

    Resolved Conflicts:
            config.yaml

    Closes-Bug: #1920982
    Change-Id: Ibbb0e412d1a821cdb56e447ca88b91afca50c8c6
    (cherry picked from commit 6c1efee2d085334cb00d0a4fbf69008b99af73f4)