Feature: support per-compute CPU|RAM|Disk allocation

[Peter Sabaini]

Beginning in Ocata, there is a behavior change where aggregate-based
overcommit ratios will no longer be honored during scheduling. Instead,
overcommit values must be set on a per compute node basis in nova.conf.[0][1]

To manage this, we would need support in the nova-cloud-controller or nova-compute.

Note that there is a blueprint for Nova which aims to bring back the old behaviour but it's Abandoned [2]

[0] from http://openstack.10931.n7.nabble.com/nova-heads-up-to-users-of-Aggregate-Core-Ram-Disk-Filter-behavior-change-in-gt-Ocata-td145409.html

[1] https://docs.openstack.org/nova/rocky/admin/configuration/schedulers.html#bug-1804125

[2] https://review.openstack.org/#/c/544683/

[Peter Sabaini (peter-sabaini)]

Also see Bug #1804125

[Drew Freiberger (afreiberger)]

We are now experiencing issues with nova-compute hypervisors on bionic-ussuri clouds registering with the Placement API as having a ram_allocation_ratio of 1.5 (nova default) which allows over-allocation of resources.

To reproduce, deploy a base bionic-ussuri cloud with nova-cloud-controller ram-allocation-ratio=0.9, then run (using the openstackclients snap) the following query:

openstack resource provider inventory list $(openstack resource provider list -f value -c uuid | head -1)

+----------------+------------------+----------+----------+----------+-----------+--------+
| resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total |
+----------------+------------------+----------+----------+----------+-----------+--------+
| VCPU | 16.0 | 1 | 56 | 0 | 1 | 56 |
| MEMORY_MB | 1.5 | 1 | 515647 | 16384 | 1 | 515647 |
| DISK_GB | 1.0 | 1 | 365 | 0 | 1 | 365 |
+----------------+------------------+----------+----------+----------+-----------+--------+

[Billy Olsen (billy-olsen)]

Confirmed by reading the associated documentation. This effectively breaks allocation ratios for clouds ocata and newer

[Drew Freiberger (afreiberger)]

As a potential workaround, the cloud operator can update the resource provider inventory database entries for allocation ratios like this:

# list each hypervisor and it's provider uuid
openstack resource provider list
# show current totals and allocation ratios for the given provider
openstack resource provider inventory list <provider uuid>
# update the allocation_ratio for VCPU, MEMORY_MB, or DISK_GB "class" for a given provider
openstack resource provider inventory class set --allocation_ratio $NEW_RATIO --total $CURRENT_TOTAL --max_unit $CURRENT_TOTAL <provider uuid> $CLASS

Unfortunately, --total must be updated while changing allocation_ratio for any class.

If you wanted to change RAM allocation_ratio across all hypervisors, for instance, you might use a loop such as:

new_ratio=0.9
resource_class=MEMORY_MB
reserved_size=512
for prov_uuid in $(openstack resource provider list -f value -c uuid); do
  current_total=$(openstack resource provider inventory show -f value -c total $prov_uuid $resource_class)
  openstack resource provider inventory class set --allocation_ratio $new_ratio --total $current_total --max_unit $current_total --reserved $reserved_size $prov_uuid $resource_class
done

This blueprint seems to suggest this may be reverted periodically, so use this workaround wisely and with caution.

https://specs.openstack.org/openstack/nova-specs/specs/stein/implemented/initial-allocation-ratios.html#manually-set-placement-allocation-ratios-are-overwritten

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/800999

[Billy Olsen (billy-olsen)]

Upon further investigation, this bug is a duplicate of bug #1677223. Marking as such.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Billy Olsen <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/charm-nova-compute/+/800999
Reason: Abandoning this in favor of https://review.opendev.org/c/openstack/charm-nova-compute/+/717888