Live migration with cpu_shared_set not updating CPU pinning

[Artom Lifshitz]

In pre-Train times, when live migrating an instance without a CPU policy (and therefore without a NUMA topology) to a dest with a vcpu_pin_set, or to a dest with a vcpu_pin_set different from the source, the instance's CPU pinning information was not updated.

Now that CPU resources are a thing, and we use cpu_dedicated_set and cpu_shared_set (the version with the new meaning, not the old cpu_shared_set), this is still a problem. IOW, live migrating a VM with no CPU policy and no NUMA topology from cpu_shared_set=0,1 to cpu_shared_set=2,3 will leave it pinned to CPUs 0,1.

[Yusuf Güngör (yusuf2)]

Hi, is there any plan to fix this?

[Christian Rohmann (christian-rohmann)]

We just ran into this issue when live-migrating lots of VMs to new hardware and observed that the cpu_shared_set (vcpu pinning) remained the same, which rendered 70% of the newly available physical cores completely useless.

But keeping the cpuset unmodified during a live-migration also becomes a big issue in heterogeneous environments with machines with lots of cpu cores and some with fewer. Libvirt likely will not accept a cpuset targeting pCPUs that do not exist on the destination host. Not a problem with cores 0-8 pinned. But a VM running on a machine with 256 cores and a corresponding shared set cannot be live-migrated away anymore.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/903706

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/877772
Committed: https://opendev.org/openstack/nova/commit/5b7dde4379092a40b473e2a45092ba3d13a93846
Submitter: "Zuul (22348)"
Branch: master

commit 5b7dde4379092a40b473e2a45092ba3d13a93846
Author: René Ribaud <email address hidden>
Date: Wed Mar 15 14:49:30 2023 +0100

    Reproducers for bug 1869804

    Live migrating a VM with no CPU policy and no NUMA topology to a host with
    cpu_shared_set configured will not update VM's configuration accordingly.

    Example: live migrating a VM from source host with cpu_shared_set=0,1 to
    destination host with cpu_shared_set=2,3 will leave the VM configuration
    pinned to CPUs 0,1 (<vcpu cpuset="0-1"> instead of <vcpu cpuset="2-3">).

    This patch adds reproducers for live migrated instances and various
    combinations of cpu_shared_set configuration.
    - From a host with cpu_shared_set to a host with different cpu_shared_set.
    - From a host with cpu_shared_set to a host without cpu_shared_set.
    - From a host without cpu_shared_set to a host with cpu_shared_set.

    This also adds the required changes to the libvirt fixture to manage
    cpuset inside the vcpu tag.

    Related-Bug: #1869804
    Change-Id: Ib294a9d3c25b9a8548347dbe00416a55db567773

[OpenStack Infra (hudson-openstack)]

Change abandoned by "ribaudr <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/nova/+/877773
Reason: Not a reasonable way to backport this.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/915139

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/903706
Committed: https://opendev.org/openstack/nova/commit/2c3f4f2da541c87c10c0ecef6d006715922c5c84
Submitter: "Zuul (22348)"
Branch: master

commit 2c3f4f2da541c87c10c0ecef6d006715922c5c84
Author: René Ribaud <email address hidden>
Date: Fri Apr 5 15:20:20 2024 +0200

    Fix: migration configuration with cpu_shared_set (object part)

    Live migrating to a host with cpu_shared_set configured will now
    update the VM's configuration accordingly.

    Example: live migrating a VM from source host with cpu_shared_set=0,1
    to destination host with cpu_shared_set=2,3 will now update the
    VM configuration.
    (<vcpu cpuset="0-1"> will be updated to <vcpu cpuset="2-3">).

    This update adds a new field, dst_cpu_shared_set_info, to the
    LibvirtLiveMigrateData object, which requires an increase in the
    object's version. As a result, this patch cannot be backported.

    Related-Bug: #1869804
    Change-Id: I806da0958fe436c989e09a52ca6b6f1bbd25a865

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/915139
Committed: https://opendev.org/openstack/nova/commit/43dadaeb901f47f8d5ff9e818e62bdb32046527e
Submitter: "Zuul (22348)"
Branch: master

commit 43dadaeb901f47f8d5ff9e818e62bdb32046527e
Author: René Ribaud <email address hidden>
Date: Fri Apr 5 15:22:41 2024 +0200

    Fix: migration configuration with cpu_shared_set (libvirt part)

    Live migrating to a host with cpu_shared_set configured will now
    update the VM's configuration accordingly.

    Example: live migrating a VM from source host with cpu_shared_set=0,1
    to destination host with cpu_shared_set=2,3 will now update the
    VM configuration.
    (<vcpu cpuset="0-1"> will be updated to <vcpu cpuset="2-3">).

    Related-Bug: #1869804
    Change-Id: I7c717503eba58088094fac05cb99b276af9a3460