Incorrect migration of instances with NUMA/CPU pinning

[Alexander Rubtsov]

Upstream bug: https://bugs.launchpad.net/nova/+bug/1417667

Description:
I configured a flavor with two vcpus and extra specs "hw:cpu_policy=dedicated" in order to enable vcpu pinning.

I booted up a number of instances such that there was one instance affined to host cpus 12 and 13 on compute-0, and another instance affined to cpus 12 and 13 on compute-2. (As reported by "virsh vcpupin" and "virsh dumpxml".)

I then triggered a live migration of one instance from compute-0 to compute-2. This resulted in both instances being affined to host cpus 12 and 13 on compute-2.

The "hw:cpu_policy=dedicated" extra spec is intended to provide dedicated host cpus for the instance. In order to provide this, on a live migration (or cold migration, or rebuild, or evacuation, or resize, etc.) nova needs to ensure that the instance is affined to host cpus that are not currently being used by other instances.

[Alexander Rubtsov (arubtsov)]

sla2 for 7.0-updates

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

version

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Sergey Nikitin (snikitin)]

Do you have problems with migration (if yes, with what migration? cold or life?) or with all of these things? (live migration, cold migration, rebuild, evacuation, resize)

Also the upstream bug has about 12 fixes (3 of them are not merged yet). I think it would be hard and risky to backport all of them because of a lot of dependent code.

[Alexander Rubtsov (arubtsov)]

Sergey,

A customer, who reported this problem, experiences this issue during block migration (nova migrate <uuid>)

[Vitaly Sedelnik (vsedelnik)]

NUMA/CPU pinning is not supported in MOS 7.0, it will be available only in 9.0 as experimental feature.

So Won't Fix for 7.0-updates, Confirmed for 9.0-updates and 10.0

[Roman Podoliaka (rpodolyaka)]

As Sergey pointed out in #3, this is a known limitation in Kilo, Liberty and Mitaka releases of Nova. It's currently being fixed in upstream, but implies a major refactoring of this part of Nova code (12 patches, 3 of which are yet to be merged to master), which means there is no way this will be backported to neither of 7.0, 8.0 or 9.0 MOS releases.

Also, this is a very specific use case (thusm this issue Medium importance) and by definition is not what is usually backported to stable branches.

Moreover, as Vitaly pointed out in #5, MOS never officially supported NUMA related features prior to 9.0 (in which support of those is claimed to be "experimental").

The bottom line is, this is going to be fixed only in Newton (10.0). For all prior releases the fix implies a major refactoring, is rather risky and does not have enough importance to be backported to stable branches (it just can't due to the fact this feature was not officially supported...), thus Won't Fix.

[Alexander Rubtsov (arubtsov)]

Moving back to Nova team, as the clarification regarding migration type had been provided by customer

[Roman Podoliaka (rpodolyaka)]

https://review.fuel-infra.org/#/q/status:merged+project:openstack/nova+branch:9.0/mitaka+topic:bug/1512880 merged - closing as fix committed

[Sergey Novikov (snovikov)]

Verified on snapshot #822 (RC2)