Comment 5 for bug 1978372

Revision history for this message
sean mooney (sean-k-mooney) wrote :

this is kind of an know issue.
the current implementation depend on itertools permutaions.
its expect to be roughly O(kn^2) where n is the number of numa nodes

i had investigated reimplemting this with a different approach a few years ago but
that effort was paused as we wanted to tack numa nodes in placement
https://specs.openstack.org/openstack/nova-specs/specs/victoria/approved/numa-topology-with-rps.html

the current approach really does not scale well beyond 4-8 numa nodes
it was developed assume up to 4

on my laptop your repoducer take 12 minutes

InstanceNUMATopology(cells=[InstanceNUMACell(8),InstanceNUMACell(9),InstanceNUMACell(10),InstanceNUMACell(11),InstanceNUMACell(12),InstanceNUMACell(13),InstanceNUMACell(14)],emulator_threads_policy=None,id=<?>,instance_uuid=<?>)

real 12m32.814s
user 12m15.340s
sys 0m2.704s

quickly replciating that in my alternative poc

https://github.com/SeanMooney/cpu-pinning/blob/master/pinning.py#L378-L415=

~/repos/cpu-pinning on  master [?] via 🐍 v3.8.13 (.venv)
[11:50:39]➜ ./run.sh
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
bug repoducer
----------------------------------------
0:21,1:69,2:22,3:70,4:23,5:71,6:24,7:72,8:25,9:73,10:26,11:74,12:27,13:75,14:28,15:76,16:29,17:77,18:30,19:78,20:31,21:79,22:32,23:80,24:33,25:81,26:34,27:82,28:35,29:83,30:36,31:84,32:37,33:85,34:38,35:86,36:39,37:87,38:40,39:88,40:41,41:89,42:42,43:90,44:43,45:91,46:44,47:92

real 0m1.074s
user 0m0.936s
sys 0m0.115s

there is just a slight performance increase using generators instead.

numa_fit_instance_to_host us a pure function of its inputs

in theory my protype could be refactored to use the real nova objects and we could extend it to the other usecauses that it does not currently support like mixed cpus and PCI devices.

or we could keep both implementation and provide a second numa filter that initaiily only implemented a subset of the features initially.
i.e. only run the existing one if the less strict generitive one found a possible match

dropping in a complete new implementation and backporting that for older branches even if we achieved feature parity is a risk that I'm not sure we would want to take for stable branches but if it was opt in and had such a large performance improvement for operators then maybe its something we should consider.