| 2024-04-15 15:00:36 |
macchese |
bug |
|
|
added bug |
| 2024-04-15 15:01:46 |
macchese |
description |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2)
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. |
|
| 2024-04-15 18:21:41 |
macchese |
description |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2)
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2)
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage.
log from destination nova-compute
504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] Instance launched has CPU info: {"arch": "x86_64", "model": "EPYC-IBPB", "vendor": "AMD", "topology": {"cells": 8, "sockets": 1, "co
res": 4, "threads": 2}, "features": ["pclmuldq", "tsc-scale", "clzero", "avx2", "osvw", "sse4a", "perfctr_core", "decodeassists", "3dnowprefetch", "rdrand", "fpu", "lbrv", "syscall", "apic"
, "cx8", "lm", "clflush", "pause-filter", "sha-ni", "sse4.1", "fxsr_opt", "avx", "sse4.2", "pfthreshold", "nrip-save", "lahf_lm", "smap", "xsavec", "cx16", "mtrr", "pge", "aes", "nx", "mmxe
xt", "fxsr", "misalignsse", "xsaveerptr", "fma", "svm-lock", "ssse3", "bmi1", "sse", "mce", "invtsc", "de", "msr", "fsgsbase", "pat", "bmi2", "vmcb-clean", "sse2", "cr8legacy", "vme", "skin
it", "arat", "tce", "npt", "sep", "ibpb", "perfctr_nb", "flushbyasid", "pse36", "mca", "popcnt", "smep", "cmp_legacy", "mmx", "wdt", "pdpe1gb", "tsc", "topoext", "xsaves", "xsave", "abm", "
rdseed", "xgetbv1", "xsaveopt", "pni", "adx", "monitor", "extapic", "svm", "pae", "cmov", "pse", "ht", "rdtscp", "movbe", "f16c", "clflushopt"]}
2024-04-15 18:15:40.776 2182518 ERROR nova.virt.libvirt.driver [None req-c7cbc9ca-4cbc-4679-832f-f07f9854d4b8 f616f0a309e741d7a1a8416852da6692 e175daa0e5464c38b424cfa2e0b457d3 - - 230876c80
d504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] CPU doesn't have compatibility. |
|
| 2024-04-15 18:22:04 |
macchese |
summary |
Live migration fails despite identical CPUs with Host filter ignoring hosts |
Live migration fails despite identical CPUs/capabilities |
|
| 2024-04-16 14:06:33 |
Uggla |
nova: status |
New |
Triaged |
|
| 2024-04-16 14:06:45 |
Uggla |
nova: status |
Triaged |
Incomplete |
|
| 2024-04-16 14:35:52 |
Uggla |
nova: status |
Incomplete |
Confirmed |
|
| 2024-04-16 16:34:23 |
macchese |
nova: status |
Confirmed |
New |
|
| 2024-04-16 17:04:00 |
macchese |
description |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2)
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage.
log from destination nova-compute
504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] Instance launched has CPU info: {"arch": "x86_64", "model": "EPYC-IBPB", "vendor": "AMD", "topology": {"cells": 8, "sockets": 1, "co
res": 4, "threads": 2}, "features": ["pclmuldq", "tsc-scale", "clzero", "avx2", "osvw", "sse4a", "perfctr_core", "decodeassists", "3dnowprefetch", "rdrand", "fpu", "lbrv", "syscall", "apic"
, "cx8", "lm", "clflush", "pause-filter", "sha-ni", "sse4.1", "fxsr_opt", "avx", "sse4.2", "pfthreshold", "nrip-save", "lahf_lm", "smap", "xsavec", "cx16", "mtrr", "pge", "aes", "nx", "mmxe
xt", "fxsr", "misalignsse", "xsaveerptr", "fma", "svm-lock", "ssse3", "bmi1", "sse", "mce", "invtsc", "de", "msr", "fsgsbase", "pat", "bmi2", "vmcb-clean", "sse2", "cr8legacy", "vme", "skin
it", "arat", "tce", "npt", "sep", "ibpb", "perfctr_nb", "flushbyasid", "pse36", "mca", "popcnt", "smep", "cmp_legacy", "mmx", "wdt", "pdpe1gb", "tsc", "topoext", "xsaves", "xsave", "abm", "
rdseed", "xgetbv1", "xsaveopt", "pni", "adx", "monitor", "extapic", "svm", "pae", "cmov", "pse", "ht", "rdtscp", "movbe", "f16c", "clflushopt"]}
2024-04-15 18:15:40.776 2182518 ERROR nova.virt.libvirt.driver [None req-c7cbc9ca-4cbc-4679-832f-f07f9854d4b8 f616f0a309e741d7a1a8416852da6692 e175daa0e5464c38b424cfa2e0b457d3 - - 230876c80
d504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] CPU doesn't have compatibility. |
charm bundle: openstack-2023.2
nova-compute: 28.0.1 channel 2023.2/stable
ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors).
ubuntu@juju:~/juju/controller$ openstack hypervisor list
+----+---------------------+-----------------+--------------+-------+
| ID | Hypervisor Hostname | Hypervisor Type | Host IP | State |
+----+---------------------+-----------------+--------------+-------+
| 7 | op1.maas | QEMU | xxxxxxxxxxxx | up |
| 8 | op4.maas | QEMU | xxxxxxxxxxxx | up |
+----+---------------------+-----------------+--------------+-------+
1)microcode is the same:
root@op1:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
root@op4:~# virsh capabilities |grep microcode
<microcode version='134222446'/>
2) libvirt capabilities features are identical:
root@op1:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
root@op4:~# virsh capabilities |grep feature
<feature name='ht'/>
<feature name='monitor'/>
<feature name='osxsave'/>
<feature name='xsaves'/>
<feature name='cmp_legacy'/>
<feature name='extapic'/>
<feature name='skinit'/>
<feature name='wdt'/>
<feature name='tce'/>
<feature name='topoext'/>
<feature name='perfctr_core'/>
<feature name='perfctr_nb'/>
<feature name='invtsc'/>
<feature name='clzero'/>
<feature name='xsaveerptr'/>
<feature name='npt'/>
<feature name='lbrv'/>
<feature name='svm-lock'/>
<feature name='nrip-save'/>
<feature name='tsc-scale'/>
<feature name='vmcb-clean'/>
<feature name='flushbyasid'/>
<feature name='decodeassists'/>
<feature name='pause-filter'/>
<feature name='pfthreshold'/>
<migration_features>
</migration_features>
<features>
</features>
<features>
</features>
3) NUMA idem
root@op1:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
root@op4:~# lscpu | grep -i numa
NUMA node(s): 8
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56
NUMA node1 CPU(s): 2,10,18,26,34,42,50,58
NUMA node2 CPU(s): 4,12,20,28,36,44,52,60
NUMA node3 CPU(s): 6,14,22,30,38,46,54,62
NUMA node4 CPU(s): 1,9,17,25,33,41,49,57
NUMA node5 CPU(s): 3,11,19,27,35,43,51,59
NUMA node6 CPU(s): 5,13,21,29,37,45,53,61
NUMA node7 CPU(s): 7,15,23,31,39,47,55,63
) live-migration
openstack server list --all-project
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+-------------------+--------------------------+---------+
| 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini |
#openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398
No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2)
log from nova-cloud-controller (debug mode) here:
https://paste.ubuntu.com/p/kF3kmcXtQH/
where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage.
log from destination nova-compute
504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] Instance launched has CPU info: {"arch": "x86_64", "model": "EPYC-IBPB", "vendor": "AMD", "topology": {"cells": 8, "sockets": 1, "co
res": 4, "threads": 2}, "features": ["pclmuldq", "tsc-scale", "clzero", "avx2", "osvw", "sse4a", "perfctr_core", "decodeassists", "3dnowprefetch", "rdrand", "fpu", "lbrv", "syscall", "apic"
, "cx8", "lm", "clflush", "pause-filter", "sha-ni", "sse4.1", "fxsr_opt", "avx", "sse4.2", "pfthreshold", "nrip-save", "lahf_lm", "smap", "xsavec", "cx16", "mtrr", "pge", "aes", "nx", "mmxe
xt", "fxsr", "misalignsse", "xsaveerptr", "fma", "svm-lock", "ssse3", "bmi1", "sse", "mce", "invtsc", "de", "msr", "fsgsbase", "pat", "bmi2", "vmcb-clean", "sse2", "cr8legacy", "vme", "skin
it", "arat", "tce", "npt", "sep", "ibpb", "perfctr_nb", "flushbyasid", "pse36", "mca", "popcnt", "smep", "cmp_legacy", "mmx", "wdt", "pdpe1gb", "tsc", "topoext", "xsaves", "xsave", "abm", "
rdseed", "xgetbv1", "xsaveopt", "pni", "adx", "monitor", "extapic", "svm", "pae", "cmov", "pse", "ht", "rdtscp", "movbe", "f16c", "clflushopt"]}
2024-04-15 18:15:40.776 2182518 ERROR nova.virt.libvirt.driver [None req-c7cbc9ca-4cbc-4679-832f-f07f9854d4b8 f616f0a309e741d7a1a8416852da6692 e175daa0e5464c38b424cfa2e0b457d3 - - 230876c80
d504b0aa43f982b0f26ebf4 230876c80d504b0aa43f982b0f26ebf4] CPU doesn't have compatibility. |
|
| 2024-04-17 12:16:52 |
Andrew Bonney |
bug |
|
|
added subscriber Andrew Bonney |
| 2024-04-23 11:17:08 |
Elod Illes |
tags |
|
libvirt live-migration |
|
| 2024-04-23 16:44:36 |
Elod Illes |
marked as duplicate |
|
2039803 |
|
