live migration will failed if the vm enabled nic multi queue and vcpu is live resize to a max value

Bug #2056513 reported by Jeffrey Zhang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
Undecided
Jeffrey Zhang

Bug Description

reproduce

* launch a 2 cpu vm with multi queue enabled, also configure a max vcpu as bellow

<vcpu placement='static' current='2'>4</vcpu>

* increase the vcpu from 2 to 4

virsh setvcpus XXX 4

* migration the node

nova live-migration XXX

will get following log in nova-compute.log

2024-03-08 11:32:05.795 8 INFO nova.compute.manager [req-1c0a7415-d81b-480a-a1f9-06b9afdb2b55 - - - - -] [None] [instance: 5a317564-9e87-4313-a8a3-c661434c5bc1] During sync_power_state the instance has a pending task (migrating). Skip.
2024-03-08 11:32:06.673 8 INFO nova.compute.manager [req-1c0a7415-d81b-480a-a1f9-06b9afdb2b55 - - - - -] [None] [instance: 5a317564-9e87-4313-a8a3-c661434c5bc1] VM Resumed (Lifecycle Event)
2024-03-08 11:32:06.843 8 ERROR nova.virt.libvirt.driver [-] [instance: 5a317564-9e87-4313-a8a3-c661434c5bc1] Live Migration failure: internal error: qemu unexpectedly closed the monitor: 2024-03-08T03:32:03.383294Z qemu-kvm: warning: CPU(s) not present in any NUMA nodes: CPU 2 [socket-id: 2, core-id: 0, thread-id: 0], CPU 3 [socket-id: 3, core-id: 0, thread-id: 0]
2024-03-08T03:32:03.383370Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config, ability to start up with partial NUMA mappings is obsoleted and will be removed in future
2024-03-08T03:32:05.584824Z qemu-kvm: get_pci_config_device: Bad config data: i=0x9a read: 5 device: 9 cmask: ff wmask: 0 w1cmask:0
2024-03-08T03:32:05.584971Z qemu-kvm: Failed to load PCIDevice:config
2024-03-08T03:32:05.585014Z qemu-kvm: Failed to load virtio-net:virtio
2024-03-08T03:32:05.585072Z qemu-kvm: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-net'
2024-03-08T03:32:05.587982Z qemu-kvm: load of migration failed: Invalid argument: libvirtError: internal error: qemu unexpectedly closed the monitor: 2024-03-08T03:32:03.383294Z qemu-kvm: warning: CPU(s) not present in any NUMA nodes: CPU 2 [socket-id: 2, core-id: 0, thread-id: 0], CPU 3 [socket-id: 3, core-id: 0, thread-id: 0]
2024-03-08 11:32:06.863 8 INFO nova.compute.manager [req-1c0a7415-d81b-480a-a1f9-06b9afdb2b55 - - - - -] [None] [instance: 5a317564-9e87-4313-a8a3-c661434c5bc1] During sync_power_state the instance has a pending task (migrating). Skip.
2024-03-08 11:32:07.019 8 ERROR nova.virt.libvirt.driver [-] [instance: 5a317564-9e87-4313-a8a3-c661434c5bc1] Migration operation has aborted

# root cause

in the dst node, nova will configure the nic multi queue from 2 to 4, which is not the same with original . So qemu failed to start

# fix

use original queue count rather than generate at dst node.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/912082

Changed in nova:
status: New → In Progress
Changed in nova:
assignee: nobody → Jeffrey Zhang (jeffrey4l)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.