We have several thousands of virtual machines with pc-i440fx-wily machine type. Hypervisors run on ubuntu 16.04 and ubuntu 18.04.
We have several problems when we try to migrate those machines to hypervisors with ubuntu 20.04.
* linux guests migrate OK, but for some weird reason windows guests (with the same XML domain definition) do not. We have the following error:
---
qemu-system-x86_64: Features 0x8000002 unsupported. Allowed features: 0x71000002
qemu-system-x86_64: Failed to load virtio-console:virtio
qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-console
---
I tried to investigate this issue and discovered following things:
- missing feature is VIRTIO_F_ANY_LAYOUT for some of virtio devices
- on xenial and bionic VIRTIO_F_ANY_LAYOUT is enabled for pc-i440fx-wily guests, observe:
---
# virsh qemu-monitor-command some-guest --hmp info qtree | grep any_layout any_layout = true any_layout = true any_layout = false any_layout = true
---
- on focal it is disabled
---
# virsh qemu-monitor-command some-guest2 --hmp info qtree | grep any_layout any_layout = false any_layout = true any_layout = false any_layout = false
---
I tried (helplessly) to compare source code for bionic and focal branches of qemu. Looks like this block code is included for the pc-i440fx-wily in focal branch and this is where any_layout is disabled:
---
GlobalProperty hw_compat_2_3[] = {
{ "virtio-blk-pci", "any_layout", "off" },
{ "virtio-balloon-pci", "any_layout", "off" },
{ "virtio-serial-pci", "any_layout", "off" },
{ "virtio-9p-pci", "any_layout", "off" },
{ "virtio-rng-pci", "any_layout", "off" },
{ TYPE_PCI_DEVICE, "x-pcie-lnksta-dllla", "off" },
{ "migration", "send-configuration", "off" },
{ "migration", "send-section-footer", "off" },
{ "migration", "store-global-state", "off" },
};
---
* also we have another problem that *might* be linked to broken definition of pc-i440fx-wily. I am not sure so I'll just mention it (maybe it will be obvious for someone familiar with source code that this problem is also due to broken definition of pc-i440fx-wily in focal and hence part of the same issue)
So even if migration bionic → focal succeeds, it's impossible to migrate guest back (focal → bionic). The problem is:
---
operation failed: guest CPU doesn't match specification: extra features: arat
---
We have several thousands of virtual machines with pc-i440fx-wily machine type. Hypervisors run on ubuntu 16.04 and ubuntu 18.04.
We have several problems when we try to migrate those machines to hypervisors with ubuntu 20.04.
* linux guests migrate OK, but for some weird reason windows guests (with the same XML domain definition) do not. We have the following error: console: virtio 04.0/virtio- console command some-guest --hmp info qtree | grep any_layout
any_ layout = true
any_ layout = true
any_ layout = false
any_ layout = true command some-guest2 --hmp info qtree | grep any_layout
any_ layout = false
any_ layout = true
any_ layout = false
any_ layout = false balloon- pci", "any_layout", "off" }, serial- pci", "any_layout", "off" }, lnksta- dllla", "off" }, tion", "off" }, footer" , "off" }, global- state", "off" },
---
qemu-system-x86_64: Features 0x8000002 unsupported. Allowed features: 0x71000002
qemu-system-x86_64: Failed to load virtio-
qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:
---
I tried to investigate this issue and discovered following things:
- missing feature is VIRTIO_F_ANY_LAYOUT for some of virtio devices
- on xenial and bionic VIRTIO_F_ANY_LAYOUT is enabled for pc-i440fx-wily guests, observe:
---
# virsh qemu-monitor-
---
- on focal it is disabled
---
# virsh qemu-monitor-
---
I tried (helplessly) to compare source code for bionic and focal branches of qemu. Looks like this block code is included for the pc-i440fx-wily in focal branch and this is where any_layout is disabled:
---
GlobalProperty hw_compat_2_3[] = {
{ "virtio-blk-pci", "any_layout", "off" },
{ "virtio-
{ "virtio-
{ "virtio-9p-pci", "any_layout", "off" },
{ "virtio-rng-pci", "any_layout", "off" },
{ TYPE_PCI_DEVICE, "x-pcie-
{ "migration", "send-configura
{ "migration", "send-section-
{ "migration", "store-
};
---
* also we have another problem that *might* be linked to broken definition of pc-i440fx-wily. I am not sure so I'll just mention it (maybe it will be obvious for someone familiar with source code that this problem is also due to broken definition of pc-i440fx-wily in focal and hence part of the same issue)
So even if migration bionic → focal succeeds, it's impossible to migrate guest back (focal → bionic). The problem is:
---
operation failed: guest CPU doesn't match specification: extra features: arat
---