Great, thanks!
That makes sense as the guest driver is part of the equation here.

So if you are willing to also verify this then (with the guests that
trigger the issue on X->X->B) I think there is nothing against this
anymore.
Do you want me to re-eval in depth and sponsor?

On Wed, Sep 9, 2020 at 1:11 PM Seyeong Kim <email address hidden> wrote:
>
> and the customer said that
> windows 2012r2, 2016 has issue but 2019 ( from their image ) is fine.
> but we don't know what setting is there actually inside those images
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1894772
>
> Title:
>   live migration of windows 2012 r2 instance with virtio balloon driver
>   fails from mitaka to queens.
>
> Status in qemu package in Ubuntu:
>   Fix Released
> Status in qemu source package in Xenial:
>   In Progress
>
> Bug description:
>   [Impact]
>
>   livemigration  of windows 2012 r2 instance with virtio balloon driver
>   from qemu 2.5(mitaka) to qemu 2.11(queens) is not working properly.
>
>   Especially instance keep moving e.g 2.5 -> 2.5 -> 2.11
>
>   Then It shows below msg from the 2nd mitaka node.
>
>   Migration: [ 94 %]error: internal error: qemu unexpectedly closed the monitor: 2020-09-07T07:45:11.799345Z qemu-system-x86_64: warning: Unknown firmware file in legacy mode: etc/msr_feature_control
>   2020-09-07T07:45:12.765618Z qemu-system-x86_64: VQ 2 size 0x80 < last_avail_idx 0x1 - used_idx 0x2
>   2020-09-07T07:45:12.765642Z qemu-system-x86_64: Failed to load virtio-balloon:virtio
>   2020-09-07T07:45:12.765648Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:07.0/virtio-balloon'
>   2020-09-07T07:45:12.766483Z qemu-system-x86_64: load of migration failed: Operation not permitted
>
>   [Test Case]
>
>   Deploy 2 mitaka-staging machines kvm host
>   Deploy 1 queens-staging machines kvm host
>
>   Setting NFS server and client between them.
>
>   Deploy windows 2012r2 guest instance with virtio balloon driver on one
>   of the mitaka host
>
>   Migrate it from mitaka to mitaka (it should be ok )
>   Migrate it from mitaka to queens ( it raises error )
>
>   I can reproduce this issue with baremetal or vm host
>
>   [Regressions]
>   As this patch is qemu related, current instance should be restarted to have this fix.
>   Also, this patch may cause failure of vm starting, migrating related to virtio drivers.
>   Especially Windows guest vm.
>
>   [Others]
>
>   I bisected this issue and found one commit below, and the others are
>   needed for this.
>
>   ####
>   From 4eae2a657d1ff5ada56eb9b4966eae0eff333b0b Mon Sep 17 00:00:00 2001
>   From: Ladi Prosek <email address hidden>
>   Date: Tue, 1 Mar 2016 12:14:03 +0100
>   Subject: [PATCH] balloon: fix segfault and harden the stats queue
>
>   The segfault here is triggered by the driver notifying the stats queue
>   twice after adding a buffer to it. This effectively resets stats_vq_elem
>   back to NULL and QEMU crashes on the next stats timer tick in
>   balloon_stats_poll_cb.
>
>   This is a regression introduced in 51b19ebe4320f3dc, although admittedly
>   the device assumed too much about the stats queue protocol even before
>   that commit. This commit adds a few more checks and ensures that the one
>   stats buffer gets deallocated on device reset.
>
>   Cc: <email address hidden>
>   Signed-off-by: Ladi Prosek <email address hidden>
>   Reviewed-by: Michael S. Tsirkin <email address hidden>
>   Signed-off-by: Michael S. Tsirkin <email address hidden>
>
>   ####
>   From 3eb769fd1cf15f16ca796ab5618efe89b23aa625 Mon Sep 17 00:00:00 2001
>   From: Gerd Hoffmann <email address hidden>
>   Date: Tue, 1 Dec 2015 12:05:14 +0100
>   Subject: [PATCH] virtio-gpu: maintain command queue
>
>   We'll go take out the commands we receive out of the virt queue and put
>   them into a linked list, to decouple virtio queue handling from actual
>   command processing.
>
>   Also move cmd processing to new virtio_gpu_handle_ctrl func, so we can
>   easily kick it from different places.
>
>   Signed-off-by: Gerd Hoffmann <email address hidden>
>
>   ####
>   From 6aa46d8ff1ee7e9ca0c4a54d75c74108bee22124 Mon Sep 17 00:00:00 2001
>   From: Paolo Bonzini <email address hidden>
>   Date: Sun, 31 Jan 2016 11:28:57 +0100
>   Subject: [PATCH] virtio: move VirtQueueElement at the beginning of the structs
>
>   The next patch will make virtqueue_pop/vring_pop allocate memory for
>   the VirtQueueElement. In some cases (blk, scsi, gpu) the device wants
>   to extend VirtQueueElement with device-specific fields and, until now,
>   the place of the VirtQueueElement within the containing struct didn't
>   matter. When allocating the entire block in virtqueue_pop/vring_pop,
>   however, the containing struct must basically be a "subclass" of
>   VirtQueueElement, with the VirtQueueElement as the first field. Make
>   that the case for blk and scsi; gpu is already doing it.
>
>   Signed-off-by: Paolo Bonzini <email address hidden>
>   Reviewed-by: Michael S. Tsirkin <email address hidden>
>   Signed-off-by: Michael S. Tsirkin <email address hidden>
>   Reviewed-by: Cornelia Huck <email address hidden>
>
>
>   ####
>   From 51b19ebe4320f3dcd93cea71235c1219318ddfd2 Mon Sep 17 00:00:00 2001
>   From: Paolo Bonzini <email address hidden>
>   Date: Thu, 4 Feb 2016 16:26:51 +0200
>   Subject: [PATCH] virtio: move allocation to virtqueue_pop/vring_pop
>
>   The return code of virtqueue_pop/vring_pop is unused except to check for
>   errors or 0.  We can thus easily move allocation inside the functions
>   and just return a pointer to the VirtQueueElement.
>
>   The advantage is that we will be able to allocate only the space that
>   is needed for the actual size of the s/g list instead of the full
>   VIRTQUEUE_MAX_SIZE items.  Currently VirtQueueElement takes about 48K
>   of memory, and this kind of allocation puts a lot of stress on malloc.
>   By cutting the size by two or three orders of magnitude, malloc can
>   use much more efficient algorithms.
>
>   The patch is pretty large, but changes to each device are testable
>   more or less independently.  Splitting it would mostly add churn.
>
>   Signed-off-by: Paolo Bonzini <email address hidden>
>   Reviewed-by: Michael S. Tsirkin <email address hidden>
>   Signed-off-by: Michael S. Tsirkin <email address hidden>
>   Reviewed-by: Cornelia Huck <email address hidden>
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1894772/+subscriptions



-- 
Christian Ehrhardt
Staff Engineer, Ubuntu Server
Canonical Ltd