colo: secondary vm crash during loadvm

[junchi]

Following document 'COLO-FT.txt', I test colo feature on my hosts. It seems goes well. But after a while the secondary vm crash. The stack is as follows:
#0 0x00007f191456dc37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f1914571028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f1914566bf6 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f1914566ca2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x0000564154ad9147 in pcibus_reset (qbus=0x564156760d10) at ../hw/pci/pci.c:311
#5 0x0000564154a07cdb in qbus_reset_one (bus=0x564156760d10, opaque=0x0) at hw/core/qdev.c:319
#6 0x0000564154a0d721 in qbus_walk_children (bus=0x564156760d10, pre_devfn=0, pre_busfn=0,
    post_devfn=0x564154a07c26 <qdev_reset_one>, post_busfn=0x564154a07c6c <qbus_reset_one>, opaque=0x0)
    at hw/core/bus.c:68
#7 0x0000564154a08b4d in qdev_walk_children (dev=0x56415675f2b0, pre_devfn=0, pre_busfn=0,
    post_devfn=0x564154a07c26 <qdev_reset_one>, post_busfn=0x564154a07c6c <qbus_reset_one>, opaque=0x0)
    at hw/core/qdev.c:617
#8 0x0000564154a0d6e5 in qbus_walk_children (bus=0x564156594d30, pre_devfn=0, pre_busfn=0,
    post_devfn=0x564154a07c26 <qdev_reset_one>, post_busfn=0x564154a07c6c <qbus_reset_one>, opaque=0x0)
    at hw/core/bus.c:59
#9 0x0000564154a07df5 in qbus_reset_all (bus=0x564156594d30) at hw/core/qdev.c:336
#10 0x0000564154a07e3a in qbus_reset_all_fn (opaque=0x564156594d30) at hw/core/qdev.c:342
#11 0x0000564154a0e222 in qemu_devices_reset () at hw/core/reset.c:69
#12 0x00005641548b3b47 in pc_machine_reset () at /vms/git/qemu/hw/i386/pc.c:2234
#13 0x0000564154972ca7 in qemu_system_reset (report=false) at vl.c:1697
#14 0x0000564154b9d007 in colo_process_incoming_thread (opaque=0x5641553c1280) at migration/colo.c:617
#15 0x00007f1914907184 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#16 0x00007f1914634bed in clone () from /lib/x86_64-linux-gnu/libc.so.6

(gdb) frame 4
#4 0x0000564154ad9147 in pcibus_reset (qbus=0x564156760d10) at ../hw/pci/pci.c:311
warning: Source file is more recent than executable.
311 assert(bus->irq_count[i] == 0);
(gdb) ^CQuit
(gdb) p bus->irq_count[i]
$1 = -1

[junchi (pork-prince)]

The qemu version is 2.9.0 release.
The 'irq_count' and 'irq_state' are sent by private vm, and loaded by secondary vm. When they sent by private vm, they maybe not in a consistent state. So sometimes 'bus->irq_count[i]' becomes '-1' on secondary vm.
I deleted the assertions and then tested it several times, it worked well

[Dr. David Alan Gilbert (dgilbert-h)]

I haven't tried COLO for a while; I've got a note I hit a similar error in the past -
I think I came to the conclusion it was the e1000 that was unhappy - probably sending an interrupt after it had stopped.
Try a different NIC. I flipped to using a virtio-net at one point (but I think I had to disable vhost, there are some patches for this recently).

[Zhang Chen (zhangckid)]

I agree with David, you can try the RTL8139.

[Thomas Huth (th-huth)]

The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now.
If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Thank you and sorry for the inconvenience.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]