qemu live migration failed

[Lidong Chen]

qemu live migration failed

the dest qemu report this error.

Receiving block device images
Completed 0 %^Mqemu-system-x86_64: block/io.c:1348: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed.

this bug is caused by this patch:
http://git.qemu-project.org/?p=qemu.git;a=commit;h=d35ff5e6b3aa3a706b0aa3bcf11400fac945b67a

rollback this commit, the problem solved.

[Lidong Chen (jemmy858585)]

blk->root->perm is 1 when blk_new_open.

the blk->root->perm is update to 3 during virtio_blk_device_realize.

but after this commit, the blk->root->perm is still 1. and cause bdrv_aligned_pwritev failed.

Breakpoint 1, blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
579 {
(gdb) bt
#0 blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
#1 0x000000000063484b in blkconf_apply_backend_options (conf=0x2de7fd0, readonly=false, resizable=true, errp=0x7fffffffd380) at hw/block/block.c:77
#2 0x00000000004a57bd in virtio_blk_device_realize (dev=0x2de7e30, errp=0x7fffffffd3e0) at /data/qemu/hw/block/virtio-blk.c:931
#3 0x00000000004f688e in virtio_device_realize (dev=0x2de7e30, errp=0x7fffffffd468) at /data/qemu/hw/virtio/virtio.c:2485
#4 0x000000000065806f in device_set_realized (obj=0x2de7e30, value=true, errp=0x7fffffffd6d8) at hw/core/qdev.c:939
#5 0x000000000083aaf5 in property_set_bool (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", opaque=0x2de9660, errp=0x7fffffffd6d8) at qom/object.c:1860
#6 0x0000000000838c46 in object_property_set (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1094
#7 0x000000000083c23f in object_property_set_qobject (obj=0x2de7e30, value=0x2e679e0, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/qom-qobject.c:27
#8 0x0000000000838f9a in object_property_set_bool (obj=0x2de7e30, value=true, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1163
#9 0x00000000007bafac in virtio_blk_pci_realize (vpci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1975
#10 0x00000000007ba966 in virtio_pci_realize (pci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1853
#11 0x000000000071e439 in pci_qdev_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/pci/pci.c:2001
#12 0x00000000007badaa in virtio_pci_dc_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/virtio/virtio-pci.c:1930
#13 0x000000000065806f in device_set_realized (obj=0x2ddf920, value=true, errp=0x7fffffffd9a8) at hw/core/qdev.c:939
#14 0x000000000083aaf5 in property_set_bool (obj=0x2ddf920, v=0x2decfd0, name=0x9b2c0e "realized", opaque=0x2ddf5d0, errp=0x7fffffffd9a8) at qom/object.c:1860
#15 0x0000000000838c46 in object_property_set (obj=0x2ddf920, v=0x2decfd0, name=0x9b2c0e "realized", errp=0x7fffffffd9a8) at qom/object.c:1094
#16 0x000000000083c23f in object_property_set_qobject (obj=0x2ddf920, value=0x2dece90, name=0x9b2c0e "realized", errp=0x7fffffffd9a8) at qom/qom-qobject.c:27
#17 0x0000000000838f9a in object_property_set_bool (obj=0x2ddf920, value=true, name=0x9b2c0e "realized", errp=0x7fffffffd9a8) at qom/object.c:1163
#18 0x00000000005bfcea in qdev_device_add (opts=0x1451320, errp=0x7fffffffda30) at qdev-monitor.c:624
#19 0x00000000005c9662 in device_init_func (opaque=0x0, opts=0x1451320, errp=0x0) at vl.c:2305
#20 0x000000000095f491 in qemu_opts_foreach (list=0xe5bd80, func=0x5c9624 <device_init_func>, opaque=0x0, errp=0x0) at util/qemu-option.c:1114
#21 0x00000000005ce9be in main (argc=46, argv=0x7fffffffdeb8, envp=0x7fffffffe030) at vl.c:4583

[Lidong Chen (jemmy858585)]

Hi Kevin:
   Can you provide some information about the original bug which you want fix?

   the original comment:
   Usually guest devices don't like other writers to the same image, so
   they use blk_set_perm() to prevent this from happening.

   i don't find where the dest qemu will use blk_set_perm during migration.
   but after apply this patch, blkconf_apply_backend_options don't update the
   blk->root->perm.

    Thanks.

On Tue, Apr 11, 2017 at 4:21 PM, Lidong Chen <email address hidden> wrote:
> blk->root->perm is 1 when blk_new_open.
>
> the blk->root->perm is update to 3 during virtio_blk_device_realize.
>
> but after this commit, the blk->root->perm is still 1. and cause
> bdrv_aligned_pwritev failed.
>
> Breakpoint 1, blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
> 579 {
> (gdb) bt
> #0 blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
> #1 0x000000000063484b in blkconf_apply_backend_options (conf=0x2de7fd0, readonly=false, resizable=true, errp=0x7fffffffd380) at hw/block/block.c:77
> #2 0x00000000004a57bd in virtio_blk_device_realize (dev=0x2de7e30, errp=0x7fffffffd3e0) at /data/qemu/hw/block/virtio-blk.c:931
> #3 0x00000000004f688e in virtio_device_realize (dev=0x2de7e30, errp=0x7fffffffd468) at /data/qemu/hw/virtio/virtio.c:2485
> #4 0x000000000065806f in device_set_realized (obj=0x2de7e30, value=true, errp=0x7fffffffd6d8) at hw/core/qdev.c:939
> #5 0x000000000083aaf5 in property_set_bool (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", opaque=0x2de9660, errp=0x7fffffffd6d8) at qom/object.c:1860
> #6 0x0000000000838c46 in object_property_set (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1094
> #7 0x000000000083c23f in object_property_set_qobject (obj=0x2de7e30, value=0x2e679e0, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/qom-qobject.c:27
> #8 0x0000000000838f9a in object_property_set_bool (obj=0x2de7e30, value=true, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1163
> #9 0x00000000007bafac in virtio_blk_pci_realize (vpci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1975
> #10 0x00000000007ba966 in virtio_pci_realize (pci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1853
> #11 0x000000000071e439 in pci_qdev_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/pci/pci.c:2001
> #12 0x00000000007badaa in virtio_pci_dc_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/virtio/virtio-pci.c:1930
> #13 0x000000000065806f in device_set_realized (obj=0x2ddf920, value=true, errp=0x7fffffffd9a8) at hw/core/qdev.c:939
> #14 0x000000000083aaf5 in property_set_bool (obj=0x2ddf920, v=0x2decfd0, name=0x9b2c0e "realized", opaque=0x2ddf5d0, errp=0x7fffffffd9a8) at qom/object.c:1860
> #15 0x0000000000838c46 in object_property_set (obj=0x2ddf920, v=0x2decfd0, name=0x9b2c0e "realized", errp=0x7fffffffd9a8) at qom/object.c:1094
> #16 0x000000000083c23f in object_property_set_qobject (obj=0x2ddf920, value=0x2dece90, name=0x9b2c0e "realized", errp=0x7fffffffd9a8) at qom/qom-qobject.c:27
> #17 0x0000000000838f9a in o...

[Kevin Wolf (kwolf-redhat)]

Am 11.04.2017 um 15:35 hat 858585 jemmy geschrieben:
> Hi Kevin:
> Can you provide some information about the original bug which you want fix?
>
> the original comment:
> Usually guest devices don't like other writers to the same image, so
> they use blk_set_perm() to prevent this from happening.
>
> i don't find where the dest qemu will use blk_set_perm during migration.
> but after apply this patch, blkconf_apply_backend_options don't update the
> blk->root->perm.

Do I understand correctly that this is not simply live migration, but
block live migration (with 'migrate -b')?

Can you please post a backtrace not of the successful case, but of the
failing assertion?

Kevin

> On Tue, Apr 11, 2017 at 4:21 PM, Lidong Chen <email address hidden> wrote:
> > blk->root->perm is 1 when blk_new_open.
> >
> > the blk->root->perm is update to 3 during virtio_blk_device_realize.
> >
> > but after this commit, the blk->root->perm is still 1. and cause
> > bdrv_aligned_pwritev failed.
> >
> > Breakpoint 1, blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
> > 579 {
> > (gdb) bt
> > #0 blk_set_perm (blk=0x14c32b0, perm=3, shared_perm=29, errp=0x7fffffffd380) at block/block-backend.c:579
> > #1 0x000000000063484b in blkconf_apply_backend_options (conf=0x2de7fd0, readonly=false, resizable=true, errp=0x7fffffffd380) at hw/block/block.c:77
> > #2 0x00000000004a57bd in virtio_blk_device_realize (dev=0x2de7e30, errp=0x7fffffffd3e0) at /data/qemu/hw/block/virtio-blk.c:931
> > #3 0x00000000004f688e in virtio_device_realize (dev=0x2de7e30, errp=0x7fffffffd468) at /data/qemu/hw/virtio/virtio.c:2485
> > #4 0x000000000065806f in device_set_realized (obj=0x2de7e30, value=true, errp=0x7fffffffd6d8) at hw/core/qdev.c:939
> > #5 0x000000000083aaf5 in property_set_bool (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", opaque=0x2de9660, errp=0x7fffffffd6d8) at qom/object.c:1860
> > #6 0x0000000000838c46 in object_property_set (obj=0x2de7e30, v=0x2e67a90, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1094
> > #7 0x000000000083c23f in object_property_set_qobject (obj=0x2de7e30, value=0x2e679e0, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/qom-qobject.c:27
> > #8 0x0000000000838f9a in object_property_set_bool (obj=0x2de7e30, value=true, name=0xaf4b53 "realized", errp=0x7fffffffd6d8) at qom/object.c:1163
> > #9 0x00000000007bafac in virtio_blk_pci_realize (vpci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1975
> > #10 0x00000000007ba966 in virtio_pci_realize (pci_dev=0x2ddf920, errp=0x7fffffffd6d8) at hw/virtio/virtio-pci.c:1853
> > #11 0x000000000071e439 in pci_qdev_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/pci/pci.c:2001
> > #12 0x00000000007badaa in virtio_pci_dc_realize (qdev=0x2ddf920, errp=0x7fffffffd7b8) at hw/virtio/virtio-pci.c:1930
> > #13 0x000000000065806f in device_set_realized (obj=0x2ddf920, value=true, errp=0x7fffffffd9a8) at hw/core/qdev.c:939
> > #14 0x000000000083aaf5 in property_set_bool (obj=0x2ddf920, v=0x2decfd0, name=0x9b2c0e "realized", opaque=0x2ddf5d0, errp=0x7fffffffd9a8) at qom/object.c:1860
> > #15 0x0...

[Thomas Huth (th-huth)]

The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now.
If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Or mark it as "Fix Released" if the problem has been solved with a newer version of QEMU already. Thank you and sorry for the inconvenience.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]