Libvirt parallel shutdown is not truly parallel when agent-based shutdown is used

[Aristarkh Zagorodnikov]

Hi,

The parallel shutdown feature that is supported by libvirt doesn't work properly when agent is used to shut down the guest. This happens because the shutdown_guest_async (https://libvirt.org/git/?p=libvirt.git;a=blob;f=tools/libvirt-guests.sh.in;hb=HEAD#l301) expects "virsh shutdown ..." to be asynchronous. While this might be the case when ACPI shutdown mode is used, when using QEMU agent to shut down (we use it to reliably shut down Windows Server 2019 guests which are not shutdown gracefully via ACPI after running for several days), the "virsh shutdown ..." blocks until shutdown is completed, effectively rendering the configured shutdown parallelism useless.

The subject version of Ubuntu is 18.04.3, with 5.3 (official HWE) kernel, related package versions are as follows:
libvirt-clients/bionic-updates,now 4.0.0-1ubuntu8.14
libvirt-daemon/bionic-updates,now 4.0.0-1ubuntu8.14
libvirt-daemon-system/bionic-updates,now 4.0.0-1ubuntu8.14
libvirt0/bionic-updates 4.0.0-1ubuntu8.14
qemu-kvm/bionic-updates 1:2.11+dfsg-1ubuntu7.21
qemu-system-common/bionic-updates 1:2.11+dfsg-1ubuntu7.21
qemu-system-x86/bionic-updates 1:2.11+dfsg-1ubuntu7.21

I understand that 18.04 is quite old, this is not a security problem, etc., but I would like to see this fixed/improved upon at least some time in the future. https://libvirt.org/bugs.html specifically states that when reporting bugs, they should first be reported to distro owners, so I'm doing exactly that.

Thank you for taking a look at this one.

[Bryce Harrington (bryce)]

Thank you, Aristarkh Zagorodnikov, for taking the time to file a bug report.

In addition to 18.04, does this feature similarly misbehave with newer versions of libvirt/qemu? If it's fixed in a newer version then perhaps there is a patch that can be backported. If it still isn't working properly in the latest versions, then it'd likely be worth reporting upstream as well.

Since there is not enough information in your report to begin triage or to differentiate between a local configuration problem and a bug in Ubuntu, I am marking this bug as 'Incomplete'. We would be grateful if you would: provide a more complete description of the problem, explain why you believe this is a bug in Ubuntu rather than a problem specific to your system, and then change the bug status back to 'New'.

[Christian Ehrhardt  (paelzer)]

This will be the same in all versions I have ever seen @Bryce.

Usually in my impression no one really cares about libvirt-guests.sh except bugs driven through Distro bug reports (of the last 9 changes 8 are Ubuntu and one is Suse). And even less for parallel as Ubuntu is the only one defaulting to that.

Thanks for the bug @Aristarkh.
We might need to add some bit there to make it truly async then - but the prio on it will not be high. I'll keep this open until I get back to it unless someone else beats me.

[Aristarkh Zagorodnikov (onyxmaster)]

Bryce, thanks for considering this. Christian, thank you for valuable information on the reality of bugfixing in this area. This is not a stopper for us since we run a small amount of "heavy" VMs per hypervisor. I'll take a look at ways to fix this, but can't promise it would be very soon.

[Christian Ehrhardt  (paelzer)]

This is the place to look at for fixes:
- https://libvirt.org/git/?p=libvirt.git;a=history;f=tools/libvirt-guests.sh.in;h=a881f6266ea4545ccbb017beac88d6e51b80fa24;hb=HEAD
- https://libvirt.org/hacking.html

If you beat me and submit something upstream before I get to this please set me on CC.