[18.10][qemu] virsh dompmwakeup fails to wake VM from dompmsuspend state (kvm)

Bug #1759509 reported by bugproxy on 2018-03-28
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Canonical Server Team
libvirt (Ubuntu)
qemu (Ubuntu)
Christian Ehrhardt 

Bug Description

== Comment: #0 - Balamuruhan S <email address hidden> - 2018-03-26 08:32:44 ==
---Problem Description---

virsh dompmwakeup fails to wake VM from dompmsuspend state, VM state remains running instead of pmsuspended even in suspended state. Guest kernel enters to suspend state and qemu/libvirt are not aware of it.

# tail -f /var/log/syslog
Mar 26 08:22:44 ubuntu systemd[1]: Started User Manager for UID 0.
Mar 26 08:23:45 ubuntu systemd-timesyncd[1213]: Network configuration changed, trying to establish connection.
Mar 26 08:23:45 ubuntu kernel: [ 1802.926114] PM: suspend entry (s2idle)

Machine Type = Boston

---Steps to Reproduce---
 1. Start a VM and ensure it reaches login prompt
2. Ensure qemu-guest-agent is available and running in the VM
3. Suspend the VM to mem using virsh dompmsuspend <domain> mem --duration 0
4. VM enters suspended state, try to wake up the VM using virsh dompmwakeup <domain>
5. Libvirt returns that VM got wokeup but ideally it doesn't

# virsh list --all
 Id Name State
 2 virt-tests-vm1 running

# virsh dompmsuspend virt-tests-vm1 mem
Domain virt-tests-vm1 successfully suspended

# echo $?

After dompmsuspend,
# virsh list --all
 Id Name State
 14 virt-tests-vm1 running
 - nrs shut off

# virsh domstate virt-tests-vm1 --reason
running (booted)

---uname output---

Host Kernel
# uname -a
Linux ltc-boston17 4.15.0-12-generic #13 SMP Thu Mar 22 14:16:58 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux

Guest Kernel:
# uname -a
Linux ubuntu 4.15.0-12-generic #13-Ubuntu SMP Wed Mar 7 21:37:03 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

A debugger is not configured

Contact Information = Balamuruhan S / <email address hidden>

Userspace tool common name: qemu

Userspace rpm:

# dpkg -l | grep qemu
ii ipxe-qemu 1.0.0+git-20180124.fbe8c52d-0ubuntu2 all PXE boot firmware - ROM images for qemu
ii ipxe-qemu-256k-compat-efi-roms 1.0.0+git-20150424.a25a16d-0ubuntu2 all PXE boot firmware - Compat EFI ROM images for qemu
ii qemu-block-extra:ppc64el 1:2.11+dfsg-1ubuntu5 ppc64el extra block backend modules for qemu-system and qemu-utils
ii qemu-kvm 1:2.11+dfsg-1ubuntu5 ppc64el QEMU Full virtualization on x86 hardware
ii qemu-slof 20170724+dfsg-1ubuntu1 all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system-common 1:2.11+dfsg-1ubuntu5 ppc64el QEMU full system emulation binaries (common files)
ii qemu-system-ppc 1:2.11+dfsg-1ubuntu5 ppc64el QEMU full system emulation binaries (ppc)
ii qemu-utils 1:2.11+dfsg-1ubuntu5 ppc64el QEMU utilities

# dpkg -l | grep libvirt
ii libvirt-bin 4.0.0-1ubuntu7 ppc64el programs for the libvirt library
ii libvirt-clients 4.0.0-1ubuntu7 ppc64el Programs for the libvirt library
ii libvirt-daemon 4.0.0-1ubuntu7 ppc64el Virtualization daemon
ii libvirt-daemon-driver-storage-rbd 4.0.0-1ubuntu7 ppc64el Virtualization daemon RBD storage driver
ii libvirt-daemon-system 4.0.0-1ubuntu7 ppc64el Libvirt daemon configuration files
ii libvirt-dev:ppc64el 4.0.0-1ubuntu7 ppc64el development files for the libvirt library
ii libvirt-glib-1.0-0:ppc64el 1.0.0-1 ppc64el libvirt GLib and GObject mapping library
ii libvirt0:ppc64el 4.0.0-1ubuntu7 ppc64el library for interfacing with different virtualization systems
ii python-libvirt 4.0.0-1 ppc64el libvirt Python bindings

The userspace tool has the following bit modes: ppc64le

Userspace tool obtained from project website: na

*Additional Instructions for Balamuruhan S / <email address hidden>:
-Post a private note with access information to the machine that the bug is occuring on.
-Attach ltrace and strace of userspace application.

bugproxy (bugproxy) on 2018-03-28
tags: added: architecture-ppc64le bugnameltc-166048 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → qemu (Ubuntu)
Changed in ubuntu-power-systems:
importance: Undecided → High
status: New → Triaged
assignee: nobody → Canonical Server Team (canonical-server)

------- Comment From <email address hidden> 2018-03-28 06:24 EDT-------
Note from Daniel, who is looking into this bug.


Quick recap: this problem happens because we don't have the wake-up from software suspend implemented in the pseries machine. In fact, at least last time I checked, only the x86 machines implement it.

I've send patches to the QEMU ML about 3 months ago to implement a new API (well, technically a change in an existing API) to detect whether the guest has this capability or not. The idea is to use it in Libvirt to prevent pm-suspend and similar commands to execute at all if the guest can't proper wake up. The patches were left behind along the way, and now people are working towards the RC releases of 2.12 (i.e just critical bug fixes).

I'll re-re-send that patches aiming for 2.13 and see if we can finally get them upstream.

Please let us know as soon as the needed patches got upstream accepted.
Changing this ticket incomplete for now.

Changed in ubuntu-power-systems:
status: Triaged → Incomplete
Manoj Iyer (manjo) on 2018-04-05
Changed in qemu (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → David Britton (davidpbritton)
importance: Undecided → High
tags: added: triage-a

Since 18.10 will likely be 2.12 based it would be nice to get the post 2.12 upstream commit IDs as soon as they are existing as we won't pick it up automagically :-)

summary: - Ubuntu18.04 - [qemu] virsh dompmwakeup fails to wake VM from
- dompmsuspend state (kvm)
+ [qemu] virsh dompmwakeup fails to wake VM from dompmsuspend state (kvm)
Changed in qemu (Ubuntu):
status: New → Incomplete
Manoj Iyer (manjo) on 2018-06-18
summary: - [qemu] virsh dompmwakeup fails to wake VM from dompmsuspend state (kvm)
+ [18.10][qemu] virsh dompmwakeup fails to wake VM from dompmsuspend state
+ (kvm)
tags: added: targetmilestone-inin1810
removed: targetmilestone-inin1804
bugproxy (bugproxy) on 2018-06-18
tags: added: targetmilestone-inin1804
removed: targetmilestone-inin1810

Ping for my question of post 2.12 commit IDs - see comment #3

tags: added: qemu-18.10

------- Comment From <email address hidden> 2018-06-22 05:54 EDT-------

> Ping for my question of post 2.12 commit IDs - see comment #3

We don't have it yet, the review process is still ongoing. I'll post it here as soon as it is pushed upstream.

tags: added: triage-g
removed: triage-a
Manoj Iyer (manjo) wrote :

IBM, 18.10 feature freeze is on the 23rd of Aug, any news on the upstream status of this patch?

Changed in qemu (Ubuntu):
assignee: David Britton (davidpbritton) → nobody
Manoj Iyer (manjo) wrote :

Have these patches made its way upstream? if so could you please post links here?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-12-04 18:02 EDT-------
The current posting is here:

Daniel said there should be a v11 soon, but right now it's marked for qemu 4.0.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-12-20 18:32 EDT-------
The patch series was pushed to QEMU upstream. The commits id are:


This series alone will *not* fix the problem reported here. I've made improvements in the system_wakeup QEMU call (which is invoked by dompmwakeup), which now should report a clearer error in the case of a pseries guest:

(qemu) system_wakeup
wake-up from suspend is not supported by this guest
To fully fix the problem in Libvirt, changes must be made to dompmsuspend so the guest will not go into suspend state if the underlying arch does not have support for it. This is now possible with these QEMU changes I've made. I can do the Libvirt side changes myself after I'm done with another Libvirt/QEMU bug I'm looking into.

I've read through the changes and think they are fine to be added to hour qemu 3.1 currently being prepared for Ubuntu 19.04

If you later on have the libvirt changes to fully evaluate the newly added result on wakeup and they are backportable in a non broken way to our libvirt in the same release as well we can add that as well.

So let me know once you have upstreamed the libvirt changes, I'll add an incomplete task for that and add the qemu portion to my current items.

tags: added: libvirt-19.04 qemu-19.04
removed: qemu-18.10
Changed in libvirt (Ubuntu):
status: New → Incomplete
Changed in qemu (Ubuntu):
status: Incomplete → Triaged
assignee: nobody → Christian Ehrhardt  (paelzer)
Changed in ubuntu-power-systems:
status: Incomplete → Triaged
Andrew Cloke (andrew-cloke) wrote :

Marking as Incomplete while waiting for changes to be upstreamed.

Changed in ubuntu-power-systems:
status: Triaged → Incomplete
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers