support query-gic-version QMP command

[dann frazier]

[Impact]
TLDR; It is not currently possible to define a libvirt KVM guest that will boot on any arm64 system. This breaks assumptions of higher level tools like nova and virt-install.

ARM systems can only boot KVM guests that have the same GIC (Generic Interrupt Controller) version as the host. GICv2 is the QEMU default but, if your system does not have a GICv2, you need to know your GIC version and pass that down to the QEMU command line (e.g. -M virt,gic_version=3). If you (or your tools) don't know to do that, your guests will just fail to boot on non-GICv2 hosts with the obscure error "Failed to set device address: No such device". Starting with Ubuntu 16.04, we support at least one GICv3-only system (Cavium ThunderX).

I consider this a hardware enablement issue, and seek SRU approval based on that criteria.

Recent changes to QEMU and libvirt have improved this situation. QEMU now exposes a "query-gic-capabilities" QMP interface that can let the caller ask what GIC types are available for guests to use on that host. libvirt can now make use of this QEMU interface, and expose that information to users via the domcapabilities interface. Further, the user can specify a gic version of "host" to have libvirt choose a detected GIC version, or the user can omit the <gic> feature altogether, and libvirt will choose a GIC version supported by the host and update the guest XML appropriately. This allows tools like virt-install and nova to generate GIC-agnostic XML that can boot on any arm64 host.

[Test Case]
Detection. Run this command on an arm64 host:

$ sudo virsh domcapabilities --machine virt --virttype kvm

After the update, you should see a <gic> capability with a list of supported GIC versions that match the host system.

Runtime1. Place the attached gic.xml file in /etc/libvirt/qemu.
$ sudo virsh define /etc/libvirt/qemu/gic.xml
$ sudo virsh start gic

Prior to this update, it will error out with:
error: XML error: malformed gic version: host

After the update, it'll boot using whatever GIC version is available on your system.

Runtime2. On a GICv3 system, edit the gic.xml file and remove the <gic version='host'/> line.
$ sudo virsh define /etc/libvirt/qemu/gic.xml
$ sudo virsh start /etc/libvirt/qemu/gic.xml

Prior to this update, this will fail with an I/O error. After this update, it will succeed, and a <gic version='3'/> setting will be added to the XML.

[Regression Risk]
These patches are all upstream, so risk on this patchset should be low.

The change to both QEMU and libvirt are backwards compatible with the existing versions in xenial. If no GIC version is specified, QEMU still defaults to GICv2. If a GIC is specified w/o a version, libvirt continues to default to GICv2.

[dann frazier (dannf)]

Here's a pull request against the current ubuntu-dev branch to backport this feature:

The following changes since commit 1c264d803ee0a087f33aabca112cedc1b325de7e:

  release (2016-03-16 17:55:28 -0700)

are available in the git repository at:

  git://git.launchpad.net/~dannf/qemu gic-caps

for you to fetch changes up to 4453f7f928c37224523d8783a010839e855f077b:

  Cherrypick upstream patches to support the query-gic-version QMP command (LP: #1566564) (2016-04-05 17:05:23 -0600)

----------------------------------------------------------------
dann frazier (1):
      Cherrypick upstream patches to support the query-gic-version QMP command (LP: #1566564)

 debian/changelog | 7 ++
 debian/patches/arm-enhance-kvm_arm_create_scratch_host_vcpu.patch | 76 +++++++++++++++++
 debian/patches/arm-implement-query-gic-capabilities.patch | 86 +++++++++++++++++++
 debian/patches/arm-qmp-add-query-gic-capabilities-interface.patch | 192 ++++++++++++++++++++++++++++++++++++++++++
 debian/patches/kvm-add-kvm_device_supported-helper-function.patch | 61 ++++++++++++++
 debian/patches/series | 4 +
 6 files changed, 426 insertions(+)
 create mode 100644 debian/patches/arm-enhance-kvm_arm_create_scratch_host_vcpu.patch
 create mode 100644 debian/patches/arm-implement-query-gic-capabilities.patch
 create mode 100644 debian/patches/arm-qmp-add-query-gic-capabilities-interface.patch
 create mode 100644 debian/patches/kvm-add-kvm_device_supported-helper-function.patch

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu12

---------------
qemu (1:2.5+dfsg-5ubuntu12) yakkety; urgency=medium

  * Cherrypick upstream patches to support the query-gic-version QMP command
    (LP: #1566564)

 -- dann frazier <email address hidden> Tue, 05 Apr 2016 16:56:11 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.3.4-1ubuntu4

---------------
libvirt (1.3.4-1ubuntu4) yakkety; urgency=medium

  * Re-enable the upstart job by renaming the file.
  * Include patchby @guessi to continally wait for libvirtd to start when
    using sysvinit or upstart. (LP: #1571209)

 -- Serge Hallyn <email address hidden> Mon, 23 May 2016 13:50:22 -0500

[Brian Murray (brian-murray)]

Hello dann, or anyone else affected,

Accepted qemu into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.5+dfsg-5ubuntu10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello dann, or anyone else affected,

Accepted libvirt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/1.3.1-1ubuntu10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dann frazier (dannf)]

Verification:

GICv3 host (Cavium ThunderX):

ubuntu@cvm3:~$ sudo virsh define /etc/libvirt/qemu/gic.xml
Domain gic defined from /etc/libvirt/qemu/gic.xml

ubuntu@cvm3:~$ sudo virsh start gic
Domain gic started

ubuntu@cvm3:~$ sudo virsh console gic
Connected to domain gic
Escape character is ^]
[ 7.815945] Freeing initrd memory: 20820K (ffff800026e6f000 - ffff8000282c4000)
[ 7.820994] kvm [1]: HYP mode not available
[ 7.824234] futex hash table entries: 256 (order: 3, 32768 bytes)
[ 7.828238] audit: initializing netlink subsys (disabled)

GICv2 Host (APM X-Gene Mustang):
ubuntu@ms10-38-mcdivitta3:~$ sudo virsh define /etc/libvirt/qemu/gic.xml
Domain gic defined from /etc/libvirt/qemu/gic.xml

ubuntu@ms10-38-mcdivitta3:~$ sudo virsh start gic
Domain gic started

ubuntu@ms10-38-mcdivitta3:~$ sudo virsh console gic

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 4.4.0-22-generic (buildd@bos01-arm64-002) (gcc version 5.3.1 20160413 (Ubuntu/Linaro 5.3.1-14ubuntu2) ) #40-Ubuntu SMP Thu May 12 22:04:09 UTC 2016 (Ubuntu 4.4.0-22.40-generic 4.4.8)
[...]

[Martin Pitt (pitti)]

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 1.3.1-1ubuntu10.1

---------------
libvirt (1.3.1-1ubuntu10.1) xenial; urgency=medium

  * Cherry pick patches from upstream to enable detection and automatic
    usage of host-supported GIC versions. (LP: #1566564)

 -- dann frazier <email address hidden> Fri, 03 Jun 2016 14:41:21 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.2

---------------
qemu (1:2.5+dfsg-5ubuntu10.2) xenial; urgency=medium

  * Cherrypick upstream patches to support the query-gic-version QMP command
    (LP: #1566564)

 -- dann frazier <email address hidden> Fri, 03 Jun 2016 14:24:25 -0600