Backport spectre/meltdown fixes on qemu for ppc64 into 16.04 and possibly 14.04 LTS releases

[bugproxy]

== Comment: #0 - Satheesh Rajendran <email address hidden> - 2018-04-19 04:26:51 ==
---Problem Description---
Backport spectre/meltdown fixes on qemu for ppc64 into all LTS releases

Contact Information = <email address hidden>

---uname output---
-

Machine Type = power8,power9

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 For pseries guests there are 3 tri-state -machine options/capabilities relating to Spectre/Meltdown mitigation: cap-cfpc, cap-sbbc, cap-ibs, which each correspond to a set of host machine capabilities advertised by the KVM kernel module in new/patched host kernels that can be used to mitigate various aspects of Spectre/Meltdown:

cap-cfpc: Cache Flush on Privilege Change
cap-sbbc: Speculation Barrier Bounds Checking
cap-ibs: Indirect Branch Serialisation

Details can be found here https://www.qemu.org/2018/02/14/qemu-2-11-1-and-spectre-update/

Needed qemu commits:

cb931c2108 target/ppc: Check mask when setting cap_ppc_safe_indirect_branch
4f5b039d2b ppc/spapr-caps: Disallow setting workaround for spapr-cap-ibs
8c5909c419 ppc/spapr-caps: Change migration macro to take full spapr-cap name
c59704b254 target/ppc/spapr: Add H-Call H_GET_CPU_CHARACTERISTICS
4be8d4e7d9 target/ppc/spapr_caps: Add new tristate cap safe_indirect_branch
09114fd817 target/ppc/spapr_caps: Add new tristate cap safe_bounds_check
8f38eaf8f9 target/ppc/spapr_caps: Add new tristate cap safe_cache
6898aed77f target/ppc/spapr_caps: Add support for tristate spapr_capabilities
8acc2ae5e9 target/ppc/kvm: Add cap_ppc_safe_[cache/bounds_check/indirect_branch]

Optional commits to introduce a machine type variant pseries-<release>-sxxm, when used would set/enable the three machine capabilities explained above automatically, if host is capable(host kernel is supported). Bug 166426
813f3cf655 ppc/spapr-caps: Define the pseries-2.12-sxxm machine type
c76c0d3090 ppc/spapr-caps: Convert cap-ibs to custom spapr-cap
aaf265ffde ppc/spapr-caps: Convert cap-sbbc to custom spapr-cap
f27aa81e72 ppc/spapr-caps: Convert cap-cfpc to custom spapr-cap
87175d1bc5 ppc/spapr-caps: Add support for custom spapr_capabilities

Userspace tool common name: qemu-kvm

The userspace tool has the following bit modes: both

Userspace rpm: qemu-kvm

Userspace tool obtained from project website: na

*Additional Instructions for <email address hidden>:
-Attach ltrace and strace of userspace application.

[Christian Ehrhardt  (paelzer)]

Hi,
thanks for breaking this out of bug 1761372 (which was about the new machine type in 18.04).

I personally Nack the backport of the machine type changes, but the last call will be the Security Team. Thank you a lot for listing them split.

Also the security Team likely has the best overview of the progress of the related kernel patches.
If you have the list at hand (or a bug where they were processed) it would be surely a great help to list the reference here as well.

Assigning to mdeslaur for initial triage by Server team.

[Christian Ehrhardt  (paelzer)]

s/Server team/Security team/ in the last sentence

[Emily Ratliff (emilyr)]

This is a tricky backport. Would IBM be able to help with this backport?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-06-18 23:22 EDT-------
Is a list of the relevant kernel patches still required for this bug?

What help is required for this backport?

[Andrew Cloke (andrew-cloke)]

From discussions with Emily, I believe the Security team are looking for a complete backport that cleanly applies to the qemu versions in 14.04 and 16.04.

Emily, please correct me if I've got this wrong.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-07-17 15:43 EDT-------
Each company might be waiting on each other for this one.
To clarify: Is Canonical's request for IBM to provide complete backports for qemu in versions 14.04 and 16.04, supplemental information to the qemu commit list provided earlier, or something else?
Thanks.

[Frank Heimes (fheimes)]

Yes, the Security team asked for doing the complete backports, that can then be applied to qemu-kvm 14.04 and 16.04.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-07-23 01:57 EDT-------
Happy to do the backport.
But can someone point me to where I can find the relevant ubuntu qemu trees the patches need to apply to?

[Dimitri John Ledkov (xnox)]

git clone https://git.launchpad.net/ubuntu/+source/qemu

Should give you all ubuntu releases of qemu for all series, with distro-patches applied and without.

I think the best starting points for you to use would be on top of origin/applied/ubuntu/bionic-devel and on top of origin/applied/ubuntu/xenial-devel.

If you can produce a `$ git format-patch` patch series that apply cleanly on top of those trees, it should be sufficiently easy enough for server team to integrate.

Note I don't think the above git tree shares a common history with upstream qemu repository, however, cherrypicking commit ranges should still work correctly.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-07-30 22:03 EDT-------
Taking a look at bionic-devel, it looks like all the relevant patches have already been backported as of 13 Jun.

commit 88bd38f84c0ec61729f2502cc87613186e36da81
Author: Christian Ehrhardt <email address hidden>
Date: Wed Jun 13 10:41:34 2018 +0200

[PATCH] ppc/spapr-caps: Define the pseries-2.12-sxxm machine type
Gbp-Pq: ubuntu/lp-1761372-7-ppc-spapr-caps-Define-the-pseries-2.12-sxxm-machine-.patch.

I will prepare a patch set for the xenial-devel branch.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-08-01 02:08 EDT-------
Patches backported to xenial-devel and pushed here:
https://github.com/sjitindarsingh/qemu/tree/ubuntu-xenial-devel

[Dimitri John Ledkov (xnox)]

marking qemu package tasks to affect xenial series only then.

[Marc Deslauriers (mdeslaur)]

Thanks for the backported commits. I have prepared a test package that includes them in the following PPA:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Please confirm it works properly on the relevant hardware, and I will include the fixes in the next round of qemu security updates.

Thanks!

[Marc Deslauriers (mdeslaur)]

Hi,

Has someone from IBM had a chance to try the test package listed in comment #13?

Thanks!

[Marc Deslauriers (mdeslaur)]

ping?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-09-21 14:06 EDT-------
(In reply to comment #25)
> ping?

Hitting and blocked with installation failure for Trusty , https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1792948/comments/3.

Pls suggest in case of any alternative ways to test.

Thanks in advance.

Regards,
-Satheesh

[Marc Deslauriers (mdeslaur)]

The test package is for Xenial, not for Trusty. Does that installation failure affect Xenial also?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-09-25 13:55 EDT-------
(In reply to comment #27)
> The test package is for Xenial, not for Trusty. Does that installation
> failure affect Xenial also?

my bad, mistaken it for Trusty.

Xenial installation has no issues, and tested the qemu package with below level

HW: Power8
FW: latest contains spectre/meltdown fixes(FW860.60 (SV860_174) (b))
# cd /sys/firmware/devicetree/base/ibm,opal/fw-features
# ls -1 */enabled
fw-branch-hints-honored/enabled
fw-count-cache-disabled/enabled
inst-l1d-flush-ori30,30,0/enabled
inst-spec-barrier-ori31,31,0/enabled
needs-l1d-flush-msr-hv-1-to-0/enabled
needs-l1d-flush-msr-pr-0-to-1/enabled
needs-spec-barrier-for-bound-checks/enabled
speculation-policy-favor-security/enabled
tm-suspend-mode/enabled

Host Kernel: 4.15.0-34-generic

qemu:
ri qemu-system-ppc 1:2.5+dfsg-5ubuntu10.32~test1 ppc64el QEMU full system emulation binaries (ppc)
ii qemu-kvm 1:2.5+dfsg-5ubuntu10.32~test1 ppc64el QEMU Full virtualization

guest: ubuntu18.10(4.17.0-9-generic)

# kvm -M pseries,cap-sbbc=workaround,cap-cfpc=workaround -smp 4 -m 8192 -serial mon:stdio -enable-kvm -vga none -nographic ~/ubuntu1810-ppc64le.qcow2
qemu-system-ppc64: Requested safe cache capability level not supported by kvm, try cap-cfpc=broken

#
looks like some patches missing backport?

probably below patch?, not sure though

https://github.com/qemu/qemu/commit/b2540203bdf4a390c3489146eae82ce237303653#diff-b0796c6a577709f249c7c5d96b9ea049

@Suraj,
Could you pls help here, thanks in advance.

Regards,
-Satheesh.

[Andrew Cloke (andrew-cloke)]

Marking as incomplete while IBM confirms whether an additional qemu patch is required.

[Diane Brent (drbrent)]

IBM to audit the patch list over this week and update LP

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-01-11 01:58 EDT-------
I talked to Michael Roth on this. It looks like everything is there for spectre mitigation but there's two patches to allow power 8 guests to use cap_cfpc=workaround:

072f416a53: target/ppc: Don't require private l1d cache on POWER8 for cap_ppc_safe_cache
b2540203bd: ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default

Looks like the first one also pre-reqs:
8fea704 target/ppc: Factor out the parsing in kvmppc_get_cpu_characteristics()

Suraj - anything else needed or you suggest there?

Finally, this is, as Michael said, cosmetic, but if you wanted you can pull it in (it's not necessary):
8c5909c419 ppc/spapr-caps: Change migration macro to take full spapr-cap name

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-01-13 19:50 EDT-------
Hi,

8c5909c419 ppc/spapr-caps: Change migration macro to take full spapr-cap name

Was part of my original backport and so should already be present. But as has already been said, this is purely
cosmetic.

The following 3 patches:
8fea704 target/ppc: Factor out the parsing in kvmppc_get_cpu_characteristics()
072f416a53: target/ppc: Don't require private l1d cache on POWER8 for cap_ppc_safe_cache
b2540203bd: ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default

Should be added to allow for cap-cfpc to be set to workaround on POWER8.

That is all I would suggest.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-02-04 04:28 EDT-------
(In reply to comment #35)
> Hi,
>
> 8c5909c419 ppc/spapr-caps: Change migration macro to take full spapr-cap name
>
> Was part of my original backport and so should already be present. But as
> has already been said, this is purely
> cosmetic.
>
>
>
> The following 3 patches:
> 8fea704 target/ppc: Factor out the parsing in
> kvmppc_get_cpu_characteristics()
> 072f416a53: target/ppc: Don't require private l1d cache on POWER8 for
> cap_ppc_safe_cache
> b2540203bd: ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default
>
> Should be added to allow for cap-cfpc to be set to workaround on POWER8.
>
> That is all I would suggest.

Satheesh, let me know if you get a chance to verify these..

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-02-04 05:09 EDT-------
(In reply to comment #35)
> Hi,
>
> 8c5909c419 ppc/spapr-caps: Change migration macro to take full spapr-cap name
>
> Was part of my original backport and so should already be present. But as
> has already been said, this is purely
> cosmetic.
>
>
>
> The following 3 patches:
> 8fea704 target/ppc: Factor out the parsing in
> kvmppc_get_cpu_characteristics()
> 072f416a53: target/ppc: Don't require private l1d cache on POWER8 for
> cap_ppc_safe_cache
> b2540203bd: ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default
>
> Should be added to allow for cap-cfpc to be set to workaround on POWER8.
>
> That is all I would suggest.

Please let know if xenial qemu package has updated with these patches, if not include and let know for further test, Thanks in advance.

Regards,
-Satheesh

[Marc Deslauriers (mdeslaur)]

b2540203bd is problematic with the way the following commit was backported:

https://github.com/sjitindarsingh/qemu/commit/3e2b2a58b4b1a8040f12166117efbb7732749c38

I will need a backported version of b2540203bd to create a test package.

Thanks!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-02-06 18:11 EDT-------
I have backported the 3 patches I mentioned previously and pushed a tree here:

https://github.com/sjitindarsingh/qemu/tree/ubuntu-xenial-devel

Note rework of "b2540203bd: ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default" to account for how previous backport was done.

[Marc Deslauriers (mdeslaur)]

I have uploaded a test package to the security team PPA here:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Unfortunately, it failed to build. I didn't investigate yet.

[Marc Deslauriers (mdeslaur)]

I've uploaded an updated package that adds a missing header and that fixes the build failure:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Please test it, and if it tests successfully, I will add the patches to the next qemu security update.

[Manoj Iyer (manjo)]

Tested PPA with xenial based on the instruction in comment #18.

$ uname -a
Linux dryden 4.15.0-45-generic #48~16.04.1-Ubuntu SMP Tue Jan 29 18:03:36 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

$ apt policy qemu-kvm
qemu-kvm:
  Installed: 1:2.5+dfsg-5ubuntu10.35~test2
  Candidate: 1:2.5+dfsg-5ubuntu10.35~test2
  Version table:
 *** 1:2.5+dfsg-5ubuntu10.35~test2 500
        500 http://ppa.launchpad.net/ubuntu-security-proposed/ppa/ubuntu xenial/main ppc64el Packages
        100 /var/lib/dpkg/status

$ sudo kvm -M pseries,cap-sbbc=workaround,cap-cfpc=workaround -smp 4 -m 8192 -serial mon:stdio -enable-kvm -vga none -nographic bionic-image1.img
WARNING: Image format was not specified for 'bionic-image1.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

SLOF **********************************************************************
QEMU Starting
 Build Date = Dec 21 2017 00:39:26
 FW Version = buildd@ release 20151103
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /vdevice/l-lan@71000002
Populating /vdevice/v-scsi@71000003
       SCSI: Looking for devices
          8000000000000000 DISK : "QEMU QEMU HARDDISK 2.5+"
          8200000000000000 CD-ROM : "QEMU QEMU CD-ROM 2.5+"
Populating /pci@800000020000000
No NVRAM common partition, re-initializing...
Scanning USB
Using default console: /vdevice/vty@71000000

  Welcome to Open Firmware

[Manoj Iyer (manjo)]

IBM, could you please test this PPA as well, since this is a security update the maintainer would like to see an ack from IBM on this bug.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-03-13 02:57 EDT-------
Tested with below levels:

Host Kernel: 4.15.0-46-generic #49~16.04.1-Ubuntu SMP Tue Feb 12 17:43:02 UTC 2019 ppc64le

# cd /sys/firmware/devicetree/base/ibm,opal/fw-features
# ls -1 */enabled
fw-branch-hints-honored/enabled
fw-count-cache-disabled/enabled
inst-l1d-flush-ori30,30,0/enabled
inst-spec-barrier-ori31,31,0/enabled
needs-l1d-flush-msr-hv-1-to-0/enabled
needs-l1d-flush-msr-pr-0-to-1/enabled
needs-spec-barrier-for-bound-checks/enabled
speculation-policy-favor-security/enabled
tm-suspend-mode/enabled

FW: FW860.60 (SV860_180) (b)

# apt policy qemu-kvm
qemu-kvm:
Installed: 1:2.5+dfsg-5ubuntu10.35~test2
Candidate: 1:2.5+dfsg-5ubuntu10.35~test2
Version table:
1:2.5+dfsg-5ubuntu10.35 400
400 http://ports.ubuntu.com/ubuntu-ports xenial-proposed/main ppc64el Packages
*** 1:2.5+dfsg-5ubuntu10.35~test2 500
500 http://ppa.launchpad.net/ubuntu-security-proposed/ppa/ubuntu xenial/main ppc64el Packages
100 /var/lib/dpkg/status
1:2.5+dfsg-5ubuntu10.34 500
500 http://us.ports.ubuntu.com/ubuntu-ports xenial-updates/main ppc64el Packages
1:2.5+dfsg-5ubuntu10.33 500
500 http://ports.ubuntu.com/ubuntu-ports xenial-security/main ppc64el Packages
1:2.5+dfsg-5ubuntu10 500
500 http://us.ports.ubuntu.com/ubuntu-ports xenial/main ppc64el Packages

Guest:

#kvm -M pseries,cap-sbbc=workaround,cap-cfpc=workaround,cap-ibs=fixed-ccd -smp 4 -m 8192 -serial mon:stdio -enable-kvm -vga none -nographic /home/ubuntu-16.04.4-ppc64le.qcow2

SLOF **********************************************************************
QEMU Starting
Build Date = Dec 21 2017 00:39:26
FW Version = buildd@ release 20151103
Press "s" to enter Open Firmware.
...
...
root@guest:~# dmesg|grep rfi
[ 0.000000] rfi-flush: fallback displacement flush available
[ 0.000000] rfi-flush: ori type flush available
[ 0.000000] rfi-flush: mttrig type flush available
[ 0.000000] rfi-flush: patched 10 locations (ori+mttrig type flush)

root@guest:~# uname -a
Linux guest 4.15.0-26-generic #28~16.04.1-Ubuntu SMP Thu Jul 5 09:29:59 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

root@guest:~# cat /sys/kernel/debug/powerpc/rfi_flush
1

Regards,
-Satheesh

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.36

---------------
qemu (1:2.5+dfsg-5ubuntu10.36) xenial-security; urgency=medium

  * Spectre/Meltdown fixes for ppc64 (LP: #1765364)
    - debian/patches/lp1765364/*.patches: add backported capabilities and
      spectre/meltdown commits.
  * SECURITY UPDATE: race during file renaming in v9fs_wstat
    - debian/patches/CVE-2018-19489.patch: add locks to hw/9pfs/9p.c.
    - CVE-2018-19489
  * SECURITY UPDATE: heap based buffer overflow in slirp
    - debian/patches/CVE-2019-6778.patch: check data length while emulating
      ident function in slirp/tcp_subr.c.
    - CVE-2019-6778

 -- Marc Deslauriers <email address hidden> Fri, 22 Mar 2019 14:19:08 -0400

[Manoj Iyer (manjo)]

This bug covers patches to qemu, marking the kernel track as invalid.