Activity log for bug #1832622

Date Who What changed Old value New value Message
2019-06-12 17:50:56 bugproxy bug added bug
2019-06-12 17:50:58 bugproxy tags architecture-ppc64le bugnameltc-176932 severity-critical targetmilestone-inin18041
2019-06-12 17:50:59 bugproxy ubuntu: assignee Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
2019-06-12 17:51:02 bugproxy affects ubuntu qemu (Ubuntu)
2019-06-12 19:12:43 Andrew Cloke bug task added ubuntu-power-systems
2019-06-12 19:12:59 Andrew Cloke ubuntu-power-systems: importance Undecided Critical
2019-06-12 19:13:43 Andrew Cloke ubuntu-power-systems: assignee Canonical Server Team (canonical-server)
2019-06-13 06:47:02 Christian Ehrhardt  nominated for series Ubuntu Xenial
2019-06-13 06:47:02 Christian Ehrhardt  bug task added qemu (Ubuntu Xenial)
2019-06-13 06:47:02 Christian Ehrhardt  nominated for series Ubuntu Bionic
2019-06-13 06:47:02 Christian Ehrhardt  bug task added qemu (Ubuntu Bionic)
2019-06-13 06:47:02 Christian Ehrhardt  nominated for series Ubuntu Cosmic
2019-06-13 06:47:02 Christian Ehrhardt  bug task added qemu (Ubuntu Cosmic)
2019-06-13 06:47:02 Christian Ehrhardt  nominated for series Ubuntu Eoan
2019-06-13 06:47:02 Christian Ehrhardt  bug task added qemu (Ubuntu Eoan)
2019-06-13 06:47:02 Christian Ehrhardt  nominated for series Ubuntu Disco
2019-06-13 06:47:02 Christian Ehrhardt  bug task added qemu (Ubuntu Disco)
2019-06-13 06:47:10 Christian Ehrhardt  qemu (Ubuntu Xenial): status New Won't Fix
2019-06-13 06:47:12 Christian Ehrhardt  qemu (Ubuntu Bionic): status New Triaged
2019-06-13 06:47:14 Christian Ehrhardt  qemu (Ubuntu Cosmic): status New Triaged
2019-06-13 06:47:17 Christian Ehrhardt  qemu (Ubuntu Disco): status New Triaged
2019-06-13 06:47:23 Christian Ehrhardt  qemu (Ubuntu Eoan): status New Triaged
2019-06-13 06:48:32 Launchpad Janitor merge proposal linked https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/368748
2019-06-13 06:49:01 Launchpad Janitor merge proposal linked https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/368749
2019-06-13 06:49:29 Launchpad Janitor merge proposal linked https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/368750
2019-06-13 06:49:55 Launchpad Janitor merge proposal linked https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/368751
2019-06-13 07:03:06 Christian Ehrhardt  description Power9 DD 2.3 CPUs running updated firmware will use a new Spectre v2 mitigation. The new mitigation improves performance of branch heavy workloads, but also requires kernel support in order to be fully secure. Without the kernel support there is a risk of a Spectre v2 attack across a process context switch, though it has not been demonstrated in practice. QEMU portion - platform definition needs to account for this new mitigation action.. so attribute for this needs to be added. In terms of support for virtualisation there are 2 sides, kvm and qemu support. Patch list for each, KVM: 2b57ecd0208f KVM: PPC: Book3S: Add count cache flush parameters to kvmppc_get_cpu_char() This is part of LP1822870 already. QEMU: 8ff43ee404 target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST 399b2896d4 target/ppc/spapr: Add workaround option to SPAPR_CAP_IBS The KVM side is upstream as of v5.1-rc1. The QEMU side is upstream as of v4.0.0-rc0. In terms of migration the state is as follows. In order to specify to the guest to use the count cache flush workaround we use the spapr-cap cap-ibs (indirect branch speculation) with the value workaround. Previously the only valid values were broken, fixed-ibs (indirect branch serialisation) and fixed-ccd (count cache disabled). And add a new cap cap-ccf-assist (count cache flush assist) to specify the availability of the hardware assisted flush variant. Note the the way spapr caps work you can migrate to a host that supports a higher value, but not to one which doesn't support the current value (i.e. only supports lower values). Where for cap-ibs these are defined as: 0 - Broken 1 - Workaround 2 - fixed-ibs 3 - fixed-ccd So the following migrations would be valid for example: broken -> fixed-ccd, broken -> workaround, workaround -> fixed-ccd While the following would be invalid: fixed-ccd -> workaround, workaround ->broken, fixed-ccd -> broken This is done to maintain at least the level of protection specified on the command line on migration. Since the workaround must be communicated to the guest kernel at boot we cannot migrate a guest from a host with fixed-ccd to one with workaround since the guest wouldn't know to do the flush and so would be wholly unprotected. This means that to migrate a guest from 2.2 and before to 2.3 would require the guest to either be have been booted with broken previously, or to be rebooted with workaround specified on the command line which would allow the migration to succeed to a 2.3. == MICHAEL D. ROTH == I've tested a backport of count-cache-flush support consisting of the following patches applied (cleanly) on top of bionic's QEMU 2.11+dfsg-1ubuntu7.14 source: target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST ppc/spapr-caps: Change migration macro to take full spapr-cap name target/ppc/spapr: Add workaround option to SPAPR_CAP_IBS target/ppc: Factor out the parsing in kvmppc_get_cpu_characteristics() The following tests were done using a DD 2.3 Witherspoon machine and the results seem to align with what's expected in the original summary: == enablement tests (using 4.15.0-51-generic in both host and guests) == with cap-ibs=workaround,cap-ccf-assist=on: mdroth@ubuntu:~$ dmesg | grep cache-flush [ 0.000000] count-cache-flush: hardware assisted flush sequence enabled with cap-ibs=workaround,cap-ccf-assist=off: mdroth@ubuntu:~$ dmesg | grep cache-flush [ 0.000000] count-cache-flush: full software flush sequence enabled. with cap-ibs=broken mdroth@ubuntu:~$ dmesg | grep cache-flush [ 0.000000] count-cache-flush: software flush disabled. == migration tests (using 4.15.0-51-generic in both host and guests) == Note that pseries-2.11-sxxm/bionic-sxxm defaults to: smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND; smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND; smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD but SPAPR_CAP_FIXED_CCD is not available on the DD 2.3 system I tested on (no fw-count-cache-disabled/enabled in host fw-features device tree), so I used pseries-2.11-sxxm,cap-ibs=broken as the base-level cross-migration: qemu 2.11+dfsg-1ubuntu7.14 -> 2.11+dfsg-1ubuntu7.14+ccf-backport source: -M bionic-sxxm,cap-ibs=broken target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off expected: warning actual: warning "cap-ibs lower level (0) in incoming stream than on destination (1))" software ccf enabled after reboot? yes target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on expected: warning actual: warning "cap-ccf-assist lower level (0) in incoming stream than on destination (1))" hardware ccf enabled after reboot? yes target: -M bionic-sxxm,cap-ibs=broken expected: success actual: success migration: 2.11+dfsg-1ubuntu7.14+ccf-backport -> 2.11+dfsg-1ubuntu7.14+ccf-backport source: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off expected: success actual: success target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on expected: warning actual: warning "cap-ccf-assist lower level (0) in incoming stream than on destination (1)" hardware ccf enabled after reboot? yes target: -M bionic-sxxm,cap-ibs=broken expected: fail actual: fail "cap-ibs higher level (1) in incoming stream than on destination (0)" source: -M bionic-sxxm,cap-ibs=workaround,ccf-assist=on target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on expected: success actual: success target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off expected: fail actual: fail, "cap-ccf-assist higher level (1) in incoming stream than on destination (0)" target: cap-ibs=broken (expected: fail, actual: ) expected: fail actual: fail "cap-ibs higher level (1) in incoming stream than on destination (0)" "cap-ccf-assist higher level (1) in incoming stream than on destination (0)" Sorry, I forgot that I needed some fix-ups for the 4th/last patch, "target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST". I've gone ahead and posted my git tree, which is based on top of the qemu_2.11+dfsg-1ubuntu7.14 source, so the 4 patches there should apply cleanly. There's are notes in the commit notes on what changes were needed for patch 4. https://github.com/mdroth/qemu/commits/spectre-ccf-ubuntu-bionic-1ubuntu7.14 [Impact] * This belongs to the overall context of spectre mitigations and even more the try to minimize the related performance impacts. On ppc64el there is a new chip revision (DD 2.3) which provides a facility that helps to better mitigate some of this. * Backport the patches that will make the feature (if supported by the HW) will pass the capability to the guest - to allow guests that support the improved mitigation to use it. [Test Case] * Start guests with and without this capability * Check if the capability is guest visible as intented * Check if there are any issues on pre DD2.3 HW * Test migrations (IBM outlined the intented paths that will work below) * The problem with the above (and also the reasons I didn't add a list of commands this time) is that it needs special HW (mentioned DD2.3 revision) of the chips which aren't available to us right now. Due to that testing / verification of this on all releases is on IBM [Regression Potential] * Adding new capabilities usually works fine, there are three common pitfalls which here are the regression potential. - (severe) the code would announce a capability that isn't really available. The guest tries to use it and crashes - (medium) several migration paths especially from systems with the new cap to older (un-updated systems) will fail. But that applies to any "from machine with Feature to machine without that feature" and isn't really a new regression. As outlined by IBM below they even tried to make it somewhat compatible (by being a new value in an existing cap) - (low) the guest will see new caps and or facilities. A really odd guest could stumble due to that (would actually be a guest bug then) Overall all of the above was considered by IBM when developing this and should be ok. For archive wide SRU considerations, this has NO effect on non ppc64el. [Other Info] * n/a --- Power9 DD 2.3 CPUs running updated firmware will use a new Spectre v2 mitigation. The new mitigation improves performance of branch heavy workloads, but also requires kernel support in order to be fully secure. Without the kernel support there is a risk of a Spectre v2 attack across a process context switch, though it has not been demonstrated in practice. QEMU portion - platform definition needs to account for this new mitigation action.. so attribute for this needs to be added. In terms of support for virtualisation there are 2 sides, kvm and qemu support. Patch list for each, KVM: 2b57ecd0208f KVM: PPC: Book3S: Add count cache flush parameters to kvmppc_get_cpu_char() This is part of LP1822870 already. QEMU: 8ff43ee404 target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST 399b2896d4 target/ppc/spapr: Add workaround option to SPAPR_CAP_IBS The KVM side is upstream as of v5.1-rc1. The QEMU side is upstream as of v4.0.0-rc0. In terms of migration the state is as follows. In order to specify to the guest to use the count cache flush workaround we use the spapr-cap cap-ibs (indirect branch speculation) with the value workaround. Previously the only valid values were broken, fixed-ibs (indirect branch serialisation) and fixed-ccd (count cache disabled). And add a new cap cap-ccf-assist (count cache flush assist) to specify the availability of the hardware assisted flush variant. Note the the way spapr caps work you can migrate to a host that supports a higher value, but not to one which doesn't support the current value (i.e. only supports lower values). Where for cap-ibs these are defined as: 0 - Broken 1 - Workaround 2 - fixed-ibs 3 - fixed-ccd So the following migrations would be valid for example: broken -> fixed-ccd, broken -> workaround, workaround -> fixed-ccd While the following would be invalid: fixed-ccd -> workaround, workaround ->broken, fixed-ccd -> broken This is done to maintain at least the level of protection specified on the command line on migration. Since the workaround must be communicated to the guest kernel at boot we cannot migrate a guest from a host with fixed-ccd to one with workaround since the guest wouldn't know to do the flush and so would be wholly unprotected. This means that to migrate a guest from 2.2 and before to 2.3 would require the guest to either be have been booted with broken previously, or to be rebooted with workaround specified on the command line which would allow the migration to succeed to a 2.3. == MICHAEL D. ROTH == I've tested a backport of count-cache-flush support consisting of the following patches applied (cleanly) on top of bionic's QEMU 2.11+dfsg-1ubuntu7.14 source:   target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST   ppc/spapr-caps: Change migration macro to take full spapr-cap name   target/ppc/spapr: Add workaround option to SPAPR_CAP_IBS   target/ppc: Factor out the parsing in kvmppc_get_cpu_characteristics() The following tests were done using a DD 2.3 Witherspoon machine and the results seem to align with what's expected in the original summary: == enablement tests (using 4.15.0-51-generic in both host and guests) == with cap-ibs=workaround,cap-ccf-assist=on:   mdroth@ubuntu:~$ dmesg | grep cache-flush   [ 0.000000] count-cache-flush: hardware assisted flush sequence enabled with cap-ibs=workaround,cap-ccf-assist=off:   mdroth@ubuntu:~$ dmesg | grep cache-flush   [ 0.000000] count-cache-flush: full software flush sequence enabled. with cap-ibs=broken   mdroth@ubuntu:~$ dmesg | grep cache-flush   [ 0.000000] count-cache-flush: software flush disabled. == migration tests (using 4.15.0-51-generic in both host and guests) == Note that pseries-2.11-sxxm/bionic-sxxm defaults to:     smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND;     smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND;     smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD but SPAPR_CAP_FIXED_CCD is not available on the DD 2.3 system I tested on (no fw-count-cache-disabled/enabled in host fw-features device tree), so I used pseries-2.11-sxxm,cap-ibs=broken as the base-level cross-migration: qemu 2.11+dfsg-1ubuntu7.14 -> 2.11+dfsg-1ubuntu7.14+ccf-backport source: -M bionic-sxxm,cap-ibs=broken   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off     expected: warning     actual: warning       "cap-ibs lower level (0) in incoming stream than on destination (1))"     software ccf enabled after reboot? yes   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on     expected: warning     actual: warning       "cap-ccf-assist lower level (0) in incoming stream than on destination (1))"     hardware ccf enabled after reboot? yes   target: -M bionic-sxxm,cap-ibs=broken     expected: success     actual: success migration: 2.11+dfsg-1ubuntu7.14+ccf-backport -> 2.11+dfsg-1ubuntu7.14+ccf-backport source: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off     expected: success     actual: success   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on     expected: warning     actual: warning       "cap-ccf-assist lower level (0) in incoming stream than on destination (1)"     hardware ccf enabled after reboot? yes   target: -M bionic-sxxm,cap-ibs=broken     expected: fail     actual: fail       "cap-ibs higher level (1) in incoming stream than on destination (0)" source: -M bionic-sxxm,cap-ibs=workaround,ccf-assist=on   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=on     expected: success     actual: success   target: -M bionic-sxxm,cap-ibs=workaround,cap-ccf-assist=off     expected: fail     actual: fail, "cap-ccf-assist higher level (1) in incoming stream than on destination (0)"   target: cap-ibs=broken (expected: fail, actual: )     expected: fail     actual: fail       "cap-ibs higher level (1) in incoming stream than on destination (0)"       "cap-ccf-assist higher level (1) in incoming stream than on destination (0)" Sorry, I forgot that I needed some fix-ups for the 4th/last patch, "target/ppc/spapr: Add SPAPR_CAP_CCF_ASSIST". I've gone ahead and posted my git tree, which is based on top of the qemu_2.11+dfsg-1ubuntu7.14 source, so the 4 patches there should apply cleanly. There's are notes in the commit notes on what changes were needed for patch 4. https://github.com/mdroth/qemu/commits/spectre-ccf-ubuntu-bionic-1ubuntu7.14
2019-06-14 13:18:41 Terry Rudd bug added subscriber Terry Rudd
2019-06-17 06:32:46 Frank Heimes ubuntu-power-systems: status New Triaged
2019-07-01 07:51:46 Christian Ehrhardt  tags architecture-ppc64le bugnameltc-176932 severity-critical targetmilestone-inin18041 architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041
2019-07-03 14:51:27 Launchpad Janitor qemu (Ubuntu Eoan): status Triaged Fix Released
2019-07-03 14:51:27 Launchpad Janitor cve linked 2018-16872
2019-07-03 14:51:27 Launchpad Janitor cve linked 2018-19665
2019-07-03 14:51:27 Launchpad Janitor cve linked 2018-20815
2019-07-03 14:51:27 Launchpad Janitor cve linked 2019-3812
2019-07-03 14:51:27 Launchpad Janitor cve linked 2019-5008
2019-07-03 14:51:27 Launchpad Janitor cve linked 2019-6501
2019-07-03 14:51:27 Launchpad Janitor cve linked 2019-6778
2019-07-03 14:51:27 Launchpad Janitor cve linked 2019-9824
2019-07-03 15:02:45 Frank Heimes ubuntu-power-systems: status Triaged In Progress
2019-07-04 06:08:04 Christian Ehrhardt  qemu (Ubuntu Disco): status Triaged Incomplete
2019-07-04 06:08:06 Christian Ehrhardt  qemu (Ubuntu Cosmic): status Triaged Incomplete
2019-07-04 06:08:08 Christian Ehrhardt  qemu (Ubuntu Bionic): status Triaged Incomplete
2019-07-08 13:38:30 Manoj Iyer qemu (Ubuntu Eoan): assignee Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) Canonical Server Team (canonical-server)
2019-07-08 13:38:42 Manoj Iyer qemu (Ubuntu Disco): assignee Canonical Server Team (canonical-server)
2019-07-08 13:38:52 Manoj Iyer qemu (Ubuntu Cosmic): assignee Canonical Server Team (canonical-server)
2019-07-08 13:39:15 Manoj Iyer qemu (Ubuntu Bionic): assignee Canonical Server Team (canonical-server)
2019-07-11 06:40:42 Christian Ehrhardt  qemu (Ubuntu Cosmic): status Incomplete Won't Fix
2019-08-13 12:08:50 Christian Ehrhardt  qemu (Ubuntu Bionic): importance Undecided Low
2019-08-13 12:08:53 Christian Ehrhardt  qemu (Ubuntu Disco): importance Undecided Low
2019-08-13 12:12:12 Frank Heimes ubuntu-power-systems: importance Critical Medium
2019-08-13 12:12:23 Frank Heimes ubuntu-power-systems: assignee Canonical Server Team (canonical-server)
2019-08-20 05:57:14 Matthew Ruffell bug added subscriber Matthew Ruffell
2019-08-21 08:07:34 Andrew Cloke qemu (Ubuntu Bionic): status Incomplete Confirmed
2019-08-21 08:21:00 Christian Ehrhardt  qemu (Ubuntu Disco): status Incomplete Confirmed
2019-08-21 08:21:04 Christian Ehrhardt  qemu (Ubuntu Disco): importance Low High
2019-08-21 08:21:06 Christian Ehrhardt  qemu (Ubuntu Bionic): importance Low High
2019-08-21 12:40:40 bugproxy attachment added Test logs https://bugs.launchpad.net/bugs/1832622/+attachment/5283696/+files/SonOfRugby_U18.04.txt
2019-08-21 14:52:43 Christian Ehrhardt  qemu (Ubuntu Eoan): assignee Canonical Server Team (canonical-server)
2019-08-21 14:52:46 Christian Ehrhardt  qemu (Ubuntu Cosmic): assignee Canonical Server Team (canonical-server)
2019-08-22 08:24:11 Christian Ehrhardt  qemu (Ubuntu Bionic): status Confirmed In Progress
2019-08-22 08:24:13 Christian Ehrhardt  qemu (Ubuntu Disco): status Confirmed In Progress
2019-08-27 09:37:51 Robie Basak qemu (Ubuntu Disco): status In Progress Fix Committed
2019-08-27 09:37:53 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2019-08-27 09:37:56 Robie Basak bug added subscriber SRU Verification
2019-08-27 09:37:59 Robie Basak tags architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-needed verification-needed-disco
2019-08-27 09:38:40 Robie Basak qemu (Ubuntu Bionic): status In Progress Fix Committed
2019-08-27 09:38:44 Robie Basak tags architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-needed verification-needed-disco architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-needed verification-needed-bionic verification-needed-disco
2019-08-27 10:33:17 Andrew Cloke ubuntu-power-systems: status In Progress Fix Committed
2019-08-27 17:22:27 Andrew Cloke tags architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-needed verification-needed-bionic verification-needed-disco architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-done-bionic verification-needed verification-needed-disco
2019-08-29 07:36:07 Andrew Cloke tags architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-done-bionic verification-needed verification-needed-disco architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-done-bionic verification-failed-disco verification-needed
2019-08-30 07:07:49 Christian Ehrhardt  bug task added linux (Ubuntu)
2019-08-30 07:08:34 Christian Ehrhardt  linux (Ubuntu Disco): status New Confirmed
2019-08-30 07:08:50 Christian Ehrhardt  linux (Ubuntu Disco): importance Undecided High
2019-08-30 07:08:58 Christian Ehrhardt  bug task deleted linux (Ubuntu Cosmic)
2019-08-30 07:09:04 Christian Ehrhardt  bug task deleted linux (Ubuntu Eoan)
2019-08-30 07:09:16 Christian Ehrhardt  bug task deleted linux (Ubuntu Xenial)
2019-08-30 07:09:27 Christian Ehrhardt  linux (Ubuntu): status New Fix Released
2019-08-30 07:09:36 Christian Ehrhardt  linux (Ubuntu Bionic): status New Fix Released
2019-08-30 07:17:35 Andrew Cloke ubuntu-power-systems: status Fix Committed Confirmed
2019-09-02 14:09:19 Andrew Cloke ubuntu-power-systems: importance Medium High
2019-09-02 15:48:45 Kleber Sacilotto de Souza linux (Ubuntu Disco): status Confirmed In Progress
2019-09-03 23:50:08 bugproxy attachment added qemu-dd2.3-sanity.txt https://bugs.launchpad.net/bugs/1832622/+attachment/5286595/+files/qemu-dd2.3-sanity.txt
2019-09-04 03:07:36 Khaled El Mously linux (Ubuntu Disco): status In Progress Fix Committed
2019-09-04 06:26:19 Andrew Cloke ubuntu-power-systems: status Confirmed Fix Committed
2019-09-04 15:40:58 Christian Ehrhardt  tags architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-done-bionic verification-failed-disco verification-needed architecture-ppc64le bugnameltc-176932 qemu-19.10 severity-critical targetmilestone-inin18041 verification-done verification-done-bionic verification-done-disco
2019-09-05 10:48:16 Łukasz Zemczak removed subscriber Ubuntu Stable Release Updates Team
2019-09-05 10:51:35 Launchpad Janitor qemu (Ubuntu Bionic): status Fix Committed Fix Released
2019-09-05 10:58:19 Launchpad Janitor qemu (Ubuntu Disco): status Fix Committed Fix Released
2019-10-04 09:48:42 Launchpad Janitor linux (Ubuntu Disco): status Fix Committed Fix Released
2019-10-04 12:21:47 Andrew Cloke ubuntu-power-systems: status Fix Committed Fix Released