4.15 s390x kernel BUG at /build/linux-Gycr4Z/linux-4.15.0/drivers/block/virtio_blk.c:565!

[Colin Ian King]

[SRU Justification]

== Impact ==
Several helper functions in the s390x code which handle accessing sysfs attributes were missing protection against races. Concurrent access would be able to trigger kernel bugs.

== Fix ==
The following two upstream commits (from v5.0 upstream) will fix the issue:

78b1a52e05c9 virtio/s390: fix race in ccw_io_helper()
2448a299ec41 virtio/s390: avoid race on vcdev->config

== Testcase ==
see below

== Risk of Regression ==
Changes are isolated to architecture code and are verified by running the stress testing, so overall should be low.

uname -a
Linux ckingvm1 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 13:42:17 UTC 2018 s390x s390x s390x GNU/Linux

and same for 4.15.0-29-generic and 4.17.0-8-generic

Steps to reproduce this bug:

git clone git://kernel.ubuntu.com/cking/stress-ng
cd stress-ng
make clean
make

And run with:

./stress-ng --sysfs 0 -t 60

.. wait a few seconds and then:

[ 119.445891] ------------[ cut here ]------------
[ 119.445898] kernel BUG at /build/linux-Gycr4Z/linux-4.15.0/drivers/block/virtio_blk.c:565!
[ 119.446093] illegal operation: 0001 ilc:1 [#3] SMP
[ 119.446100] Modules linked in: binfmt_misc zfs(PO) zunicode(PO) zavl(PO) icp(PO) isofs zcommon(PO) znvpair(PO) spl(O) ghash_s390 prng aes_s390 des_s390 des_generic vfio_ccw sha512_s390 sha256_s390 vfio_mdev sha1_s390 sha_common mdev vfio_iommu_type1 vfio sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables x_tables btrfs zstd_compress zlib_deflate raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 linear virtio_net crc32_vx_s390 virtio_blk
[ 119.446166] CPU: 1 PID: 5420 Comm: stress-ng-sysfs Tainted: P D O 4.15.0-33-generic #36-Ubuntu
[ 119.446168] Hardware name: IBM 2964 N63 400 (KVM/Linux)
[ 119.446170] Krnl PSW : 0000000012d313d3 00000000405835bc (virtblk_cache_type_show+0x82/0x88 [virtio_blk])
[ 119.446177] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[ 119.446194] Krnl GPRS: de6dc5c2779af7d7 000000007ffaba20 0000000000000040 0000000000006545
[ 119.446196] 000003ff800058da 0000000000006546 000000006bf537c0 000000006b60a100
[ 119.446198] 0000000000000000 0000000000690648 000000007cc3de40 000000007a74b000
[ 119.446202] 000003ff80008210 0000000000000000 000003ff800058da 000000007ac1bce8
[ 119.446210] Krnl Code: 000003ff80005912: ebbff0a80004 lmg %r11,%r15,168(%r15)
[ 119.446210] 000003ff80005918: c0f400000560 brcl 15,3ff800063d8
[ 119.446210] #000003ff8000591e: a7f40001 brc 15,3ff80005920
[ 119.446210] >000003ff80005922: 0707 bcr 0,%r7
[ 119.446210] 000003ff80005924: 0707 bcr 0,%r7
[ 119.446210] 000003ff80005926: 0707 bcr 0,%r7
[ 119.446210] 000003ff80005928: c00400000000 brcl 0,3ff80005928
[ 119.446210] 000003ff8000592e: eb6ff0480024 stmg %r6,%r15,72(%r15)
[ 119.446226] Call Trace:
[ 119.446229] ([<000003ff800058da>] virtblk_cache_type_show+0x3a/0x88 [virtio_blk])
[ 119.446234] [<0000000000690684>] dev_attr_show+0x3c/0x80
[ 119.446240] [<0000000000424ab4>] sysfs_kf_seq_show+0xbc/0x1a8
[ 119.446259] [<00000000003b048c>] seq_read+0xec/0x4c8
[ 119.446262] [<00000000003821ea>] vfs_read+0x8a/0x150
[ 119.446274] [<0000000000382786>] SyS_read+0x66/0xe0
[ 119.446278] [<00000000008e3028>] system_call+0xdc/0x2c8
[ 119.446279] Last Breaking-Event-Address:
[ 119.446281] [<000003ff8000591e>] virtblk_cache_type_show+0x7e/0x88 [virtio_blk]
[ 119.446283]
[ 119.446284] ---[ end trace 2c2403d726047e4a ]---

For 4.17.0-8-generic:
[ 25.170715] kernel BUG at drivers/block/virtio_blk.c:574!
[ 25.170795] illegal operation: 0001 ilc:1 [#1] SMP
[ 25.170797] Modules linked in: lttng_statedump(OE) lttng_clock(OE) lttng_lib_ring_buffer(OE) binfmt_misc zfs(PO) zunicode(PO) zavl(PO) icp(PO) isofs zcommon(PO) znvpair(PO) spl(O) ghash_s390 prng aes_s390 des_s390 des_generic sha512_s390 sha256_s390 sha1_s390 sha_common vfio_ccw vfio_mdev mdev vfio_iommu_type1 vfio sch_fq_codel ib_iser rdma_cm iw_cm ib_cm nfsd ib_core auth_rpcgss iscsi_tcp nfs_acl lockd grace libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip_tables x_tables btrfs zstd_compress zlib_deflate raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 linear virtio_net virtio_blk crc32_vx_s390
[ 25.170835] CPU: 0 PID: 5590 Comm: stress-ng-sysfs Tainted: P OE 4.17.0-8-generic #9-Ubuntu
[ 25.170837] Hardware name: IBM 2964 N63 400 (KVM/Linux)
[ 25.170839] Krnl PSW : 0000000005f0c968 0000000026542d57 (virtblk_cache_type_show+0x7c/0x80 [virtio_blk])
[ 25.170846] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[ 25.170849] Krnl GPRS: de6dc5c209bf9e6f 000000007ff91320 0000000000000040 0000000000000891
[ 25.170850] 0000000000000000 0000000000000892 0000000000000000 000000007176e800
[ 25.170852] 000000006f447f00 0000000000684300 000000006f492b40 000000006ca7a000
[ 25.170853] 000003ff80018218 000000007b5e6e40 000003ff8001561a 000000006f447ce8
[ 25.170861] Krnl Code: 000003ff8001564c: ebbff0a80004 lmg %r11,%r15,168(%r15)
[ 25.170861] 000003ff80015652: c0f40000065b brcl 15,3ff80016308
[ 25.170861] #000003ff80015658: a7f40001 brc 15,3ff8001565a
[ 25.170861] >000003ff8001565c: 0707 bcr 0,%r7
[ 25.170861] 000003ff8001565e: 0707 bcr 0,%r7
[ 25.170861] 000003ff80015660: c00400000000 brcl 0,3ff80015660
[ 25.170861] 000003ff80015666: eb6ff0480024 stmg %r6,%r15,72(%r15)
[ 25.170861] 000003ff8001566c: a7f13f80 tmll %r15,16256
[ 25.170878] Call Trace:
[ 25.170881] ([<000003ff8001561a>] virtblk_cache_type_show+0x3a/0x80 [virtio_blk])
[ 25.170888] [<000000000068433c>] dev_attr_show+0x3c/0x78
[ 25.170895] [<000000000042539c>] sysfs_kf_seq_show+0xbc/0x1a0
[ 25.170899] [<00000000003b16b8>] seq_read+0x180/0x4f8
[ 25.170903] [<000000000038433a>] vfs_read+0x8a/0x148
[ 25.170905] [<0000000000384882>] ksys_read+0x62/0xd0
[ 25.170909] [<00000000008db738>] system_call+0xdc/0x2c8
[ 25.170910] Last Breaking-Event-Address:
[ 25.170912] [<000003ff80015658>] virtblk_cache_type_show+0x78/0x80 [virtio_blk]
[ 25.170913]
[ 25.170914] ---[ end trace 14f89544f0f55795 ]---

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1788432

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Colin Ian King (colin-king)]

Maybe related to: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/s390x/l/linux/20180822_104342_a8a5e@/log.gz

[Colin Ian King (colin-king)]

FYI, I've added easier detection of which sysfs file is causing this with commit: http://kernel.ubuntu.com/git/cking/stress-ng.git/commit/?id=690b6392645219702f410318a3c99bee7c4f192b

[Colin Ian King (colin-king)]

I think this is a race in ccw_io_helper(). The stress-ng sysfs stressor is running multiple threaded reads of /sys/devices/css0/0.0.0000/0.0.0000/virtio0/block/vda/cache_type which results in reads via virtio_cread8() and ultimately reads using virtio_ccw_get_config()

I added debug into virtio_cread8 and it returns 1 99% of the time, and sometimes in a threaded read we get it returning garbage, such as 200 in the following trace:

[ 39.767777] virtio_cread8 32 -> 1
[ 39.767933] virtio_cread8 32 -> 1
[ 39.787712] virtio_cread8 32 -> 200
[ 39.787810] kernel BUG at drivers/block/virtio_blk.c:576!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-08-30 06:23 EDT-------
Colin King,

it might also be a QEMU issue. Is there a chance to try a 16.04 host instead of an 18.04 host?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-08-30 08:06 EDT-------
I've tried to reproduce the issue but failed miserably. I installed a fresh Ubuntu 18.04.1 made a cqow image essentially copying the filesystem for the guest. That means I was running the 4.15.0-29-generic kernel (reported as affected) both as host and guest. After an hour of running with 50
hogs (as root) I hit OOM but not the reported bug:

# uname -r
4.15.0-29-generic
# ~/git/stress-ng/stress-ng --sysfs 50 -t 5h
stress-ng: info: [1148] dispatching hogs: 50 sysfs
[ 3993.143284] Out of memory: Kill process 587 (networkd-dispat) score 3 or sacrifice child
[ 3993.143295] Killed process 587 (networkd-dispat) total-vm:99944kB, anon-rss:7792kB, file-rss:2512kB, shmem-rss:0kB
[ 4012.251714] Out of memory: Kill process 619 (libvirtd) score 3 or sacrifice child
[ 4012.251742] Killed process 619 (libvirtd) total-vm:1349072kB, anon-rss:6980kB, file-rss:2728kB, shmem-rss:0kB

Can you provide some more information about your setup (e.g. qemu version, domain xml or qemu command line, etc.)?

[Colin Ian King (colin-king)]

Host information:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

uname -a
Linux s2lp5 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 09:18:59 UTC 2018 s390x s390x s390x GNU/Linux

qemu: 2.5+dfsg-5ubuntu10.30

this was created using uvt:

#!/bin/bash
#
# Quick and dirty script to create temporary guests which are reachable
# from the kernel VPN via uvt-kvm. After creation one has to login once
# via "virsh console <vm-name>" to figure out the ip address. :/

BASEDIR=$(dirname $0)

if [ "$1" = "" ]; then
 echo "$(basename $0) <vm-name>"
 exit 1
fi
VMNAME="$1"
shift
uvt-kvm create --cpu 2 --memory 2048 --disk 20 --password ubuntu \
 --template $BASEDIR/uvt-template.xml $VMNAME arch=s390x "$@"

uvt-template.xml:

<domain type="kvm">
    <os>
        <type arch="s390x" machine="s390-ccw-virtio">hvm</type>
    </os>
    <iothreads>1</iothreads>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>preserve</on_crash>
    <devices>
        <emulator>/usr/bin/qemu-system-s390x</emulator>
        <interface type="direct">
            <source dev="encc000.2719" mode="bridge"/>
            <model type="virtio"/>
        </interface>
        <console type="pty">
            <target type="sclp"/>
        </console>
    </devices>
</domain>

Hope that's enough info

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-08-30 11:27 EDT-------
Ah ok. So you run an 18.04 guest under an 16.04 host?
Does the problem goes away with a newer QEMU? (e.g. from the cloud archive)

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-08-31 05:28 EDT-------
Colin, thanks for the input. Based on Christians realization, that your hypervisor may have been 16.4 all along, and on the qemu version you stated, I managed to reproduce the bug with upstream tag v2.5.0. From there I bisected to commit e32652f7594 "virtio-ccw: respond to READ_STATUS command" (which seems to fix the problem). I still have to figure out the whys tough.

Anyway, you should stop seeing the bug with QEMU 2.8 (or more recent).

Cheers,
Halil

[Colin Ian King (colin-king)]

OK, thanks for the info so far, apologies for being slow to get back to you.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-12-03 10:18 EDT-------
Fixes available on the relevant Linux mailing lists. Connie posted a pull request https://www.spinics.net/lists/kvm/msg175437.html some two months ago, and pinged Michael last week. Hope the fixes will find their way into the mainline kernel soon.

[Dimitri John Ledkov (xnox)]

Looks like this got pulled into mainline:

$ git describe 2448a299ec416a80f699940a86f4a6d9a4f643b1
v4.20-rc5-2-g2448a299ec41

$ git describe 78b1a52e05c9db11d293342e8d6d8a230a04b4e7
v4.20-rc5-3-g78b1a52e05c9

Should we request kernel team to pull these into ubuntu SRUs trees? or have these been alredy been CC'ed for stable trees and are part of point releases anyway?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-02-12 13:23 EDT-------
(In reply to comment #22)
> Looks like this got pulled into mainline:
>
> $ git describe 2448a299ec416a80f699940a86f4a6d9a4f643b1
> v4.20-rc5-2-g2448a299ec41
>
> $ git describe 78b1a52e05c9db11d293342e8d6d8a230a04b4e7
> v4.20-rc5-3-g78b1a52e05c9
>
> Should we request kernel team to pull these into ubuntu SRUs trees? or have
> these been alredy been CC'ed for stable trees and are part of point releases
> anyway?

Yep, CC stable is in place. E.g.: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?h=linux-4.4.y&qt=author&q=Pasic

[Colin Ian King (colin-king)]

Just to confirm that 2448a299ec416a80f699940a86f4a6d9a4f643b1 and 78b1a52e05c9db11d293342e8d6d8a230a04b4e7 are in the Ubuntu 5.0 kernel since Ubuntu-5.0.0-0.1

[Frank Heimes (fheimes)]

Just double checked and can confirm that the commits "virtio/s390: avoid race on vcdev->config" and "virtio/s390: fix race in ccw_io_helper()" landed in disco-proposed kernel "Ubuntu-5.0.0-7.8" (as "2448a29" and "78b1a52").
Hence changing disco entry to Fix Committed.

[Stefan Bader (smb)]

Just checked and Disco 5.0.0-7.8 is now released.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-cosmic' to 'verification-done-cosmic'. If the problem still exists, change the tag 'verification-needed-cosmic' to 'verification-failed-cosmic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Frank Heimes (fheimes)]

Successfully verified on cosmic:

ubuntu@hwe0002:~/stress-ng$ apt-cache policy linux-generic
linux-generic:
  Installed: 4.18.0.18.19
  Candidate: 4.18.0.18.19
  Version table:
 *** 4.18.0.18.19 500
        500 http://us.ports.ubuntu.com/ubuntu-ports cosmic-proposed/main s390x Packages
        100 /var/lib/dpkg/status
     4.18.0.17.18 500
        500 http://us.ports.ubuntu.com/ubuntu-ports cosmic-updates/main s390x Packages
        500 http://ports.ubuntu.com/ubuntu-ports cosmic-security/main s390x Packages
     4.18.0.10.11 500
        500 http://us.ports.ubuntu.com/ubuntu-ports cosmic/main s390x Packages
ubuntu@hwe0002:~/stress-ng$ uname -r
4.18.0-18-generic
ubuntu@hwe0002:~/stress-ng$ ./stress-ng --sysfs 0 -t 60
stress-ng: info: [11889] dispatching hogs: 4 sysfs
stress-ng: info: [11889] successful run completed in 60.00s (1 min, 0.00 secs)
ubuntu@hwe0002:~/stress-ng$

and on bionic:
ubuntu@hwe0007:~/stress-ng$ apt-cache policy linux-generic
linux-generic:
  Installed: 4.15.0.48.50
  Candidate: 4.15.0.48.50
  Version table:
 *** 4.15.0.48.50 500
        500 http://us.ports.ubuntu.com/ubuntu-ports bionic-proposed/main s390x Packages
        100 /var/lib/dpkg/status
     4.15.0.47.49 500
        500 http://us.ports.ubuntu.com/ubuntu-ports bionic-updates/main s390x Packages
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main s390x Packages
     4.15.0.20.23 500
        500 http://us.ports.ubuntu.com/ubuntu-ports bionic/main s390x Packages
ubuntu@hwe0007:~/stress-ng$ uname -r
4.15.0-48-generic
ubuntu@hwe0007:~/stress-ng$ ./stress-ng --sysfs 0 -t 60
stress-ng: info: [9386] dispatching hogs: 4 sysfs
stress-ng: info: [9386] successful run completed in 60.00s (1 min, 0.00 secs)
ubuntu@hwe0007:~/stress-ng$

Adjusting tags accordingly...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.18.0-18.19

---------------
linux (4.18.0-18.19) cosmic; urgency=medium

  * linux: 4.18.0-18.19 -proposed tracker (LP: #1822796)

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts
    - [Packaging] resync retpoline extraction

  * 3b080b2564287be91605bfd1d5ee985696e61d3c in ubuntu_btrfs_kernel_fixes
    triggers system hang on i386 (LP: #1812845)
    - btrfs: raid56: properly unmap parity page in finish_parity_scrub()

  * [SRU][B/C/OEM]IOMMU: add kernel dma protection (LP: #1820153)
    - ACPI / property: Allow multiple property compatible _DSD entries
    - PCI / ACPI: Identify untrusted PCI devices
    - iommu/vt-d: Force IOMMU on for platform opt in hint
    - iommu/vt-d: Do not enable ATS for untrusted devices
    - thunderbolt: Export IOMMU based DMA protection support to userspace
    - iommu/vt-d: Disable ATS support on untrusted devices

  * Huawei Hi1822 NIC has poor performance (LP: #1820187)
    - net-next: hinic: fix a problem in free_tx_poll()
    - hinic: remove ndo_poll_controller
    - net-next/hinic: add checksum offload and TSO support
    - hinic: Fix l4_type parameter in hinic_task_set_tunnel_l4
    - net-next/hinic:replace multiply and division operators
    - net-next/hinic:add rx checksum offload for HiNIC
    - net-next/hinic:fix a bug in set mac address
    - net-next/hinic: fix a bug in rx data flow
    - net: hinic: fix null pointer dereference on pointer hwdev
    - hinic: optmize rx refill buffer mechanism
    - net-next/hinic:add shutdown callback
    - net-next/hinic: replace disable_irq_nosync/enable_irq

  * [CONFIG] please enable highdpi font FONT_TER16x32 (LP: #1819881)
    - Fonts: New Terminus large console font
    - [Config]: enable highdpi Terminus 16x32 font support

  * [19.04 FEAT] qeth: Enhanced link speed - kernel part (LP: #1814892)
    - s390/qeth: report 25Gbit link speed

  * Avoid potential memory corruption on HiSilicon SoCs (LP: #1819546)
    - iommu/arm-smmu-v3: Avoid memory corruption from Hisilicon MSI payloads

  * CVE-2017-5715
    - x86/speculation: Apply IBPB more strictly to avoid cross-process data leak
    - x86/speculation: Propagate information about RSB filling mitigation to sysfs
    - x86/speculation: Add RETPOLINE_AMD support to the inline asm CALL_NOSPEC
      variant
    - x86/retpoline: Make CONFIG_RETPOLINE depend on compiler support
    - x86/retpoline: Remove minimal retpoline support
    - x86/speculation: Update the TIF_SSBD comment
    - x86/speculation: Clean up spectre_v2_parse_cmdline()
    - x86/speculation: Remove unnecessary ret variable in cpu_show_common()
    - x86/speculation: Move STIPB/IBPB string conditionals out of
      cpu_show_common()
    - x86/speculation: Disable STIBP when enhanced IBRS is in use
    - x86/speculation: Rename SSBD update functions
    - x86/speculation: Reorganize speculation control MSRs update
    - sched/smt: Make sched_smt_present track topology
    - x86/Kconfig: Select SCHED_SMT if SMP enabled
    - sched/smt: Expose sched_smt_present static key
    - x86/speculation: Rework SMT state change
    - x86/l1tf: Show actual SMT state
    - x86/speculation: R...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.15.0-48.51

---------------
linux (4.15.0-48.51) bionic; urgency=medium

  * linux: 4.15.0-48.51 -proposed tracker (LP: #1822820)

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts
    - [Packaging] resync retpoline extraction

  * 3b080b2564287be91605bfd1d5ee985696e61d3c in ubuntu_btrfs_kernel_fixes
    triggers system hang on i386 (LP: #1812845)
    - btrfs: raid56: properly unmap parity page in finish_parity_scrub()

  * [P9][LTCTest][Opal][FW910] cpupower monitor shows multiple stop Idle_Stats
    (LP: #1719545)
    - cpupower : Fix header name to read idle state name

  * [amdgpu] screen corruption when using touchpad (LP: #1818617)
    - drm/amdgpu/gmc: steal the appropriate amount of vram for fw hand-over (v3)
    - drm/amdgpu: Free VGA stolen memory as soon as possible.

  * [SRU][B/C/OEM]IOMMU: add kernel dma protection (LP: #1820153)
    - ACPICA: AML parser: attempt to continue loading table after error
    - ACPI / property: Allow multiple property compatible _DSD entries
    - PCI / ACPI: Identify untrusted PCI devices
    - iommu/vt-d: Force IOMMU on for platform opt in hint
    - iommu/vt-d: Do not enable ATS for untrusted devices
    - thunderbolt: Export IOMMU based DMA protection support to userspace
    - iommu/vt-d: Disable ATS support on untrusted devices

  * Add basic support to NVLink2 passthrough (LP: #1819989)
    - powerpc/powernv/npu: Do not try invalidating 32bit table when 64bit table is
      enabled
    - powerpc/powernv: call OPAL_QUIESCE before OPAL_SIGNAL_SYSTEM_RESET
    - powerpc/powernv: Export opal_check_token symbol
    - powerpc/powernv: Make possible for user to force a full ipl cec reboot
    - powerpc/powernv/idoa: Remove unnecessary pcidev from pci_dn
    - powerpc/powernv: Move npu struct from pnv_phb to pci_controller
    - powerpc/powernv/npu: Move OPAL calls away from context manipulation
    - powerpc/pseries/iommu: Use memory@ nodes in max RAM address calculation
    - powerpc/pseries/npu: Enable platform support
    - powerpc/pseries: Remove IOMMU API support for non-LPAR systems
    - powerpc/powernv/npu: Check mmio_atsd array bounds when populating
    - powerpc/powernv/npu: Fault user page into the hypervisor's pagetable

  * Huawei Hi1822 NIC has poor performance (LP: #1820187)
    - net-next: hinic: fix a problem in free_tx_poll()
    - hinic: remove ndo_poll_controller
    - net-next/hinic: add checksum offload and TSO support
    - hinic: Fix l4_type parameter in hinic_task_set_tunnel_l4
    - net-next/hinic:replace multiply and division operators
    - net-next/hinic:add rx checksum offload for HiNIC
    - net-next/hinic:fix a bug in set mac address
    - net-next/hinic: fix a bug in rx data flow
    - net: hinic: fix null pointer dereference on pointer hwdev
    - hinic: optmize rx refill buffer mechanism
    - net-next/hinic:add shutdown callback
    - net-next/hinic: replace disable_irq_nosync/enable_irq

  * [CONFIG] please enable highdpi font FONT_TER16x32 (LP: #1819881)
    - Fonts: New Terminus large console font
    - [Config]: enable highdpi Terminus 16x32 font support

  * [19.04 FEAT] qeth: Enhanced link...

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-04-29 03:46 EDT-------
IBM Bugzilla status-> closed, Fix Released for all requested distros

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for linux-azure has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.