dm-queue-length module is not included in installer/initramfs

[bugproxy]

[Impact]
Multipath users using EMC XtremIO storage as boot device or at install time may run into this issue. With the module unavailable the device is more often than not unavailable. Any users changing path selector to 'queue-length' with other storage devices may also be affected.

[Test case]
1) Install on multipath system using EMC XtremIO storage / OR:
 a) Start d-i install on qemu with multipath enabled
 b) exit to d-i menu
 c) modify /etc/multipath.conf to define path selector as 'queue-length' for the local qemu device.
 d) restart multipathd if necessary.
2) Try to complete the install, setting up storage as multipath and using the multipath device as boot disk.
3) Reboot to disk.

In a success case, the install should complete successfully without requiring manual configuration from the user to support the multipath storage past the normal detection of multipath and partitioning.

In a failure case, the install may not complete, or rebooting may fail or lead to a system booted on a single path of the multipath device (ie. / on /dev/sda2 rather than /dev/mpatha2).

[Regression Potential]
The inclusion of a new multipath path selector driver should not cause any regressions, but any failure to detect, configure or boot on multipath devices following this change on XtremIO hardware or otherwise would constitute a regression potentially caused by this change.

---

---Problem Description---
dm-queue-length module is not included in installer/initramfs

On Ubuntu, multipath devices using the 'queue-length' path selector
are non-functional on both the installer and initramfs environments;
because the 'dm-queue-length' kernel module is not included in them.

The multipath-modules.udeb (src:linux) does not include it in the installer,
nor multipath-tools-boot (src:multipath-tools) installs it in the initramfs.

One example is the EMC XtremIO storage, which has 'queue-length' defined as
its path selector in the default multipath configuration, at least on 16.04.

Other products may be affected if they are manually configured to use that
path selector (e.g., via /etc/multipath.conf), and the mere switch of that
might render the system _unbootable_ if booting from multipath, since the
initramfs is affected.

More recently this and another storage changed default path selectors out
of 'queue-length', however, it's virtually possible for any storage system
to be affected, with the described manual configuration change. So, this
change is also desired on for the next stable release, 17.04, and later.

Patches are provided for 16.04 and 17.04.

Error logs in LP comment #6.

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[Mauricio Faria de Oliveira (mfo)]

For easier reading, I'll break the description field apart,
to split the comments that got merged there, into actual comments at Launchpad.

[Mauricio Faria de Oliveira (mfo)]

Error logs:
----------

This error log is seen in the installer, and multipath devices are not
created for such storage system:

...
Mar 14 06:41:54 kernel: [ 0.000000] Kernel command line: tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false --- quiet /install disk-detect/multipath/enable=true
...
Mar 14 06:47:09 main-menu[3743]: INFO: Menu item 'disk-detect' selected
...
Mar 14 06:47:14 anna-install: Installing multipath-udeb
...
Mar 14 06:47:16 kernel: [ 324.968027] device-mapper: table: 252:0: multipath: unknown path selector type
Mar 14 06:47:16 kernel: [ 324.968030] device-mapper: ioctl: error adding target to table
...
Mar 14 06:47:17 kernel: [ 326.484059] device-mapper: table: 252:0: multipath: unknown path selector type
Mar 14 06:47:17 kernel: [ 326.484062] device-mapper: ioctl: error adding target to table
Mar 14 06:47:17 disk-detect: Mar 14 06:47:16 | mpatha: ignoring map
...
Mar 14 06:47:17 disk-detect: Mar 14 06:47:17 | mpathg: ignoring map
Mar 14 06:47:18 disk-detect: No multipath devices detected

And after installing the system in an individual/single path device,
and trying to switch to multipath via multipath-tools-boot, it also
happens in the initramfs:

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Begin: Waiting for udev to settle (multipath) ... done.
[ 89.709480] device-mapper: table: 252:0: multipath: unknown path selector type
[ 89.709537] device-mapper: ioctl: error adding target to table
[ 89.756251] device-mapper: table: 252:0: multipath: unknown path selector type
[ 89.837132] device-mapper: ioctl: error adding target to table
[ 89.912346] device-mapper: table: 252:0: multipath: unknown path selector type
[ 90.001719] device-mapper: ioctl: error adding target to table
[ 90.300203] device-mapper: table: 252:0: multipath: unknown path selector type
[ 90.303521] device-mapper: ioctl: error adding target to table
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...
Begin: Running /scripts/local-block ... done.
done.

Gave up waiting for root device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! UUID=0568fc48-2921-4ba2-b1b5-127c1d5a310e does not exist. Dropping to a shell!

BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

[Mauricio Faria de Oliveira (mfo)]

Patch verification on 16.04:
---------------------------

multipath-tools source package:

# sed -n '/vendor.*XtremIO/,/selector/ p' libmultipath/hwtable.c
                .vendor = "XtremIO",
                .product = "XtremApp",
                .features = DEFAULT_FEATURES,
                .hwhandler = DEFAULT_HWHANDLER,
                .selector = "queue-length 0",

- initramfs:
  ---------

Before (module not included in initramfs):

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.5.0+git1.656f8865-5ubuntu2.4

# lsinitramfs /boot/initrd.img | grep dm-queue-length
#

After (module is included in initramfs):

# dpkg -i multipath-tools*.deb kpartx*.deb

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.5.0+git1.656f8865-5ubuntu2.4dmqueuelength1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko

- kernel udeb:
  -----------

# dpkg-deb -c multipath-modules-4.4.0-66-generic-di_4.4.0-66.87dmqueuelength1_ppc64el.udeb | grep dm-queue-length
-rw-r--r-- root/root 12174 2017-03-14 17:17 ./lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko

[Mauricio Faria de Oliveira (mfo)]

Patch verification on 17.04:
---------------------------

Before:

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.6.4-3ubuntu1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
#

After:

# dpkg -i multipath-tools*.deb kpartx*.deb

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.6.4-3ubuntu1dmqueuelength1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko

- kernel udeb:
  -----------

# dpkg-deb -c multipath-modules-4.10.0-11-generic-di_4.10.0-11.13dmqueuelength1_ppc64el.udeb | grep dm-queue-length
-rw-r--r-- root/root 13128 2017-03-15 11:14 ./lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko

[Mauricio Faria de Oliveira (mfo)]

It's possible to reproduce this problem in a qemu-kvm guest,
with an emulated disk: force the queue-length path selector.

# name=mfo-1704
# disk=/var/lib/libvirt/images/$name.qcow2
# iso=/var/lib/libvirt/images/zesty-server-ppc64el.iso.2017-03-14

# qemu-img create -f qcow2 $disk 128g

# virt-install \
  --name $name \
  --cdrom $iso \
  --vcpus 8,sockets=1,cores=1,threads=8 \
  --memory 8192 \
  --controller type=scsi,model=virtio-scsi \
  --disk format=qcow2,path=$disk \
  --disk device=cdrom,readonly=true,path=$iso \
  --network bridge=virbr0,model=virtio

Before the disk-detection stage (e.g., set-up users and passwords),
Go Back, Execute a shell, and set up multipath.conf, and resume install:

~ # cat <<EOF >/etc/multipath.conf
defaults {
  path_selector "queue-length 0"
  user_friendly_names yes
  find_multipaths no
}
EOF

~ # exit

[Mauricio Faria de Oliveira (mfo)]

@taco-screen-team

Please assign this bug to @cyphermox for the multipath-tools components, at least, as a suggestion.
He's handled most of the multipath-tools related bugs/patches we've been contributing with.

Thank you.

UPDATE: this bug also relates to the component 'linux' (for multipath-modules udeb) in addition to 'multipath-tools'

[Michael Hohnbaum (hohnbaum)]

Steve,

Can you please assign to Foundations (request from IBM is cyphermox).

Thanks.

                         Michael

On 03/16/2017 12:09 AM, Launchpad Bug Tracker wrote:
> bugproxy (bugproxy) has assigned this bug to you for Ubuntu:
>
> == Comment: #0 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:01:18 ==
> ---Problem Description---
> dm-queue-length module is not included in installer/initramfs
>
> On Ubuntu, multipath devices using the 'queue-length' path selector
> are non-functional on both the installer and initramfs environments;
> because the 'dm-queue-length' kernel module is not included in them.
>
> The multipath-modules.udeb (src:linux) does not include it in the installer,
> nor multipath-tools-boot (src:multipath-tools) installs it in the initramfs.
>
> One example is the EMC XtremIO storage, which has 'queue-length' defined as
> its path selector in the default multipath configuration, at least on 16.04.
>
> Other products may be affected if they are manually configured to use that
> path selector (e.g., via /etc/multipath.conf), and the mere switch of that
> might render the system _unbootable_ if booting from multipath, since the
> initramfs is affected.
>
> More recently this and another storage changed default path selectors out
> of 'queue-length', however, it's virtually possible for any storage system
> to be affected, with the described manual configuration change. So, this
> change is also desired on for the next stable release, 17.04, and later.
>
> Patches are provided for 16.04 and 17.04.
>
>
> Error logs:
> ----------
>
> This error log is seen in the installer, and multipath devices are not
> created for such storage system:
>
> ...
> Mar 14 06:41:54 kernel: [ 0.000000] Kernel command line: tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false --- quiet /install disk-detect/multipath/enable=true
> ...
> Mar 14 06:47:09 main-menu[3743]: INFO: Menu item 'disk-detect' selected
> ...
> Mar 14 06:47:14 anna-install: Installing multipath-udeb
> ...
> Mar 14 06:47:16 kernel: [ 324.968027] device-mapper: table: 252:0: multipath: unknown path selector type
> Mar 14 06:47:16 kernel: [ 324.968030] device-mapper: ioctl: error adding target to table
> ...
> Mar 14 06:47:17 kernel: [ 326.484059] device-mapper: table: 252:0: multipath: unknown path selector type
> Mar 14 06:47:17 kernel: [ 326.484062] device-mapper: ioctl: error adding target to table
> Mar 14 06:47:17 disk-detect: Mar 14 06:47:16 | mpatha: ignoring map
> ...
> Mar 14 06:47:17 disk-detect: Mar 14 06:47:17 | mpathg: ignoring map
> Mar 14 06:47:18 disk-detect: No multipath devices detected
>
>
> And after installing the system in an individual/single path device,
> and trying to switch to multipath via multipath-tools-boot, it also
> happens in the initramfs:
>
> Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
> Begin: Running /scripts/local-premount ... Begin: Waiting for udev to settle (multipath) ... done.
> [ 89.709480] device-mapper: table: 252:0: multipath: unknown path selector type
> [ 89.709537] device-mapper: ioctl: error adding target to table
> [ 89....

[Mathieu Trudel-Lapierre (cyphermox)]

This goes without saying, thanks for filing this bug report.

[Mathieu Trudel-Lapierre (cyphermox)]

Triaged and assigned to me for the multipath-tools and d-i work. I've notified the kernel team on IRC as well since I'll need to have multipath-modules-udeb containing dm-queue-length before I can apply the changes there. Then we can SRU the whole thing as well.

[Stefan Bader (smb)]

Prepared a patch and sent to the kernel-team mailing list.

[Mauricio Faria de Oliveira (mfo)]

Thanks, @cyphermox and @smb.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.6.4-3ubuntu2

---------------
multipath-tools (0.6.4-3ubuntu2) zesty; urgency=medium

  * debian/initramfs/hooks:
    - Remove old dm-emc module; it hasn't been in kernel since 2.6.27.
    - Add dm-queue-length: users may want to change from the default selector
      and should be able to do so. (LP: #1673350)
  * debian/patches/path_selector.patch, debian/patches/series: remove
    path_selector.patch to get back to the "right" default selector of
    service-time, which has been included in the kernel module packages for
    a while now and has better throughput than round-robin. It's also the
    default upstream. (LP: #1634161)

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 17 Mar 2017 09:37:36 -0400

[Mathieu Trudel-Lapierre (cyphermox)]

AFAICT there wouldn't really be any change required to initramfs-tools -- everything is contained in multipath-tools-boot.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-03-21 11:39 EDT-------
> AFAICT there wouldn't really be any change required to initramfs-tools --
> everything is contained in multipath-tools-boot.

Right. That seems to have been set for screening/triaging purposes during bug mirroring / initially reporting the bug on LP, but this is definitely only multipath-tools and linux (plus some d-i updates, I guess, for the version number of the linux kernel).

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.10.0-14.16

---------------
linux (4.10.0-14.16) zesty; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1673805

  * msleep() bug causes Nuvoton I2C TPM device driver delays (LP: #1667567)
    - tpm: msleep() delays - replace with usleep_range() in i2c nuvoton driver
    - SAUCE: tpm: add sleep only for retry in i2c_nuvoton_write_status()

  * C++ demangling support missing from perf (LP: #1396654)
    - [Config] added binutils-dev to Build-deps

  * dm-queue-length module is not included in installer/initramfs (LP: #1673350)
    - [Config] d-i: Also add dm-queue-length to multipath modules

  * move aufs.ko from -extra to linux-image package (LP: #1673498)
    - [config] aufs.ko moved to linux-image package

  * Using an NVMe drive causes huge power drain (LP: #1664602)
    - nvme: Add a quirk mechanism that uses identify_ctrl
    - nvme: Enable autonomous power state transitions

  * Broadcom bluetooth modules sometimes fail to initialize (LP: #1483101)
    - Bluetooth: btbcm: Add a delay for module reset

  * Need support of Broadcom bluetooth device [413c:8143] (LP: #1166113)
    - Bluetooth: btusb: Add support for 413c:8143

  * Zesty update to v4.10.3 stable release (LP: #1673118)
    - serial: 8250_pci: Add MKS Tenta SCOM-0800 and SCOM-0801 cards
    - KVM: s390: Disable dirty log retrieval for UCONTROL guests
    - KVM: VMX: use correct vmcs_read/write for guest segment selector/base
    - Bluetooth: Add another AR3012 04ca:3018 device
    - phy: qcom-ufs: Don't kfree devres resource
    - phy: qcom-ufs: Fix misplaced jump label
    - s390/qdio: clear DSCI prior to scanning multiple input queues
    - s390/dcssblk: fix device size calculation in dcssblk_direct_access()
    - s390/kdump: Use "LINUX" ELF note name instead of "CORE"
    - s390/chsc: Add exception handler for CHSC instruction
    - s390: TASK_SIZE for kernel threads
    - s390/topology: correct allocation of topology information
    - s390: make setup_randomness work
    - s390: use correct input data address for setup_randomness
    - net: mvpp2: fix DMA address calculation in mvpp2_txq_inc_put()
    - cxl: Prevent read/write to AFU config space while AFU not configured
    - cxl: fix nested locking hang during EEH hotplug
    - brcmfmac: fix incorrect event channel deduction
    - mnt: Tuck mounts under others instead of creating shadow/side mounts.
    - IB/ipoib: Fix deadlock between rmmod and set_mode
    - IB/IPoIB: Add destination address when re-queue packet
    - IB/mlx5: Fix out-of-bound access
    - IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS
    - IB/srp: Avoid that duplicate responses trigger a kernel bug
    - IB/srp: Fix race conditions related to task management
    - Btrfs: fix data loss after truncate when using the no-holes feature
    - orangefs: Use RCU for destroy_inode
    - memory/atmel-ebi: Fix ns <-> cycles conversions
    - tracing: Fix return value check in trace_benchmark_reg()
    - ktest: Fix child exit code processing
    - ceph: remove req from unsafe list when unregistering it
    - target: Fix NULL dereference during LUN lookup + active I/O shutdown
    - drivers/pci/hotplug: Han...

[Mathieu Trudel-Lapierre (cyphermox)]

The hw-detect change is uploaded, so once it's in the archive, installs on hardware requiring dm-queue-length should work without issues.

We're still blocked on having the kernel include dm-queue-length and dm-service-time for the installer for the hw-detect side of this SRU to yakkety and xenial.

As for the multipath-tools part of the SRUs, I'll upload them in the morning.

[Launchpad Janitor (janitor)]

This bug was fixed in the package hw-detect - 1.117ubuntu4

---------------
hw-detect (1.117ubuntu4) zesty; urgency=medium

  * disk-detect.sh: unconditionally modprobe dm-service-time and
    dm-queue-length multipath path selector modules alongside dm-round-robin.
    (LP: #1673350)

 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 27 Mar 2017 21:23:41 -0400

[bugproxy (bugproxy)]

Default Comment by Bridge

[Brian Murray (brian-murray)]

Hello bugproxy, or anyone else affected,

Accepted multipath-tools into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.5.0+git1.656f8865-5ubuntu7.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello bugproxy, or anyone else affected,

Accepted multipath-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.5.0+git1.656f8865-5ubuntu2.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Kleber Sacilotto de Souza (kleber-souza)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Kleber Sacilotto de Souza (kleber-souza)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-yakkety' to 'verification-done-yakkety'. If the problem still exists, change the tag 'verification-needed-yakkety' to 'verification-failed-yakkety'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-75.96

---------------
linux (4.4.0-75.96) xenial; urgency=low

  * linux: 4.4.0-75.96 -proposed tracker (LP: #1684441)

  * [Hyper-V] hv: util: move waiting for release to hv_utils_transport itself
    (LP: #1682561)
    - Drivers: hv: util: move waiting for release to hv_utils_transport itself

linux (4.4.0-74.95) xenial; urgency=low

  * linux: 4.4.0-74.95 -proposed tracker (LP: #1682041)

  * [Hyper-V] hv: vmbus: Raise retry/wait limits in vmbus_post_msg()
    (LP: #1681893)
    - Drivers: hv: vmbus: Raise retry/wait limits in vmbus_post_msg()

linux (4.4.0-73.94) xenial; urgency=low

  * linux: 4.4.0-73.94 -proposed tracker (LP: #1680416)

  * CVE-2017-6353
    - sctp: deny peeloff operation on asocs with threads sleeping on it

  * vfat: missing iso8859-1 charset (LP: #1677230)
    - [Config] NLS_ISO8859_1=y

  * Regression: KVM modules should be on main kernel package (LP: #1678099)
    - [Config] powerpc: Add kvm-hv and kvm-pr to the generic inclusion list

  * linux-lts-xenial 4.4.0-63.84~14.04.2 ADT test failure with linux-lts-xenial
    4.4.0-63.84~14.04.2 (LP: #1664912)
    - SAUCE: apparmor: fix link auditing failure due to, uninitialized var

  * regession tests failing after stackprofile test is run (LP: #1661030)
    - SAUCE: fix regression with domain change in complain mode

  * Permission denied and inconsistent behavior in complain mode with 'ip netns
    list' command (LP: #1648903)
    - SAUCE: fix regression with domain change in complain mode

  * unexpected errno=13 and disconnected path when trying to open /proc/1/ns/mnt
    from a unshared mount namespace (LP: #1656121)
    - SAUCE: apparmor: null profiles should inherit parent control flags

  * apparmor refcount leak of profile namespace when removing profiles
    (LP: #1660849)
    - SAUCE: apparmor: fix ns ref count link when removing profiles from policy

  * tor in lxd: apparmor="DENIED" operation="change_onexec"
    namespace="root//CONTAINERNAME_<var-lib-lxd>" profile="unconfined"
    name="system_tor" (LP: #1648143)
    - SAUCE: apparmor: Fix no_new_privs blocking change_onexec when using stacked
      namespaces

  * apparmor oops in bind_mnt when dev_path lookup fails (LP: #1660840)
    - SAUCE: apparmor: fix oops in bind_mnt when dev_path lookup fails

  * apparmor auditing denied access of special apparmor .null fi\ le
    (LP: #1660836)
    - SAUCE: apparmor: Don't audit denied access of special apparmor .null file

  * apparmor label leak when new label is unused (LP: #1660834)
    - SAUCE: apparmor: fix label leak when new label is unused

  * apparmor reference count bug in label_merge_insert() (LP: #1660833)
    - SAUCE: apparmor: fix reference count bug in label_merge_insert()

  * apparmor's raw_data file in securityfs is sometimes truncated (LP: #1638996)
    - SAUCE: apparmor: fix replacement race in reading rawdata

  * unix domain socket cross permission check failing with nested namespaces
    (LP: #1660832)
    - SAUCE: apparmor: fix cross ns perm of unix domain sockets

  * Xenial update to v4.4.59 stable release (LP: #1678960)
    - xfrm: policy: init locks early
    - virtio_balloon: init ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.8.0-49.52

---------------
linux (4.8.0-49.52) yakkety; urgency=low

  * linux: 4.8.0-49.52 -proposed tracker (LP: #1684427)

  * [Hyper-V] hv: util: move waiting for release to hv_utils_transport itself
    (LP: #1682561)
    - Drivers: hv: util: move waiting for release to hv_utils_transport itself

linux (4.8.0-48.51) yakkety; urgency=low

  * linux: 4.8.0-48.51 -proposed tracker (LP: #1682034)

  * [Hyper-V] hv: vmbus: Raise retry/wait limits in vmbus_post_msg()
    (LP: #1681893)
    - Drivers: hv: vmbus: Raise retry/wait limits in vmbus_post_msg()

linux (4.8.0-47.50) yakkety; urgency=low

  * linux: 4.8.0-47.50 -proposed tracker (LP: #1679678)

  * CVE-2017-6353
    - sctp: deny peeloff operation on asocs with threads sleeping on it

  * CVE-2017-5986
    - sctp: avoid BUG_ON on sctp_wait_for_sndbuf

  * vfat: missing iso8859-1 charset (LP: #1677230)
    - [Config] NLS_ISO8859_1=y

  * [Hyper-V] pci-hyperv: Use device serial number as PCI domain (LP: #1667527)
    - net/mlx4_core: Use cq quota in SRIOV when creating completion EQs

  * Regression: KVM modules should be on main kernel package (LP: #1678099)
    - [Config] powerpc: Add kvm-hv and kvm-pr to the generic inclusion list

  * linux-lts-xenial 4.4.0-63.84~14.04.2 ADT test failure with linux-lts-xenial
    4.4.0-63.84~14.04.2 (LP: #1664912)
    - SAUCE: apparmor: fix link auditing failure due to, uninitialized var

  * regession tests failing after stackprofile test is run (LP: #1661030)
    - SAUCE: fix regression with domain change in complain mode

  * Permission denied and inconsistent behavior in complain mode with 'ip netns
    list' command (LP: #1648903)
    - SAUCE: fix regression with domain change in complain mode

  * unexpected errno=13 and disconnected path when trying to open /proc/1/ns/mnt
    from a unshared mount namespace (LP: #1656121)
    - SAUCE: apparmor: null profiles should inherit parent control flags

  * apparmor refcount leak of profile namespace when removing profiles
    (LP: #1660849)
    - SAUCE: apparmor: fix ns ref count link when removing profiles from policy

  * tor in lxd: apparmor="DENIED" operation="change_onexec"
    namespace="root//CONTAINERNAME_<var-lib-lxd>" profile="unconfined"
    name="system_tor" (LP: #1648143)
    - SAUCE: apparmor: Fix no_new_privs blocking change_onexec when using stacked
      namespaces

  * apparmor oops in bind_mnt when dev_path lookup fails (LP: #1660840)
    - SAUCE: apparmor: fix oops in bind_mnt when dev_path lookup fails

  * apparmor auditing denied access of special apparmor .null fi\ le
    (LP: #1660836)
    - SAUCE: apparmor: Don't audit denied access of special apparmor .null file

  * apparmor label leak when new label is unused (LP: #1660834)
    - SAUCE: apparmor: fix label leak when new label is unused

  * apparmor reference count bug in label_merge_insert() (LP: #1660833)
    - SAUCE: apparmor: fix reference count bug in label_merge_insert()

  * apparmor's raw_data file in securityfs is sometimes truncated (LP: #1638996)
    - SAUCE: apparmor: fix replacement race in reading rawdata

  * unix domain socket cross permission check failing with n...

[Mathieu Trudel-Lapierre (cyphermox)]

Verification done for yakkety's multipath-tools SRU:

multipath-tools 0.5.0+git1.656f8865-5ubuntu7.3

I verified that multipath-tools correctly ships its new hook to include dm-service-time and dm-queue-length, and the system with that update boots successfully.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.5.0+git1.656f8865-5ubuntu7.3

---------------
multipath-tools (0.5.0+git1.656f8865-5ubuntu7.3) yakkety; urgency=medium

  * debian/initramfs/hooks: drop dm-emc (it hasn't existed since 2.6.27) and
    add dm-queue-length: it's required on some hardware, such as XtremIO, where
    it is the default path selector. (LP: #1673350)

 -- Mathieu Trudel-Lapierre <email address hidden> Tue, 28 Mar 2017 09:58:21 -0400

[Łukasz Zemczak (sil2100)]

As part of a recent change in the Stable Release Update verification policy we would like to inform that for a bug to be considered verified for a given release a verification-done-$RELEASE tag needs to be added to the bug where $RELEASE is the name of the series the package that was tested (e.g. verification-done-xenial). Please note that the global 'verification-done' tag can no longer be used for this purpose.

Thank you!

[Łukasz Zemczak (sil2100)]

Hello bugproxy, or anyone else affected,

Accepted hw-detect into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/hw-detect/1.117ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello bugproxy, or anyone else affected,

Accepted hw-detect into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/hw-detect/1.117ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-yakkety to verification-done-yakkety. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-yakkety. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-07-11 16:24 EDT-------
Verified on 16.04 with the daily ISO (2017/07/11).

Just noticed a glitch with grub-installer and multipath,
but that's not related to this bug. I'll report a new one.

1) The dm-queue-length module is present in the installer udebs:

~ # uname -r
4.4.0-83-generic

~ # find /lib/modules/ -name dm-queue-length.ko
/lib/modules/4.4.0-83-generic/kernel/drivers/md/dm-queue-length.ko

2) And it can be used by multipath in the installer:

~ # cat <<EOF >/etc/multipath.conf
defaults {
path_selector "queue-length 0"
user_friendly_names yes
find_multipaths no
}
EOF

~ # multipath
...

~ # multipath -l
mpatha (0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0) dm-0 QEMU,QEMU HARDDISK
size=128G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=0 status=active
`- 0:0:0:0 sda 8:0 active undef running

3) The hw-detect/disk-detect is loading the modules too.

~ # grep dm- /bin/disk-detect
<...>
log-output -t disk-detect modprobe -v -b dm-round-robin || true
log-output -t disk-detect modprobe -v -b dm-service-time || true
log-output -t disk-detect modprobe -v -b dm-queue-length || true

4) Versions used by the installer:

~ # grep multipath /var/log/syslog
<...>
Jul 11 18:52:49 anna[8788]: DEBUG: retrieving multipath-modules-4.4.0-83-generic-di 4.4.0-83.106
Jul 11 18:52:49 anna[8788]: DEBUG: retrieving multipath-udeb 0.5.0+git1.656f8865-5ubuntu2.5
<...>

~ # grep hw-detect /var/log/syslog
<...>
Jul 11 18:52:00 anna[4126]: DEBUG: retrieving hw-detect 1.117ubuntu2.2
<...>

5) initramfs is installed with the dm-queue-length module:

~ # grep dm-queue-length /target/usr/share/initramfs-tools/hooks/multipath
for x in dm-multipath dm-service-time dm-round-robin dm-queue-length; do

(initramfs) find /lib/modules -name dm-queue-length.ko
/lib/modules/4.4.0-85-generic/kernel/drivers/md/dm-queue-length.ko

6) multipath root uses the dm-queue-length module:

(initramfs) multipath -l
mpatha (0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0) dm-0 QEMU,QEMU HARDDISK
size=128G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=0 status=active
`- 0:0:0:0 sda 8:0 active undef running

ubuntu@ubuntu:~$ mount | grep ' / '
/dev/mapper/mpatha-part2 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)

ubuntu@ubuntu:~$ sudo multipath -l mpatha
mpatha (0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0) dm-0 QEMU,QEMU HARDDISK
size=128G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=0 status=active
`- 0:0:0:0 sda 8:0 active undef running

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.5.0+git1.656f8865-5ubuntu2.5

---------------
multipath-tools (0.5.0+git1.656f8865-5ubuntu2.5) xenial; urgency=medium

  * debian/initramfs/hooks: drop dm-emc (it hasn't existed since 2.6.27) and
    add dm-queue-length: it's required on some hardware, such as XtremIO, where
    it is the default path selector. (LP: #1673350)

 -- Mathieu Trudel-Lapierre <email address hidden> Tue, 28 Mar 2017 09:58:21 -0400

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for multipath-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package hw-detect - 1.117ubuntu2.2

---------------
hw-detect (1.117ubuntu2.2) xenial; urgency=medium

  * disk-detect.sh: unconditionally modprobe dm-service-time and
    dm-queue-length multipath path selector modules alongside dm-round-robin.
    (LP: #1673350)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 14 Jun 2017 16:15:14 -0400