Comment 0 for bug 1673350

== Comment: #0 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:01:18 ==
---Problem Description---
dm-queue-length module is not included in installer/initramfs

On Ubuntu, multipath devices using the 'queue-length' path selector
are non-functional on both the installer and initramfs environments;
because the 'dm-queue-length' kernel module is not included in them.

The multipath-modules.udeb (src:linux) does not include it in the installer,
nor multipath-tools-boot (src:multipath-tools) installs it in the initramfs.

One example is the EMC XtremIO storage, which has 'queue-length' defined as
its path selector in the default multipath configuration, at least on 16.04.

Other products may be affected if they are manually configured to use that
path selector (e.g., via /etc/multipath.conf), and the mere switch of that
might render the system _unbootable_ if booting from multipath, since the
initramfs is affected.

More recently this and another storage changed default path selectors out
of 'queue-length', however, it's virtually possible for any storage system
to be affected, with the described manual configuration change. So, this
change is also desired on for the next stable release, 17.04, and later.

Patches are provided for 16.04 and 17.04.

Error logs:
----------

This error log is seen in the installer, and multipath devices are not
created for such storage system:

...
Mar 14 06:41:54 kernel: [ 0.000000] Kernel command line: tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false --- quiet /install disk-detect/multipath/enable=true
...
Mar 14 06:47:09 main-menu[3743]: INFO: Menu item 'disk-detect' selected
...
Mar 14 06:47:14 anna-install: Installing multipath-udeb
...
Mar 14 06:47:16 kernel: [ 324.968027] device-mapper: table: 252:0: multipath: unknown path selector type
Mar 14 06:47:16 kernel: [ 324.968030] device-mapper: ioctl: error adding target to table
...
Mar 14 06:47:17 kernel: [ 326.484059] device-mapper: table: 252:0: multipath: unknown path selector type
Mar 14 06:47:17 kernel: [ 326.484062] device-mapper: ioctl: error adding target to table
Mar 14 06:47:17 disk-detect: Mar 14 06:47:16 | mpatha: ignoring map
...
Mar 14 06:47:17 disk-detect: Mar 14 06:47:17 | mpathg: ignoring map
Mar 14 06:47:18 disk-detect: No multipath devices detected

And after installing the system in an individual/single path device,
and trying to switch to multipath via multipath-tools-boot, it also
happens in the initramfs:

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Begin: Waiting for udev to settle (multipath) ... done.
[ 89.709480] device-mapper: table: 252:0: multipath: unknown path selector type
[ 89.709537] device-mapper: ioctl: error adding target to table
[ 89.756251] device-mapper: table: 252:0: multipath: unknown path selector type
[ 89.837132] device-mapper: ioctl: error adding target to table
[ 89.912346] device-mapper: table: 252:0: multipath: unknown path selector type
[ 90.001719] device-mapper: ioctl: error adding target to table
[ 90.300203] device-mapper: table: 252:0: multipath: unknown path selector type
[ 90.303521] device-mapper: ioctl: error adding target to table
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
...
Begin: Running /scripts/local-block ... done.
done.

Gave up waiting for root device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! UUID=0568fc48-2921-4ba2-b1b5-127c1d5a310e does not exist. Dropping to a shell!

BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Contact Information = <email address hidden>

---uname output---
4.4.0-31, 4.10.0-11

Machine Type = N/A

---boot type---
CDROM / ISO image

---Install repository type---
CDROM

---Point of failure---
Other failure during installation (stage 1)

== Comment: #1 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:12:30 ==
Patch for 16.04 multipath-tools

== Comment: #2 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:12:56 ==
Patch for 16.04 linux

== Comment: #3 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:13:18 ==
Patch for 17.04 multipath-tools

== Comment: #4 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:16:27 ==
Patch verification on 16.04:
---------------------------

multipath-tools source package:

# sed -n '/vendor.*XtremIO/,/selector/ p' libmultipath/hwtable.c
                .vendor = "XtremIO",
                .product = "XtremApp",
                .features = DEFAULT_FEATURES,
                .hwhandler = DEFAULT_HWHANDLER,
                .selector = "queue-length 0",

- initramfs:
  ---------

Before (module not included in initramfs):

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.5.0+git1.656f8865-5ubuntu2.4

# lsinitramfs /boot/initrd.img | grep dm-queue-length
#

After (module is included in initramfs):

# dpkg -i multipath-tools*.deb kpartx*.deb

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.5.0+git1.656f8865-5ubuntu2.4dmqueuelength1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko

- kernel udeb:
  -----------

# dpkg-deb -c multipath-modules-4.4.0-66-generic-di_4.4.0-66.87dmqueuelength1_ppc64el.udeb | grep dm-queue-length
-rw-r--r-- root/root 12174 2017-03-14 17:17 ./lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko

Patch verification on 17.04:
---------------------------

Before:

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.6.4-3ubuntu1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
#

After:

# dpkg -i multipath-tools*.deb kpartx*.deb

# dpkg -s multipath-tools-boot | grep ^Version:
Version: 0.6.4-3ubuntu1dmqueuelength1

# lsinitramfs /boot/initrd.img | grep dm-queue-length
lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko

- kernel udeb:
  -----------

# dpkg-deb -c multipath-modules-4.10.0-11-generic-di_4.10.0-11.13dmqueuelength1_ppc64el.udeb | grep dm-queue-length
-rw-r--r-- root/root 13128 2017-03-15 11:14 ./lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko

== Comment: #5 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:20:14 ==
Patch for 17.04 linux

== Comment: #9 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:24:40 ==
@taco-screen-team

Please assign this bug to @cyphermox for the multipath-tools components, at least, as a suggestion.
He's handled most of the multipath-tools related bugs/patches we've been contributing with.

Thank you.

== Comment: #10 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:40:17 ==
It's possible to reproduce this problem in a qemu-kvm guest,
with an emulated disk: force the queue-length path selector.

# name=mfo-1704
# disk=/var/lib/libvirt/images/$name.qcow2
# iso=/var/lib/libvirt/images/zesty-server-ppc64el.iso.2017-03-14

# qemu-img create -f qcow2 $disk 128g

# virt-install \
  --name $name \
  --cdrom $iso \
  --vcpus 8,sockets=1,cores=1,threads=8 \
  --memory 8192 \
  --controller type=scsi,model=virtio-scsi \
  --disk format=qcow2,path=$disk \
  --disk device=cdrom,readonly=true,path=$iso \
  --network bridge=virbr0,model=virtio

Before the disk-detection stage (e.g., set-up users and passwords),
Go Back, Execute a shell, and set up multipath.conf, and resume install:

~ # cat <<EOF >/etc/multipath.conf
defaults {
  path_selector "queue-length 0"
  user_friendly_names yes
  find_multipaths no
}
EOF

~ # exit