[18.04] multipath-tools: Backport 2 patches to Ubuntu 18.04 (NVMe disks are detected as multipath disks)

[bugproxy]

== Comment: #0 - Rodrigo R. Rosatti Galvao <email address hidden> - 2017-08-16 12:51:37 ==

NVMe disks are being detected as multipath disks, as pointed in the BZ 157373. https://bugzilla.linux.ibm.com/show_bug.cgi?id=157373

There are two upstream commits in multipath-tools that solve this issue:

4dd25783e139 Fill NVMe specific path info (https://git.opensvc.com/gitweb.cgi?p=multipath-tools/.git;a=commitdiff;h=4dd25783e13909cba0c38ed8bfedf76dc5a38c7b)

c2db19ced1ec libmultipath: fix over-long NVME WWIDs (https://git.opensvc.com/gitweb.cgi?p=multipath-tools/.git;a=commitdiff;h=c2db19ced1ec)

In order to solve this issue on Ubuntu, the 2 patches must be backported to multipath-tools package.

[David Britton (dpb)]

Hello -- Thanks for finding upstream activity that you are interested in putting into Ubuntu. This bug is a bit lacking on something that is actionable by anyone though. As a rough first pass:

1) no *detailed* description of a problem, just upstream patches that look interesting.
2) no description of environment where problem is noticed.
3) link to a downstream bug tracker (ibm.com) that is not world readable

In the end, this leads to inability to work on this or assign any priority with other bugs reported in multipath-tools

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-04 08:19 EDT-------
Hello, here's the problem description that was pointed in another bug in our downstream bug tracker (ibm.com):

---Problem Description---
NVMe disks are detected as multipath disks.
They are NVMe namespaces from a single subsystem / controller.

[anaconda root@ltciofvtr-spoon1 ~]# lsmod nvme
Usage: lsmod
[anaconda root@ltciofvtr-spoon1 ~]# lsmod | grep nvme
nvmem_core 21408 1 at24
nvme 32482 64
nvme_core 53962 33 nvme
[anaconda root@ltciofvtr-spoon1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n8 259:8 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n26 259:26 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n16 259:16 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
loop1 7:1 0 2G 1 loop
??live-base 253:1 0 2G 1 dm
??live-rw 253:0 0 2G 0 dm /
nvme0n6 259:6 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n24 259:24 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n14 259:14 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n32 259:32 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
sdb 8:16 1 931.5G 0 disk
??sdb2 8:18 1 1G 0 part
??sdb3 8:19 1 930.5G 0 part
? ??rhelaa_ltciofvtr--spoon1-root 253:6 0 50G 0 lvm
? ??rhelaa_ltciofvtr--spoon1-home 253:4 0 876.5G 0 lvm
? ??rhelaa_ltciofvtr--spoon1-swap 253:5 0 4G 0 lvm
??sdb1 8:17 1 4M 0 part
nvme0n4 259:4 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n22 259:22 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n12 259:12 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 0 28G 0 part
nvme0n30 259:30 0 28G 0 disk
??mpatha 253:2 0 28G 0 mpath
??mpatha1 253:3 ...

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-25 14:49 EDT-------
Hi, I'll try to give a better explanation about what's happening:

1. Problem: After create multiple namespaces for a NVMe device, they are being detected as mpath disks, which is not correct. The comment above showed this behavior.

2. Solution: there are 2 upstream commits that solve this issue. (As shown in the Bug Description)

The nvme devices are being blacklisted by default in multipath. You can check that by: multipathd -k'show config' | grep nvme -B3

So, in order to reproduce the issue, one must insert the following in /etc/multipath.conf:

blacklist_exceptions {
devnode "^nvme"
}

After applied those two patches, the issue was fixed:

Running 'multipath -v3 | grep uid_attribute'. (Now it uses ID_WWN instead of ID_SERIAL for NVMe devices)

Sep 25 08:23:09 | nvme0n1: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n10: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n11: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n12: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n13: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n14: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n15: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n16: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n17: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n18: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n19: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n2: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n20: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n21: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n22: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n23: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n24: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n25: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n26: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n27: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n28: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n29: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n3: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n30: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n31: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n32: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n4: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n5: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n6: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n7: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n8: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | nvme0n9: uid_attribute = ID_WWN (controller setting)
Sep 25 08:23:09 | sdb: uid_attribute = ID_SERIAL (config file default)
Sep 2...

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2017-09-25 14:51 EDT-------

[Julian Andres Klode (juliank)]

Fixed by uploading 0.7.4 in 0.7.4-2ubuntu1

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.7.4-2ubuntu1

---------------
multipath-tools (0.7.4-2ubuntu1) bionic; urgency=medium

  * Merge from Debian. (LP: #1710697, LP: #1711749) Remaining changes:
    - d/control:
      - multipath-udeb: add sg3-udeb Depends
    - d/rules: Move udev rules to priority 95, because rules that load modules
      should be >90.
    - d/multipath-tools.preinst: modprobe dm-multipath; This will make sure
      that multipathd will be able to start.
    - Split kpartx initramfs bits into kpartx-boot for dmraid (LP 941874)
      - d/initramfs/kpartx.hook
      - d/kpartx-boot.postinst
      - d/kpartx-boot.postrm
      - d/control: Add kpartx-boot package for dmraid
      - d/rules: Install kpartx initramfs hook
      - d/kpartx.install: install all arch /lib* kpartx udev rules
    - patches to multipath source
      - d/p/1000--set-umask-in-multipathd.patch: Set umask in multipathd.
      - d/p/kpartx_more_loopback_fixes.patch: fix loopback mounted
        files some more: since we stat() the loopback device node, we can't rely
        on S_ISREG() tests to handle this case, and should look at the device
        itself instead. (LP 1543430)
      - d/p/enable-find-multipaths.patch: re-enable find_multipaths
        by default -- see the removed 'add_find-multipaths.patch' (LP 1463046)
   - multipath initramfs fixes for booting from multipathed devices
      - d/initramfs/hooks: also copy wwids file on the installed system to
        ensure all paths come up on boot. (LP 1479929)
      - d/initramfs/hooks: install multipathd and required directories.
      - d/initramfs/hooks: copy multipath udev rules to initramfs
      - d/initramfs/hooks: do not copy kpartx rules to initramfs
      - d/initramfs/local-bottom: remember to stop multipathd.
      - d/initramfs/local-premount: wait for udev to settle before the call to
        resolve_device() in local_mount_root(), so the by-uuid/ symlinks have a
        chance to be updated by the multipath udev rules (LP 1503286).
      - d/initramfs/local-premount: Run multipath with -B so not to assign names
        nor change /etc/multipath/bindings during initramfs (LP 1561103)
      - d/rules: install d/initramfs/local-bottom
      - d/rules: install d/initramfs/local-premount
   - Disable -fexceptions on multipath-udeb (LP 1489379): the flag causes
     libchecktur.so to link with libgcc_s.so.1 (even with -static-libgcc),
     which is not available in the installer environment.
     - d/p/disable-fexceptions-udeb.patch: conditionally disable -fexceptions
       with CFLAGS_DISABLE_FEXCEPTIONS.
     - d/rules: set CFLAGS_DISABLE_FEXCEPTIONS to build multipath-udeb.
   - Remove d/initramfs/local-top (redundant with other initramfs scripts):
     - init-top: take over loading modules (dm-multipath and SCSI device
       handlers); move the missing dm-emc there (now scsi-dh-emc;
       see BTS 567014).
     - d/rules: do not install local-top anymore.
     - remove d/initramfs/local-top
    - Fixes typo in package description (availibility -> availability)
      (LP 1231182)
    - d/control mark transition of file /etc/init.d/multipath-tools-boot
      according to...

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-12 04:19 EDT-------
Hi,

I have verified that multipath-tools (0.7.4-2ubuntu1) has the patches we wanted to be backported.
This bug can be closed!

Thanks,
Rodrigo