multipath creates binding for Removable(USB) drives

[Mike Rushton]

[Impact]
Users installing with a USB key attached to the installing system, or adding a USB key on an installed system with multipath enabled will see the USB devices picked up by multipath. This is confusing because it changes the paths required to be used to access/mount/format the USB drives to paths under /dev/mapper, with mpath names.

[Test Case]
1) Attach a USB key on a system with multipath enabled; either from the installer (booting the installer with USB attached), or on an installed system.
2) See that the device is picked up by multipath-tools and shows up in the output of 'multipath -ll' and under /dev/mapper with the same name as provided by multipath -ll.

[Regression Potential]
Some USB 3.0 devices *can* support multipath; users of such custom configurations will see USB devices not being considered as multipath devices. Other devices which should be considered as multipathed may not be picked up anymore.

----

On a system with a multipath configuration, multipath bindings are being created for removable(USB) drives rendering them inaccessible as normal removable drives.

in this case, the usb drive is detected as /dev/sdi:

# lsblk|grep -A2 sdi
sdi 8:128 1 14.7G 0 disk
├─sdi1 8:129 1 14.7G 0 part
└─mpath4 (dm-9) 252:9 0 14.7G 0 mpath
  └─mpath4-part1 (dm-10) 252:10 0 14.7G 0 part

# cat /etc/multipath/bindings
# This file was created by curtin while installing the system.
mpath0 1IBM IPR-0 5EC2590000000080
# End of content generated by curtin.
# Everything below is maintained by multipath subsystem.
mpath1 1IBM IPR-0 5EC2590000000060
mpath2 1IBM IPR-0 5EC2590000000040
mpath3 1IBM IPR-0 5EC2590000000020
mpath4 1ADATA USB Flash Drive

Upon trying to format /dev/sdi1:

# mkfs.ext4 /dev/sdi1
mke2fs 1.42.9 (4-Feb-2014)
/dev/sdi1 is apparently in use by the system; will not make a filesystem here!

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: multipath-tools 0.4.9-3ubuntu7.2
ProcVersionSignature: User Name 3.16.0-41.55~14.04.1-generic 3.16.7-ckt11
Uname: Linux 3.16.0-41-generic ppc64le
ApportVersion: 2.14.1-0ubuntu3.10
Architecture: ppc64el
Date: Thu Jun 25 20:44:13 2015
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: multipath-tools
UpgradeStatus: No upgrade log present (probably fresh install)

[Mike Rushton (leftyfb)]

After following this post: https://www.redhat.com/archives/dm-devel/2008-September/msg00235.html

I have prepended the following to /lib/udev/rules.d/95-multipath.rules:

#Weed out the linear map.
KERNEL=="dm-*", PROGRAM=="/sbin/dmsetup table -j %M -m %m", RESULT=="*linear*", OPTIONS="last_rule"

# And block devices that come from usb:
ACTION=="add", KERNEL=="sd*", SUBSYSTEM=="usb", OPTIONS="last_rule"

This prevents /dev/sdi from showing up under lsblk completely and is now missing as a block device completely:

$ ls /dev/sdi*
ls: cannot access /dev/sdi*: No such file or directory

I think this is some progress, but not working as intended yet.

[Stefan Bader (smb)]

This is a problem that comes from USB and (S)ATA devices showing up as SCSI disks and the default multipath not really expecting those mixed environments. The best way to handle this is to add blacklisting in /etc/multipath.conf and the most flexible seems to key on vendor/product like this:

blacklist {
        device {
                vendor "*"
                product "*"
        }
}
blacklist_exceptions {
        device {
                vendor "IBM"
                product "*"
        }
}

Possibly with a bit more detail in the exceptions allowing only certain models or so...

[Mathieu Trudel-Lapierre (cyphermox)]

Chris, I've been working on multipath-tools a lot this week, so while I'm still touching the package, I'll take care of this one... reassigning.

Looks like there is a patch from RH which might well work to avoid this kind of situation, so I'm going to test it and upload if it fixes these issues.

[Steve Langasek (vorlon)]

I understand the fix for this bug will also need to be backported to trusty once available.

[Mathieu Trudel-Lapierre (cyphermox)]

Yes, it needs to be backported since it was meant for 0.5.0.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.5.0-7ubuntu2

---------------
multipath-tools (0.5.0-7ubuntu2) wily; urgency=medium

  * debian/patches/add_find-multipaths.patch: properly handle local non-mpath
    devices. (LP: #1463046)
  * debian/patches/ignore_devices_nodename_fail.patch: ignore devices for which
    sysfs_get_tgt_nodename fails.
  * debian/patches/skip_usb_devices.patch: skip USB devices during discovery.
    (LP: #1468897)

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 10 Jul 2015 15:53:37 -0400

[Chris J Arges (arges)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris J Arges (arges)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu12.15.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mathieu Trudel-Lapierre (cyphermox)]

verification-failed: multipath-tools still claims USB devices (as tested in cert lab). I'm doing some more backporting to try to identify the patch that will fix this correctly.

[Adam Conrad (adconrad)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mauricio Faria de Oliveira (mfo)]

Hi Mathieu,

Michael Hohnbaum mentioned this should be verified in order for the multipath-tools upload be moved from -proposed to -updates.

I'm not sure what's the process now, since the ubuntu7.4 upload actually reverts the patch for this bug.

Can you clarify what happens now? Is this verification-failed, and as the new upload reverts the patch, it's OK to go to -updates?

Thanks

[Mathieu Trudel-Lapierre (cyphermox)]

This particular bug does not need to be verified, as mentioned, this is a revert of a "fix" in proposed, not the fix itself. The other bugs attached to the multipath-tools upload for ubuntu7.4 do need to be verified however.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.4.9-3ubuntu7.4

---------------
multipath-tools (0.4.9-3ubuntu7.4) trusty; urgency=medium

  * Remove 0024-ignore-usb.patch: Ignore USB devices. Verification fails
    for this fix; it needs more work.

multipath-tools (0.4.9-3ubuntu7.3) trusty; urgency=medium

  * Added debian/patches/0015-shared-lock-for-udev.patch (LP: #1431650)
  * Support disks with non 512-byte sectors (LP: #1441930)
  * Correctly write FC timeout attributes to sysfs. (LP: #1435706)
  * Ignore USB devices. (LP: #1468897)

 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 27 Jul 2015 13:48:39 -0400

[Mathieu Trudel-Lapierre (cyphermox)]

Reopening, since this was reverted in the updates, work is still in progress.

[Mike Rushton (leftyfb)]

Updated information. It seems the issue only comes about with USB 3.0 devices. USB 2.0 devices do not get lumped in with multipath upon detection.

dmesg output from inserting a USB 3.0 device and then a USB 3.0 device. Both into USB 3.0 ports:

[ 223.050845] usb 4-4: new SuperSpeed USB device number 3 using xhci_hcd
[ 223.069996] usb 4-4: New USB device found, idVendor=125f, idProduct=312b
[ 223.070000] usb 4-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 223.070004] usb 4-4: Product: ADATA USB Flash Drive
[ 223.070006] usb 4-4: Manufacturer: ADATA
[ 223.070009] usb 4-4: SerialNumber: 0000000000000454
[ 223.070589] usb-storage 4-4:1.0: USB Mass Storage device detected
[ 223.071125] scsi4 : usb-storage 4-4:1.0
[ 224.071718] scsi 4:0:0:0: Direct-Access ADATA USB Flash Drive 1.00 PQ: 0 ANSI: 6
[ 224.073117] sd 4:0:0:0: Attached scsi generic sg32 type 0
[ 224.073458] sd 4:0:0:0: [sdi] 30869504 512-byte logical blocks: (15.8 GB/14.7 GiB)
[ 224.073729] sd 4:0:0:0: [sdi] Write Protect is off
[ 224.073737] sd 4:0:0:0: [sdi] Mode Sense: 23 00 00 00
[ 224.073992] sd 4:0:0:0: [sdi] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA
[ 224.079661] sdi: sdi1
[ 224.080909] sd 4:0:0:0: [sdi] Attached SCSI removable disk
[ 224.675188] usb 3-3: new high-speed USB device number 3 using xhci_hcd
[ 224.810797] usb 3-3: New USB device found, idVendor=13fe, idProduct=3600
[ 224.810802] usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 224.810805] usb 3-3: Product: MKNUFDPR2GB
[ 224.810807] usb 3-3: Manufacturer: MUSHKIN
[ 224.810810] usb 3-3: SerialNumber: 07C10403C8956553
[ 224.811540] usb-storage 3-3:1.0: USB Mass Storage device detected
[ 224.812218] usb-storage 3-3:1.0: Quirks match for vid 13fe pid 3600: 4000
[ 224.812419] scsi5 : usb-storage 3-3:1.0
[ 225.841184] scsi 5:0:0:0: Direct-Access MUSHKIN MKNUFDPR2GB PMAP PQ: 0 ANSI: 0 CCS
[ 225.842314] sd 5:0:0:0: Attached scsi generic sg33 type 0
[ 226.527048] sd 5:0:0:0: [sdj] 3911680 512-byte logical blocks: (2.00 GB/1.86 GiB)
[ 226.527209] sd 5:0:0:0: [sdj] Write Protect is off
[ 226.527213] sd 5:0:0:0: [sdj] Mode Sense: 23 00 00 00
[ 226.527363] sd 5:0:0:0: [sdj] No Caching mode page found
[ 226.527442] sd 5:0:0:0: [sdj] Assuming drive cache: write through
[ 226.560690] sdj: sdj1
[ 226.562590] sd 5:0:0:0: [sdj] Attached SCSI removable disk

lsblk for relevant devices:

sdi 8:128 1 14.7G 0 disk
├─sdi1 8:129 1 14.7G 0 part
└─mpath4 (dm-8) 252:8 0 14.7G 0 mpath
  └─mpath4-part1 (dm-10) 252:10 0 14.7G 0 part
sdj 8:144 1 1.9G 0 disk
└─sdj1 8:145 1 1.9G 0 part

[Steve Langasek (vorlon)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mike Rushton (leftyfb)]

Tested multipath-tools version Installed: 0.4.9-3ubuntu7.5 from the proposed repository in Trusty. The test shows the problem resolved.

[Mauricio Faria de Oliveira (mfo)]

Hi @mathieu-tl,

Marking as verification-failed since the version from -proposed hits a segfault in 'multipath -l'.
Here are some details/stack trace:

It segfaults:

 # multipath -l
 Segmentation fault

Trusty / 14.04.3:

 # lsb_release -d
 Description: Ubuntu 14.04.3 LTS

 # uname -a
 Linux plzp17 3.19.0-29-generic #31~14.04.1-Ubuntu SMP Thu Sep 10 09:45:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux

Using multipath-tools from trusty-proposed (0.4.9-3ubuntu7.5)

 # apt-cache policy multipath-tools
 multipath-tools:
   Installed: 0.4.9-3ubuntu7.5
   Candidate: 0.4.9-3ubuntu7.5
   Version table:
  *** 0.4.9-3ubuntu7.5 0
  500 http://ports.ubuntu.com/ubuntu-ports/ trusty-proposed/main ppc64el Packages
  100 /var/lib/dpkg/status
      0.4.9-3ubuntu7.4 0
  500 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main ppc64el Packages
      0.4.9-3ubuntu7 0
  500 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ppc64el Packages

 # dpkg -l | grep 'multipath\|kpartx'
 ii kpartx 0.4.9-3ubuntu7.5 ppc64el create device mappings for partitions
 ii kpartx-boot 0.4.9-3ubuntu7.5 all Provides kpartx during boot
 ii multipath-tools 0.4.9-3ubuntu7.5 ppc64el maintain multipath block device access
 ii multipath-tools-boot 0.4.9-3ubuntu7.5 all Support booting from multipath devices

Problem contained in libmultipath, not any external library/component, in the initialization phase:

 # gdb multipath
 GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
 ...
 (gdb) run -l
 Starting program: /sbin/multipath -l
 ...
 Program received signal SIGSEGV, Segmentation fault.
 0x00003fffb7e3fd70 in vector_alloc_slot () from /lib/libmultipath.so.0
 (gdb) bt
 #0 0x00003fffb7e3fd70 in vector_alloc_slot () from /lib/libmultipath.so.0
 #1 0x00003fffb7e3e200 in keyword_alloc () from /lib/libmultipath.so.0
 #2 0x00003fffb7e3e2c4 in install_keyword_root () from /lib/libmultipath.so.0
 #3 0x00003fffb7e5ac34 in init_keywords () from /lib/libmultipath.so.0
 #4 0x00003fffb7e4a54c in load_config () from /lib/libmultipath.so.0
 #5 0x0000000010002bcc in ?? ()
 #6 0x00003fffb7c94d00 in generic_start_main (main=0x10002b10, argc=<optimized out>,
     argv=0x3ffffffffa28, auxvec=0x3ffffffffb30, init=<optimized out>, rtld_fini=<optimized out>,
     stack_end=<optimized out>, fini=<optimized out>) at ../csu/libc-start.c:287
 #7 0x00003fffb7c94ef8 in __libc_start_main (argc=<optimized out>, argv=<optimized out>,
     ev=<optimized out>, auxvec=<optimized out>, rtld_fini=<optimized out>, stinfo=<optimized out>,
     stack_on_entry=<optimized out>) at ../sysdeps/unix/sysv/linux/powerpc/libc-start.c:93
 #8 0x0000000000000000 in ?? ()
 (gdb)

Downgrading to version in trusty-updates (0.4.9-3ubuntu7.4):

 # apt-get install {multipath-tools,kpartx}{,-boot}=0.4.9-3ubuntu7.4
 ...
 The following packages will be DOWNGRADED:
   kpartx kpartx-boot multipath-tools multipath-tools-boot
 ...
 Get:1 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main multipath-tools-boot all 0.4.9-3ubuntu7.4 [6,070 B]
 Get:2 http...

[Brian Murray (brian-murray)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mike Rushton (leftyfb)]

Verification failed:

ubuntu@cameron:~$ apt-cache policy multipath-tools
multipath-tools:
  Installed: 0.4.9-3ubuntu7.6
  Candidate: 0.4.9-3ubuntu7.6
  Version table:
 *** 0.4.9-3ubuntu7.6 0
        500 http://ports.ubuntu.com/ubuntu-ports/ trusty-proposed/main ppc64el Packages
        100 /var/lib/dpkg/status
     0.4.9-3ubuntu7.4 0
        500 http://ports.ubuntu.com/ubuntu-ports/ trusty-updates/main ppc64el Packages
     0.4.9-3ubuntu7 0
        500 http://ports.ubuntu.com/ubuntu-ports/ trusty/main ppc64el Packages

sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 7.5G 0 disk
├─sda1 8:1 1 7.5G 0 part
└─mpath8 (dm-10) 252:10 0 7.5G 0 mpath
  └─mpath8-part1 (dm-12) 252:12 0 7.5G 0 part

ubuntu@cameron:~$ cat /etc/multipath/bindings
# This file was created by curtin while installing the system.
mpath0 1IBM IPR-0 5EC2A60000000080
# End of content generated by curtin.
# Everything below is maintained by multipath subsystem.
mpath1 1IBM IPR-0 5EC2A600000000E0
mpath2 1IBM IPR-0 5EC2A60000000060
mpath3 1IBM IPR-0 5EC2A60000000020
mpath4 1IBM IPR-0 5EC2A60000000040
mpath5 1IBM IPR-0 5EC2A600000000C0
mpath6 1IBM IPR-0 5EC2A600000000A0
mpath7 1IBM IPR-0 5EC2A60000000100
mpath8 3200049454505080f

[Steve Langasek (vorlon)]

Mike, did you install this system using a version of the installer that includes the multipath-udeb from trusty-proposed? If not this is not a valid test of the SRU. Your comment does not establish how you installed the system, it only shows that you have a curtin-created /etc/multipath/bindings that lists the USB device

[Steve Langasek (vorlon)]

Mike, did you install this system using a version of the installer that includes the multipath-udeb from trusty-proposed? If not this is not a valid test of the SRU. Your comment does not establish how you installed the system, it only shows that you have a curtin-created /etc/multipath/bindings that lists the USB device, it does not show that this bindings file was created by the trusty-proposed version of multipath-tools. If the bindings file has been created, multipath-tools would of course still honor its contents at runtime.

[Mike Rushton (leftyfb)]

The following shows that the 0.4.9-3ubuntu7.6 version of multipath-tools does in fact ignore USB when creating the bindings. Marking as verification-done.

ubuntu@cameron:~$ cat /etc/multipath/bindings
# This file was created by curtin while installing the system.
mpath0 1IBM IPR-0 5EC2A60000000080
# End of content generated by curtin.
# Everything below is maintained by multipath subsystem.
mpath1 1IBM IPR-0 5EC2A600000000E0
mpath2 1IBM IPR-0 5EC2A60000000060
mpath3 1IBM IPR-0 5EC2A60000000020
mpath4 1IBM IPR-0 5EC2A60000000040
mpath5 1IBM IPR-0 5EC2A600000000C0
mpath6 1IBM IPR-0 5EC2A600000000A0
mpath7 1IBM IPR-0 5EC2A60000000100
mpath8 3200049454505080f
ubuntu@cameron:~$ sudo sed -i '/^mpath8/d' /etc/multipath/bindings
ubuntu@cameron:~$ cat /etc/multipath/bindings
# This file was created by curtin while installing the system.
mpath0 1IBM IPR-0 5EC2A60000000080
# End of content generated by curtin.
# Everything below is maintained by multipath subsystem.
mpath1 1IBM IPR-0 5EC2A600000000E0
mpath2 1IBM IPR-0 5EC2A60000000060
mpath3 1IBM IPR-0 5EC2A60000000020
mpath4 1IBM IPR-0 5EC2A60000000040
mpath5 1IBM IPR-0 5EC2A600000000C0
mpath6 1IBM IPR-0 5EC2A600000000A0
mpath7 1IBM IPR-0 5EC2A60000000100
ubuntu@cameron:~$ sudo /etc/init.d/multipath-tools stop ; sudo /etc/init.d/multipath-tools start
 * Stopping multipath daemon multipathd
   ...done.
 * Starting multipath daemon multipathd
   ...done.
ubuntu@cameron:~$ cat /etc/multipath/bindings
# This file was created by curtin while installing the system.
mpath0 1IBM IPR-0 5EC2A60000000080
# End of content generated by curtin.
# Everything below is maintained by multipath subsystem.
mpath1 1IBM IPR-0 5EC2A600000000E0
mpath2 1IBM IPR-0 5EC2A60000000060
mpath3 1IBM IPR-0 5EC2A60000000020
mpath4 1IBM IPR-0 5EC2A60000000040
mpath5 1IBM IPR-0 5EC2A600000000C0
mpath6 1IBM IPR-0 5EC2A600000000A0
mpath7 1IBM IPR-0 5EC2A60000000100

[Steve Langasek (vorlon)]

The verification-* tag flip-flop is due to a private conversation about concerns over the correctness of the test case. These are resolved, so marking v-done again.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.4.9-3ubuntu7.6

---------------
multipath-tools (0.4.9-3ubuntu7.6) trusty; urgency=medium

  * debian/patches/0043-alloc-keywords.patch: fix uninitialized keywords
    vector for configuration parsing for when there is no multipath.conf file
    present.

multipath-tools (0.4.9-3ubuntu7.5) trusty; urgency=medium

  * Cherry-picked patches to ignore local non-mpath devices:
    (LP: #1386637, #1468897)
    - 0001-multipath-add-checker_timeout-default-config-option.patch
    - 0002-Make-params-variable-local.patch
    - 0003-libmultipath-Fix-possible-string-overflow.patch
    - 0004-Update-hwtable-factorization.patch
    - 0005-Fixup-strip-trailing-whitespaces-for-getuid-return-v.patch
    - 0006-Remove-sysfs_attr-cache.patch
    - 0007-Move-setup_thread_attr-to-uevent.c.patch
    - 0008-Use-lists-for-uevent-processing.patch
    - 0009-Start-uevent-service-handler-from-main-thread.patch
    - 0010-libmultipath-rework-sysfs-handling.patch
    - 0011-Rework-sysfs-device-handling-in-multipathd.patch
    - 0012-Only-check-offline-status-for-SCSI-devices.patch
    - 0013-Check-for-offline-path-in-get_prio.patch
    - 0014-libmultipath-Remove-duplicate-calls-to-path_offline.patch
    - 0015-Update-dev_loss_tmo-for-no_path_retry.patch
    - 0016-Reload-map-for-device-read-only-setting-changes.patch
    - 0017-multipath-get-right-sysfs-value-for-checker_timeout.patch
    - 0018-multipath-handle-offlined-paths.patch
    - 0019-multipath-fix-scsi-timeout-code.patch
    - 0020-multipath-make-tgt_node_name-work-for-iscsi-devices.patch
    - 0021-multipath-cleanup-dev_loss_tmo-issues.patch
    - 0022-Fix-for-setting-0-to-fast_io_fail.patch
    - 0023-Fix-fast_io_fail-capping.patch
    - 0024-multipath-enable-getting-uevents-through-libudev.patch
    - 0025-Use-devpath-as-argument-for-sysfs-functions.patch
    - 0026-multipathd-remove-references-to-sysfs_device.patch
    - 0027-multipathd-use-struct-path-as-argument-for-event-pro.patch
    - 0028-Add-global-udev-reference-pointer-to-config.patch
    - 0029-Use-udev-enumeration-during-discovery.patch
    - 0030-use-struct-udev_device-during-discovery.patch
    - 0031-More-debugging-output-when-synchronizing-path-states.patch
    - 0032-Use-struct-udev_device-instead-of-sysdev.patch
    - 0033-discovery-Fixup-cciss-discovery.patch
    - 0035-Use-udev-devices-during-discovery.patch
    - 0036-Remove-all-references-to-hand-craftes-sysfs-code.patch
    - 0037-multipath-libudev-cleanup-and-bugfixes.patch
    - 0038-multipath-check-if-a-device-belongs-to-multipath.patch
    - 0039-multipath-and-wwids_file-multipath.conf-option.patch
    - 0040-multipath-Check-blacklists-as-soon-as-possible.patch
    - 0041-add-wwids-file-cleanup-options.patch
    - 0042-add-find_multipaths-option.patch
  * debian/rules: don't ship 95-multipath.rules udev rules anymore; they are
    not necessary with multipath-tools listening for udev events directly.
  * debian/multipath.udev: removed.
  * debian/multipath-tools.postinst: run multipath -W so that we get a properly
    formed wwids file on upgrade.
  * debian/initramfs/hooks: copy the wwids file in the initramfs so that it is
  ...

[Chris J Arges (arges)]

The verification of the Stable Release Update for multipath-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Brian Murray (brian-murray)]

Hello Mike, or anyone else affected,

Accepted multipath-tools into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu12.15.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Ubuntu Foundations Team Bug Bot (crichton)]

The fix for this bug has been awaiting testing feedback in the -proposed repository for vivid for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

[Mauricio Faria de Oliveira (mfo)]

The verification for vivid is no longer relevant as it's EOL by Feb, 2016. Thanks.

[Martin Pitt (pitti)]

The version of multipath-tools in the proposed pocket of Vivid that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.