| 2010-12-08 01:25:34 |
Ron |
bug |
|
|
added bug |
| 2010-12-08 21:30:18 |
Serge Hallyn |
multipath-tools (Ubuntu): status |
New |
Confirmed |
|
| 2010-12-08 21:30:31 |
Serge Hallyn |
multipath-tools (Ubuntu): importance |
Undecided |
High |
|
| 2010-12-08 21:38:34 |
Serge Hallyn |
multipath-tools (Ubuntu): assignee |
|
Serge Hallyn (serge-hallyn) |
|
| 2010-12-08 21:42:49 |
Serge Hallyn |
multipath-tools (Ubuntu): status |
Confirmed |
Fix Committed |
|
| 2010-12-15 19:58:05 |
Joseph Salisbury |
bug |
|
|
added subscriber Joseph Salisbury |
| 2011-01-03 18:37:01 |
Boris Devouge |
bug |
|
|
added subscriber Boris Devouge |
| 2011-01-04 21:16:47 |
Serge Hallyn |
multipath-tools (Ubuntu): status |
Fix Committed |
In Progress |
|
| 2011-01-07 19:38:49 |
Serge Hallyn |
multipath-tools (Ubuntu): status |
In Progress |
Fix Released |
|
| 2011-01-10 18:11:25 |
Serge Hallyn |
description |
Binary package hint: multipath-tools-boot
After installing "multipath-tools-boot", initramfs fails to mount /root by uuid with error: "failed: Device or resource busy". And boot process falls into (initramfs) prompt.
**Boot/Screen Output:**
Begin: Running /scripts/local-premount ... done.
mount: mounting /dev/disk/by-uuid/082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 on /root
failed: Device or resource busy
Begin: Running /scripts/local-bottom ... done.
done.
<....> other failure messages mounting (dev, sys, proc, etc.) <....>
(initramfs)
**Further Debugging & Info:**
Procedures & commands, run shortly after (initramfs) prompt appears.
(initramfs) ls -l /dev/disk/by-uuid/
lrwxrwxrwx 1 10 082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 -> ../../dm1
[ This appears correct!?! ]
(initramfs) mount /dev/disk/by-uuid/082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 /root
[ SUCCESS ! ]
**WORKAROUND:**
I have created an initramfs script to slow-down the root mounting attempt:
File: /usr/share/initramfs-tools/scripts/local-top/waitfor-multipath
Content:
#!/bin/sh
VERBOSITY=0
PREREQ="multipath"
prereqs() { echo "$PREREQ"; }
case $1 in
prereqs)
prereqs
exit 0
;;
esac
echo "Waiting 5 seconds for multipaths & udev?? to settle down..."
sleep 5
exit 0
<----END---->
My guess for a general/global bug-fix is multipath-tools-boot script to include a slight delay/hold-down timer; to allow outside device-mapping procedures to complete. ..or a confirmation that dm devices have finished mapping.
**Observations:**
It appears multipath-tools-boot initramfs script executes successfully VERY quickly, and initramfs scripts continue with attempting to mount the root file system. Although, dm-# /dev/disk/by-uuid mappings are not instantly ready. This is why manually typing the same mount command at prompt is immediately successful.
_To duplicate error in a non-boot scenario:_
This exact same behavior can be observed & duplicated on running servers; when mounting a multipath (SAN) volume. As example; mounting multipath-SAN targets will create: /dev/sda, /dev/sdb, /dev/sdc, etc.... And multipath-tools+devmapper will then create the shared/linked /dev/dm-# mappings. You are then only supposed to use the /dev/dm-# virtual devices to access multipathed SAN devices (multipaths.)
.... If you then attempt to mount a direct SAN volume /dev/sd?# an error is raised: "mount: /dev/sd?# already mounted or /<mount_point> busy"
**Software & Versions:**
OS: Ubuntu 10.10 (Maverick - server)
Related Packages & versions:
multipath-tools 0.4.8-14ubuntu4
multipath-tools-boot 0.4.8-14ubuntu4
udev 162-2.1
initramfs-tools 0.98.1ubuntu6
initramfs-tools-bin 0.98.1ubuntu6
open-iscsi 2.0.871-0ubuntu5
open-iscsi-utils 2.0.871-0ubuntu5
libdevmapper1.02.1 2:1.02.39-1ubuntu6
**Hardware Details:**
Servers affected: HP Proliant DL385 (G5, G5p, G6) (and others I'm sure.)
Storage/SAN Type: iSCSI
Storage/SAN Hardware: HP StorageWorks MSA2312i (and others I'm sure.)
This problem affects servers with iSCSI-boot NICs, AND servers which have a USB-Key with /boot partition.
Please let me know if more information is necessary! |
SRU Justification:
1. Impact: servers with multipath-tools-boot installed cannot mount root on boot
2. How bug addressed: The initramfs local-top script for multipath-tools waits for udev to handle all device renames resulting from multipath starting up before continuing on to mountroot.
3. Patch: is attached as a debdiff
4. To reproduce: install a server with multipath storage controller, install multipath-tools-boot, and reboot.
5. Regression potential: the fix is localized to an initramfs script shipped with multipath-tools-boot, so if there are regressions they should only affect multipath users, who cannot boot without some fix anyway.
Binary package hint: multipath-tools-boot
After installing "multipath-tools-boot", initramfs fails to mount /root by uuid with error: "failed: Device or resource busy". And boot process falls into (initramfs) prompt.
**Boot/Screen Output:**
Begin: Running /scripts/local-premount ... done.
mount: mounting /dev/disk/by-uuid/082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 on /root
failed: Device or resource busy
Begin: Running /scripts/local-bottom ... done.
done.
<....> other failure messages mounting (dev, sys, proc, etc.) <....>
(initramfs)
**Further Debugging & Info:**
Procedures & commands, run shortly after (initramfs) prompt appears.
(initramfs) ls -l /dev/disk/by-uuid/
lrwxrwxrwx 1 10 082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 -> ../../dm1
[ This appears correct!?! ]
(initramfs) mount /dev/disk/by-uuid/082f71b9-47dc-4d4b-bdfa-d5ca61f1e045 /root
[ SUCCESS ! ]
**WORKAROUND:**
I have created an initramfs script to slow-down the root mounting attempt:
File: /usr/share/initramfs-tools/scripts/local-top/waitfor-multipath
Content:
#!/bin/sh
VERBOSITY=0
PREREQ="multipath"
prereqs() { echo "$PREREQ"; }
case $1 in
prereqs)
prereqs
exit 0
;;
esac
echo "Waiting 5 seconds for multipaths & udev?? to settle down..."
sleep 5
exit 0
<----END---->
My guess for a general/global bug-fix is multipath-tools-boot script to include a slight delay/hold-down timer; to allow outside device-mapping procedures to complete. ..or a confirmation that dm devices have finished mapping.
**Observations:**
It appears multipath-tools-boot initramfs script executes successfully VERY quickly, and initramfs scripts continue with attempting to mount the root file system. Although, dm-# /dev/disk/by-uuid mappings are not instantly ready. This is why manually typing the same mount command at prompt is immediately successful.
_To duplicate error in a non-boot scenario:_
This exact same behavior can be observed & duplicated on running servers; when mounting a multipath (SAN) volume. As example; mounting multipath-SAN targets will create: /dev/sda, /dev/sdb, /dev/sdc, etc.... And multipath-tools+devmapper will then create the shared/linked /dev/dm-# mappings. You are then only supposed to use the /dev/dm-# virtual devices to access multipathed SAN devices (multipaths.)
.... If you then attempt to mount a direct SAN volume /dev/sd?# an error is raised: "mount: /dev/sd?# already mounted or /<mount_point> busy"
**Software & Versions:**
OS: Ubuntu 10.10 (Maverick - server)
Related Packages & versions:
multipath-tools 0.4.8-14ubuntu4
multipath-tools-boot 0.4.8-14ubuntu4
udev 162-2.1
initramfs-tools 0.98.1ubuntu6
initramfs-tools-bin 0.98.1ubuntu6
open-iscsi 2.0.871-0ubuntu5
open-iscsi-utils 2.0.871-0ubuntu5
libdevmapper1.02.1 2:1.02.39-1ubuntu6
**Hardware Details:**
Servers affected: HP Proliant DL385 (G5, G5p, G6) (and others I'm sure.)
Storage/SAN Type: iSCSI
Storage/SAN Hardware: HP StorageWorks MSA2312i (and others I'm sure.)
This problem affects servers with iSCSI-boot NICs, AND servers which have a USB-Key with /boot partition.
Please let me know if more information is necessary!
|
|
| 2011-01-10 18:18:36 |
Serge Hallyn |
attachment added |
|
Debdiff for lucid and maverick packages. https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/686832/+attachment/1790329/+files/debdiff |
|
| 2011-01-10 18:20:32 |
Serge Hallyn |
nominated for series |
|
Ubuntu Lucid |
|
| 2011-01-10 18:20:32 |
Serge Hallyn |
nominated for series |
|
Ubuntu Maverick |
|
| 2011-01-10 18:22:06 |
Serge Hallyn |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2011-01-31 19:01:16 |
Serge Hallyn |
tags |
by-uuid initramfs iscsi multipath multipath-tools-boot san |
by-uuid initramfs iscsi multipath multipath-tools-boot san verification-done |
|
| 2011-02-21 13:57:14 |
Laurent |
bug |
|
|
added subscriber Laurent |
| 2011-06-03 21:31:26 |
Serge Hallyn |
bug task added |
|
multipath-tools (Ubuntu Lucid) |
|
| 2011-06-03 21:31:39 |
Serge Hallyn |
bug task added |
|
multipath-tools (Ubuntu Maverick) |
|
| 2011-06-06 20:12:37 |
Clint Byrum |
multipath-tools (Ubuntu Lucid): status |
New |
Fix Committed |
|
| 2011-06-06 20:12:40 |
Clint Byrum |
bug |
|
|
added subscriber SRU Verification |
| 2011-06-06 20:12:43 |
Clint Byrum |
tags |
by-uuid initramfs iscsi multipath multipath-tools-boot san verification-done |
by-uuid initramfs iscsi multipath multipath-tools-boot san |
|
| 2011-06-06 20:12:44 |
Clint Byrum |
tags |
by-uuid initramfs iscsi multipath multipath-tools-boot san |
by-uuid initramfs iscsi multipath multipath-tools-boot san verification-needed |
|
| 2011-06-07 03:27:36 |
Clint Byrum |
multipath-tools (Ubuntu Lucid): importance |
Undecided |
High |
|
| 2011-06-07 03:27:36 |
Clint Byrum |
multipath-tools (Ubuntu Lucid): status |
Fix Committed |
Triaged |
|
| 2011-06-07 03:27:52 |
Clint Byrum |
tags |
by-uuid initramfs iscsi multipath multipath-tools-boot san verification-needed |
by-uuid initramfs iscsi multipath multipath-tools-boot san |
|
| 2011-06-28 13:13:55 |
Serge Hallyn |
multipath-tools (Ubuntu Maverick): status |
New |
Triaged |
|
| 2011-06-28 13:13:59 |
Serge Hallyn |
multipath-tools (Ubuntu Maverick): importance |
Undecided |
High |
|
| 2011-06-28 20:46:47 |
Serge Hallyn |
multipath-tools (Ubuntu): assignee |
Serge Hallyn (serge-hallyn) |
|
|
| 2012-05-03 10:04:37 |
JC Hulce |
multipath-tools (Ubuntu Maverick): status |
Triaged |
Invalid |
|
| 2015-06-17 11:01:07 |
Rolf Leggewie |
multipath-tools (Ubuntu Lucid): status |
Triaged |
Won't Fix |
|
