constantly changes /dev/disk/by-id/{scsi,wwn}-* LUN symlinks with multipathing
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu |
Invalid
|
Undecided
|
Unassigned | ||
Lucid |
Invalid
|
Undecided
|
Unassigned | ||
Maverick |
Invalid
|
Undecided
|
Unassigned | ||
Natty |
Invalid
|
Undecided
|
Unassigned | ||
multipath-tools (Ubuntu) |
Fix Released
|
High
|
Peter Petrakis | ||
Lucid |
Fix Released
|
Undecided
|
Unassigned | ||
Maverick |
Fix Released
|
Undecided
|
Unassigned | ||
Natty |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
= SRU Justification =
== Impact ==
Multipath-tools is inadvertedly generating UDEV CHANGE events for the SD
block devices under it's control. These change events feedback into the udev
rules, increasing cpu utlilzation, and ruining the multipath aliasing feature,
which allows one to rename a multipath path from a series of letters and
numbers, to a human readable label. It gives users the impression that
the SAN is unstable.
== Solution ==
Change the open() flags in the priority checkers to read only from read write, this
stops sd from generating a change event after the file descriptor has been
closed.
== Reproduction ==
Is easy, and doesn't even require a SAN. Since we're dealing with simple SCSI
inquiry cmds any block device will do. Simply install multipath-tools and
execute one of the priority checkers like so:
/sbin/mpath_
also, have a window open monitoring udev, udevadm monitor, ensure
no change events to that block device are occurring before hand.
TEST CASE:
root@kickseed:~# udevadm monitor &
[1] 16950
root@kickseed:~# monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent
root@kickseed:~#
root@kickseed:~# /sbin/mpath_
query command indicates error0
root@kickseed:~# KERNEL[
UDEV [1308688009.823569] change /devices/
root@kickseed:~#
root@kickseed:~#
root@kickseed:~# /sbin/mpath_
130
mpath_prio_alua doesn't generate any change events since it's open
flags do not include O_RDRW to begin with.
== regression potential ==
None, it's broken to begin with.
-------
Binary package hint: udev
udevd constantly changes LUN device node symlinks (devices/LUNs, not the partition nodes) in /dev/disk/by-id. udevd uses ~15% of CPU and system time is using ~50-60%.
For example:
[<email address hidden>:pts/0 /dev/disk/by-id> ls -l wwn-0x60a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 scsi-360a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 scsi-360a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 wwn-0x60a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 wwn-0x60a980004
======
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 scsi-360a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 scsi-360a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 wwn-0x60a980004
lrwxrwxrwx 1 root root 9 2010-09-21 16:12 wwn-0x60a980004
All other device nodes stay the same, such as the device nodes for the partitions:
[<email address hidden>:pts/0 /dev/disk/by-id> ls -l scsi-360a980004
lrwxrwxrwx 1 root root 10 2010-09-21 15:47 scsi-360a980004
======
lrwxrwxrwx 1 root root 10 2010-09-21 15:47 scsi-360a980004
I'm not entirely sure whether this is udev's problem or something related to multipathing. Our most recent experience with multipathing is the last LTS release (hardy), which doesn't exhibit this behavior given similar configurations.
[<email address hidden>:pts/0 ~> sudo multipath -ll
rootvol (360a98000486e5
[size=36G]
\_ round-robin 0 [prio=8][active]
\_ 2:0:2:0 sda 8:0 [active][ready]
\_ 3:0:2:0 sde 8:64 [active][ready]
\_ round-robin 0 [prio=2][enabled]
\_ 3:0:3:0 sdg 8:96 [active][ready]
\_ 2:0:3:0 sdc 8:32 [active][ready]
syslog-data (360a98000486e5
[size=1.
\_ round-robin 0 [prio=8][active]
\_ 2:0:2:1 sdb 8:16 [active][ready]
\_ 3:0:2:1 sdf 8:80 [active][ready]
\_ round-robin 0 [prio=2][enabled]
\_ 3:0:3:1 sdh 8:112 [active][ready]
\_ 2:0:3:1 sdd 8:48 [active][ready]
[<email address hidden>:pts/0 ~> cat /etc/multipath.conf
multipaths {
multipath {
}
multipath {
}
}
devices {
device {
}
}
Changed in multipath-tools (Ubuntu): | |
importance: | Undecided → High |
status: | New → Confirmed |
Changed in multipath-tools (Ubuntu): | |
status: | Confirmed → In Progress |
Changed in udev (Ubuntu): | |
status: | New → Invalid |
Changed in multipath-tools (Ubuntu): | |
status: | In Progress → Fix Released |
Changed in multipath-tools (Ubuntu Lucid): | |
status: | New → Confirmed |
Changed in multipath-tools (Ubuntu Maverick): | |
status: | New → Confirmed |
Changed in multipath-tools (Ubuntu Natty): | |
status: | New → Confirmed |
Changed in ubuntu: | |
status: | Invalid → Confirmed |
tags: |
added: verification-done removed: verification-done-natty verification-needed |
tags: | added: testcase |
I should mention that this is with fully up-to-date lucid.