multipath does not set fiberchannel params correctly

Bug #1578654 reported by Dan Streetman
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
multipath-tools (Ubuntu)
Fix Released
Medium
Dan Streetman
Trusty
Fix Released
Medium
Dan Streetman

Bug Description

[Impact]

multipath has configuration settings such as "dev_loss_tmo" and "fast_io_fail_tmo" which multipath uses to change the underlying device's parameters. However, it fails to set any of them; it is trying to write the parameters to the sysfs device's directory, instead of the attribute file inside the dir. This results in configuration parameters that the user provides not having any effect at all, and no error logs being shown.

The commit 050b24b33d3c60e29f7820d2fb75e84a9edde528 ("multipath: fix setting sysfs fc timeout parameters") from upstream multipath-tools fixes the bug by actually using the sysfs attribute files, instead of the sysfs device directory.

[Test Case]

A trusty system with fiberchannel drive(s), with multiple redundant paths to the drive(s), is required. The FC drives must be configured and appear in the system, i.e. as /dev/sd* or similar node that multipath scans.

Note that multipath-tools in trusty does not (yet) support setting sysfs attributes for iSCSI devices, only FC.

Check the current value, replace 'rport-N:N-N' with the rport number, e.g. 'rport-0:0-1':

# cat /sys/class/fc_remote_ports/rport-N:N-N/fast_io_fail_tmo

Install multipath-tools, and edit (or create) the /etc/multipath.conf file. Add into the defaults section the setting for fast_io_fail_tmo; if it's already 5, replace 5 shown in the example with a different value than it's currently set to:

defaults {
  fast_io_fail_tmo 5
}

If the multipath-tools-boot package is installed, rebuild the initramfs (with update-initramfs -u).

Reboot for the parameter to take effect.

After the system boots back up, multipath should have the drive(s) listed, e.g. in its 'multipath -ll' listing.

Check the sysfs node that should have been modified (again replace rport-* with the correct numbers):

# cat /sys/class/fc_remote_ports/rport-N:N-N/fast_io_fail_tmo

It should show the value that was configured in multipath.conf.

[Regression Potential]

none. This fixes multipath to correctly set sysfs attributes, as it did not work at all before.

Revision history for this message
Dan Streetman (ddstreet) wrote :
Changed in multipath-tools (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Dan Streetman (ddstreet) wrote :

bug is only in trusty; precise has older code that is not broken, and wily/xenial already have this commit.

Dan Streetman (ddstreet)
description: updated
description: updated
summary: - multipath does not set sysfs params correctly
+ multipath does not set fiberchannel params correctly
Revision history for this message
Dan Streetman (ddstreet) wrote :

Verified without patch, multipath does not change fiberchannel dev_loss_tmo parameter, and with patch multipath does change parameter.

Dan Streetman (ddstreet)
Changed in multipath-tools (Ubuntu Trusty):
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Dan Streetman (ddstreet) wrote :
Revision history for this message
Dan Streetman (ddstreet) wrote :

second version of the debdiff attached, updated as an unrelated patch was added to multipath while working on this bug.

Louis Bouchard (louis)
tags: added: sts-sponsor
Revision history for this message
Dan Streetman (ddstreet) wrote :

v3: same as previous debdiff, but with (LP: #1578654) corrected in the debian/changelog (was missing "#" in v2)

Louis Bouchard (louis)
tags: added: sts-sru
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Dan, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.13 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in multipath-tools (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed
Dan Streetman (ddstreet)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Dan Streetman (ddstreet) wrote :

Tester reported:

Looks good to me.

I enabled the proposed repository, and downgraded multipath-tools-boot and associated packages to 0.4.9-3ubuntu7.12. I rebooted and confirmed that the dev_loss_tmos were stuck at 30 despite the config saying to put them at 5. Then I ran an upgrade which pulled version 0.4.9-3ubuntu7.13 down and rebooted again. When it came back up, they were set to 5 as expected.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package multipath-tools - 0.4.9-3ubuntu7.13

---------------
multipath-tools (0.4.9-3ubuntu7.13) trusty; urgency=medium

  * debian/patches/0001-multipath-fix-setting-sysfs-fc-timeout-parameters.patch:
    multipath was not setting fiberchannel sysfs params correctly. (LP: #1578654)

 -- Dan Streetman <email address hidden> Tue, 07 Jun 2016 15:23:21 +0200

Changed in multipath-tools (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for multipath-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Louis Bouchard (louis)
tags: removed: sts-sponsor
Louis Bouchard (louis)
tags: removed: sts-sru
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.