Setting "prio const" in multipath.conf has no effect

[Vassilis Vatikiotis]

We are trying to activate the active/active configuration in multipathd, for 2 paths between an AMS2100 array and a Ubuntu 12.04 Server.

We set "prio const" in /etc/multipath.conf. After starting the multipath daemon, we observe the following:
a) # multipath -ll
mpath0 (360060e80104dac0004f349c800000000) dm-0 HITACHI,DF600F
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 8:0:0:0 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
  `- 7:0:0:0 sdb 8:16 active ready running

Paths have different priorities.

b) multipath -v4
Jun 15 16:39:01 | sdb: not found in pathvec
Jun 15 16:39:01 | sdb: mask = 0x1f
Jun 15 16:39:01 | sdb: dev_t = 8:16
Jun 15 16:39:01 | sdb: size = 209715200
Jun 15 16:39:01 | sdb: subsystem = scsi
Jun 15 16:39:01 | sdb: vendor = HITACHI
Jun 15 16:39:01 | sdb: product = DF600F
Jun 15 16:39:01 | sdb: rev = 0000
Jun 15 16:39:01 | sdb: h:b:t:l = 7:0:0:0
Jun 15 16:39:01 | sdb: serial = 830530000000
Jun 15 16:39:01 | sdb: get_state
Jun 15 16:39:01 | loading /lib/multipath/libchecktur.so checker
Jun 15 16:39:01 | sdb: path checker = tur (controller setting)
Jun 15 16:39:01 | sdb: state = running
Jun 15 16:39:01 | sdb: state = 3
Jun 15 16:39:01 | loading /lib/multipath/libpriohds.so prioritizer
Jun 15 16:39:01 | sdb: prio = hds (controller setting)
Jun 15 16:39:01 | sdb: prio args = (null) (controller setting)
Jun 15 16:39:01 | sdb: hds prio: VENDOR: HITACHI
Jun 15 16:39:01 | sdb: hds prio: PRODUCT: DF600F
Jun 15 16:39:01 | sdb: hds prio: SERIAL: 0x3000
Jun 15 16:39:01 | sdb: hds prio: LDEV: 0x0000
Jun 15 16:39:01 | sdb: hds prio: CTRL: 1
Jun 15 16:39:01 | sdb: hds prio: PORT: E
Jun 15 16:39:01 | sdb: hds prio: CTRL ODD, LDEV EVEN, PRIO 0 <---------------- PRIO HERE
Jun 15 16:39:01 | sdb: hds prio = 0
Jun 15 16:39:01 | sdb: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting)
Jun 15 16:39:01 | sdb: uid = 360060e80104dac0004f349c800000000 (callout)
Jun 15 16:39:01 | Discover device /sys/block/sdc
Jun 15 16:39:01 | sdc: not found in pathvec
Jun 15 16:39:01 | sdc: mask = 0x1f
Jun 15 16:39:01 | sdc: dev_t = 8:32
Jun 15 16:39:01 | sdc: size = 209715200
Jun 15 16:39:01 | sdc: subsystem = scsi
Jun 15 16:39:01 | sdc: vendor = HITACHI
Jun 15 16:39:01 | sdc: product = DF600F
Jun 15 16:39:01 | sdc: rev = 0000
Jun 15 16:39:01 | sdc: h:b:t:l = 8:0:0:0
Jun 15 16:39:01 | sdc: serial = 830530000000
Jun 15 16:39:01 | sdc: get_state
Jun 15 16:39:01 | sdc: path checker = tur (controller setting)
Jun 15 16:39:01 | sdc: state = running
Jun 15 16:39:01 | sdc: state = 3
Jun 15 16:39:01 | sdc: prio = hds (controller setting)
Jun 15 16:39:01 | sdc: prio args = (null) (controller setting)
Jun 15 16:39:01 | sdc: hds prio: VENDOR: HITACHI
Jun 15 16:39:01 | sdc: hds prio: PRODUCT: DF600F
Jun 15 16:39:01 | sdc: hds prio: SERIAL: 0x3000
Jun 15 16:39:01 | sdc: hds prio: LDEV: 0x0000
Jun 15 16:39:01 | sdc: hds prio: CTRL: 2
Jun 15 16:39:01 | sdc: hds prio: PORT: E
Jun 15 16:39:01 | sdc: hds prio: CTRL EVEN, LDEV EVEN, PRIO 1 <--------------- PRIO HERE
Jun 15 16:39:01 | sdc: hds prio = 1
Jun 15 16:39:01 | sdc: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting)
Jun 15 16:39:01 | sdc: uid = 360060e80104dac0004f349c800000000 (callout)

In this output, the 2 paths have different priorities and even if we have set "prio const" in our configuration file, multipathd sets prio hds on its own.

c) # echo "show config" | multipathd -k
 device {
  vendor "HITACHI "
  product "DF600F.*"
  path_grouping_policy multibus
  getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
  path_checker tur
  checker tur
  prio const
  no_path_retry queue
 }
This is the last device entry in the configuration of the running daemon.

d) iostat -m 2
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 775.50 96.72 0.00 193 0
dm-1 0.00 0.00 0.00 0 0
dm-2 0.00 0.00 0.00 0 0
dm-3 0.00 0.00 0.00 0 0
sdb 0.00 0.00 0.00 0 0
sdc 775.50 96.72 0.00 193 0 <----- All I/O on one device only.

NOTE: AMS2100 and higher models support active/active architecture and all paths are considered owner paths and can accept I/O from all available/assigned ports.

Below are system specifics:
1) lsb_release -rd
Description: Ubuntu 12.04 LTS
Release: 12.04

2) Package version
multipath-tools:
  Installed: 0.4.9-3ubuntu5
  Candidate: 0.4.9-3ubuntu5
  Version table:
 *** 0.4.9-3ubuntu5 0
        500 http://gr.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status

[Peter Petrakis (peter-petrakis)]

Please provide the entire multipath.conf file, in the meanwhile try stopping the
daemon, perform a table flush (multipath -F) , then simply run multipath -v4
(no daemon) and observe the results.

[Vassilis Vatikiotis (vvatikiotis)]

multipath.conf follows. The file is heavily commented so I trimmed all comments out

defaults {
 user_friendly_names yes
}
blacklist {
 devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[-1-9]*"
 devnode "^hd[a-z][[0-9]*]"
 wwid 3600508b1001cea3b91ee45c710e467a7 #sda
 devnode "sr0"
}

devices {
 device {
  vendor "HITACHI "
  product "DF600F.*"
  no_path_retry queue
  prio const
  path_grouping_policy multibus
  getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
  checker tur
 }
}

Here's the _full_ output of multipath -ll
# multipath -ll
Error: : Inappropriate ioctl for device
cciss TUR failed in CCISS_GETLUNINFO: Inappropriate ioctl for device <----
mpath0 (360060e80104dac0004f349c800000000) dm-0 HITACHI,DF600F
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 5:0:0:0 sdc 8:32 active ready running
`-+- policy='round-robin 0' prio=0 status=enabled
  `- 4:0:0:0 sdb 8:16 active ready running

I cannot flush even when the multipath and iscsi daemon are stopped.
# multipath -F
Jul 10 20:46:12 | mpath0: map in use

mpath0 is used by device-mapper
# dmsetup ls
mpath0-part1 (252, 1)
mpath0 (252, 0)

[Peter Petrakis (peter-petrakis)]

If your device is mounted, you won't be able to flush it, flush == destroy.

OK. so it's not just you, I can recreate this.

Coincidentally, I also have an HP RAID controller and see the same warning
message. This device *is blacklisted* but apparently it's still getting probed.
Shouldn't be causing any problems, though blacklisted should mean what it says.

[Rafael David Tinoco (rafaeldtinoco)]

Thank you for taking the time to report this bug. In an effort to keep an up-to-date and valid list of bugs to work on, I have reviewed this report to verify it still requires effort and occurs on an Ubuntu release in standard support, and it seems it does not.

It is unfortunate that we were unable to resolve this defect, however there appears to be no further action possible at this time. I am therefore moving the bug to 'Won't fix'.

If you disagree or have new information, we would be grateful if you could please add a comment stating why and then change the status of the bug to 'New', perhaps providing a reproducer in a supported Ubuntu release.