Comment 20 for bug 1887558

Revision history for this message
Mitchell Dzurick (mitchdz) wrote : Re: Multipath JBOD storage devices are not shown via /dev/mapper but each path as a single device.

Fundamentally my setup is different than the scenario set up in the bug. I was initially confused why multipathing was even happening, so I will explain the scenario here. In case someone else is also confused.

The hardware is a JBOD enclosure where each disk has redundant I/O, hence the multipathing.

On boot, multipath correctly groups
* /dev/sd{a,b} -> /dev/mapper/mpatha
* /dev/sd{c,d} -> /dev/mapper/mpathb
* /dev/sd{e,f} -> /dev/mapper/mpathc
* /dev/sd{g,h} -> /dev/mapper/mpathd

At this point, the system has 4 NVME drives to use as the caching device, where the JBOD drives are the backing device.
make-bcache -C /dev/nvme0n1p1 -B /dev/mapper/mpatha --writeback
make-bcache -C /dev/nvme1n1p1 -B /dev/mapper/mpathb --writeback
make-bcache -C /dev/nvme2n1p1 -B /dev/mapper/mpathc --writeback
make-bcache -C /dev/nvme3n1p1 -B /dev/mapper/mpathd --writeback

At this point, everything is happy. We have multipathd devices inside a JBOD enclosure, where the physical disks are tagged as bcache devices.

The issue seems to be happening when you reboot the system, and the bcache udev rules grab the drives and multipath fails to make mpath{a,b,c,d} - because the drives are in use by bcache.

I don't think I see this on my system because this is an issue during boot when multipathd is not running.

With that said, the fact that the disks are part of a JBOD enclosure I don't think is an issue here, rather on boot disks with redundant i/o that are marked with bcache FS do not get initialized correctly, so I think there is still a way to reproduce this in a VM which would be nice so we can port this into a proper dep8 tests.

The problem with reboot testing in my setup is that on boot the virtual iscsi device is not available, I have to login manually. I'm not the most familiar with tgtadm and iscsiadm so not sure if these tools can make the device available at boot time which seems to be needed for this test.