Do not list block-devices/partitions that are part of multipath devices for partitioning

[Mauricio Faria de Oliveira]

Problem:
-------

The block devices/partitions that are part of a multipath device should not be listed on partitioning dialogs.
(they are locked by the device-mapper, and are not allowed to be partitioned/formatted individually)

There's already code to do this, but it's currently not working.
The attached patch fixes it. More details/description below.

Impact:
------

The block devices/partitions that are part of a multipath device are being listed.

On 'Partition disks' manual dialog ('overview of your currently configured partitions'),
or the 'Guided partitioning -> Guided - use entire disk' dialog ('select disk to partition'):

  ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
  │ │
  │ Note that all data on the disk you select will be erased, but not │
  │ before you have confirmed that you really want to make the changes. │
  │ │
  │ Select disk to partition: │
  │ │
  │ Multipath mpath0 (WWID mpath0)Multipath (0QEMU (WWID QEMU)Multipa │
  │ SCSI1 (0,0,0) (sda) - 137.4 GB QEMU QEMU HARDDISK │
  │ SCSI1 (0,1,0) (sdb) - 137.4 GB QEMU QEMU HARDDISK │
  │ │
  │ <Go Back> │
  │ │
  └─────────────────────────────────────────────────────────────────────────┘

  ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
  │ │
  │ This is an overview of your currently configured partitions and mount │
  │ points. Select a partition to modify its settings (file system, mount │
  │ point, etc.), a free space to create partitions, or a device to │
  │ initialize its partition table. │
  │ │
  │ Multipath mpath0 (WWID mpath0)Multipath (0QEMU (WWID QEMU)Multipa │
  │ > 1.0 MB FREE SPACE ▒ │
  │ > #1 7.3 MB K │
  │ > #2 131.8 GB F ext4 / ▒ │
  │ > #3 5.6 GB F swap swap ▒ │
  │ > 1.0 MB FREE SPACE ▒ │
  │ SCSI1 (0,0,0) (sda) - 137.4 GB QEMU QEMU HARDDISK ▒ │
  │ > 1.0 MB FREE SPACE ▒ │
  │ > #1 7.3 MB ▒ │
  │ > #2 131.8 GB │
  │ │
  │ <Go Back> │
  │ │
  └─────────────────────────────────────────────────────────────────────────┘

And after the multipath device is partitioned, you'll also see LVM VGs for them.
Even though there are /none/:

~ # pvdisplay
~ # vgdisplay
  No volume groups found
~ # lvdisplay
  No volume groups found

  ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
  │ │
  │ This is an overview of your currently configured partitions and mount │
  │ points. Select a partition to modify its settings (file system, mount │
  │ point, etc.), a free space to create partitions, or a device to │
  │ initialize its partition table. │
  │ │
  │ LVM VG mpath0p1, LV mpath0p1 - 7.3 MB Linux device-mapper (linear │
  │ LVM VG mpath0p2, LV mpath0p2 - 131.8 GB Linux device-mapper (line ▒ │
  │ > #1 131.8 GB ▒ │
  │ LVM VG mpath0p3, LV mpath0p3 - 5.6 GB Linux device-mapper (linear ▒ │
  │ > #1 5.6 GB ▒ │
  │ SCSI1 (0,0,0) (sda) - 137.4 GB QEMU QEMU HARDDISK │
  │ > 1.0 MB FREE SPACE ▒ │
  │ > #1 7.3 MB K ▒ │
  │ > #2 131.8 GB ▒ │
  │ > #3 5.6 GB │
  │ │
  │ <Go Back> │
  │ │
  └─────────────────────────────────────────────────────────────────────────┘

Solution proposal:
-----------------

Part 1) The block devices (sda, sdb, etc..)

It's in function 'part_of_multipath()' at the '/lib/partman/init.d/30parted' file.
The grep expression is wrong/out of date as the 'multipath' command output changed over time.

Currently:

        # The block devices that make up the multipath:
        # Output looks like \_ 4:0:0:1 sdc 8:32 ...
        for mpdev in $(multipath -l | \
                       grep '_ \([#0-9]\+:\)\{3\}[#0-9]\+ [hs]d[a-z]\+ [0-9]\+:[0-9]\+' | \
                       cut -f4 -d' '); do

But this is the output of 'multipath -l', which has no matches with that expression:

 ~ # multipath -l
 mpath0 (0QEMU QEMU HARDDISK abcdef) dm-0 QEMU,QEMU HARDDISK
 size=128G features='0' hwhandler='0' wp=rw
 |-+- policy='round-robin 0' prio=-1 status=active
 | `- 0:0:0:0 sda 8:0 active undef running
 `-+- policy='round-robin 0' prio=-1 status=enabled
   `- 0:0:1:0 sdb 8:16 active undef running

 ~ # multipath -l | grep '_ \([#0-9]\+:\)\{3\}[#0-9]\+ [hs]d[a-z]\+ [0-9]\+:[0-9]\+'
 ~ #

Notice that the 'decoration symbols/line prefixes' changed (no longer '\_' as in that comment).
Also, it's not always identical -- there's an extra pipe '|' and space in the sda line,
but not in the sdb line (a problem for using 'cut -f " "' later).

So it seems a good idea not to match those symbols (subject to change again, and not identical on all lines),
but to match starting with the SCSI numbers, which seem less likely to change formats later.

Changes:
- grep options: use '-o' option
- grep expression: remove the heading string '_ ', starting on the SCSI numbers.

~ # multipath -l | grep -o '\([#0-9]\+:\)\{3\}[#0-9]\+ [hs]d[a-z]\+ [0-9]\+:[0-9]\+'
0:0:0:0 sda 8:0
0:0:1:0 sdb 8:16

Now it's OK to use 'cut'; with correct results:

~ # multipath -l | grep -o '\([#0-9]\+:\)\{3\}[#0-9]\+ [hs]d[a-z]\+ [0-9]\+:[0-9]\+' | cut -d " " -f2
sda
sdb

With the patch (attached) applied, the block devices part of the multipath device
are not listed anymore:

  ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
  │ │
  │ Note that all data on the disk you select will be erased, but not │
  │ before you have confirmed that you really want to make the changes. │
  │ │
  │ Select disk to partition: │
  │ │
  │ Multipath mpath0 (WWID mpath0)Multipath (0QEMU (WWID QEMU)Multipa │
  │ │
  │ <Go Back> │
  │ │
  └─────────────────────────────────────────────────────────────────────────┘

  ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
  │ │
  │ This is an overview of your currently configured partitions and mount │
  │ points. Select a partition to modify its settings (file system, mount │
  │ point, etc.), a free space to create partitions, or a device to │
  │ initialize its partition table. │
  │ │
  │ │
  │ Multipath mpath0 (WWID mpath0)Multipath (0QEMU (WWID QEMU)Multipa ▒ │
  │ > 1.0 MB FREE SPACE ▒ │
  │ > #1 7.3 MB K ▒ │
  │ > #2 131.8 GB F ext4 / ▒ │
  │ > #3 5.6 GB F swap swap ▒ │
  │ > 1.0 MB FREE SPACE ▒ │
  │ ▒ │
  │ Undo changes to partitions │
  │ Finish partitioning and write changes to disk │
  │ │
  │ <Go Back> │
  │ │
  └─────────────────────────────────────────────────────────────────────────┘

Part 2) The LVM VGs/LVs (LVM VG mpath0p1, LV mpath0p1, etc)

The part_of_multipath() function first checks is_multipath_part() (lib/partman/lib/base.sh)
to verify whether a partition is part of a multipath disk.

That is currently not working because of something related to bug 1424734 (different disk-partition
separators used by parted and that code / 'p' vs. '-part'). This is currently open/ongoing discussion.

It's possible to add code to handle both cases/disk-separators in that function.
The patch adds that too.
And now the non-existent LVM VGs/LVs are no longer listed.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "partman-base_multipath_part_listing.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Mauricio Faria de Oliveira (mfo)]

Hi,

Hold on for the change to lib/base.sh; sorry.
It causes a regression on the dialog.

[Mauricio Faria de Oliveira (mfo)]

Got it.

The regression on the dialog is not a problem in the attached patch.
It's actually hit because the attached patch uncovered another problem, again related to bug 1424734 (different multipath disk-partition separators / 'p' vs '-part').
This probably requires another patch or discussion for the behavior described below.

The regression was that, after the partman confirmation dialog ("confirm_changes partman", in /bin/partman), rather then proceeding to the formatting stage, the installer goes back to the partitioning dialog.

It's caused because the /lib/partman/commit.d/31multipath_partition script is now exiting with non-zero exit code, making /bin/partman stop processing the other scripts in commit.d/, including the formatting ones.

The non-zero exit code comes from a kpartx command, that fails due to the different disk-partition separators (see bug 1424734), yielding a 'device busy' error in syslog.

If the attached patch is not in place, this non-zero exit code is supressed by other (incorrect) iterations of the for loop..
Other partman's /device/ dirs are created for each /partition/ (which is incorrect).
Then, the for loop just goes over the failed kpartx command, and that non-zero exit code is forgotten.

One can observe that with the trace of /lib/partman/commit.d/31multipath_partition.
(add "set -x; exec 2>/31multipath_partitions.stderr" before the for-loop, and 'exit $?' after it)
Check that the for-loop goes over the kpartx error when there are devices for each mpath0 partition (mpath0p1, mpath0p2 and mpath0p3).. but exits with the error when they're not there anymore (as that's the last iteration in the for-loop)

Without the patch:

 ~ # cat /31partition_multipath.stderr
 + [ -d /var/lib/partman/devices/=dev=mapper=mpath0 ]
 + cd /var/lib/partman/devices/=dev=mapper=mpath0
 + [ -f device ]
 + [ -f multipath ]
 + cat device
 + kpartx -d -p -part /dev/mapper/mpath0
 + cat device
 + kpartx -a -p -part /dev/mapper/mpath0
 device-mapper: create ioctl on mpath0-part1 failed: Device or resource busy
 create/reload failed on mpath0-part1
 device-mapper: create ioctl on mpath0-part2 failed: Device or resource busy
 create/reload failed on mpath0-part2
 device-mapper: create ioctl on mpath0-part3 failed: Device or resource busy
 create/reload failed on mpath0-part3
 + [ -d /var/lib/partman/devices/=dev=mapper=mpath0p1 ]
 + cd /var/lib/partman/devices/=dev=mapper=mpath0p1
 + [ -f device ]
 + [ -f multipath ]
 + continue
 + [ -d /var/lib/partman/devices/=dev=mapper=mpath0p2 ]
 + cd /var/lib/partman/devices/=dev=mapper=mpath0p2
 + [ -f device ]
 + [ -f multipath ]
 + continue
 + [ -d /var/lib/partman/devices/=dev=mapper=mpath0p3 ]
 + cd /var/lib/partman/devices/=dev=mapper=mpath0p3
 + [ -f device ]
 + [ -f multipath ]
 + continue
 + exit 0

With the patch:

 ~ # cat /31partition_multipath.stderr
 + [ -d /var/lib/partman/devices/=dev=mapper=mpath0 ]
 + cd /var/lib/partman/devices/=dev=mapper=mpath0
 + [ -f device ]
 + [ -f multipath ]
 + cat device
 + kpartx -d -p -part /dev/mapper/mpath0
 + cat device
 + kpartx -a -p -part /dev/mapper/mpath0
 device-mapper: create ioctl on mpath0-part1 failed: Device or resource busy
 create/rel...

[Mauricio Faria de Oliveira (mfo)]

Adding a patch for the issue described in the partman-multipath's commit.d/partition_multipath script.