Fix handling of multipathed disks with 4k-sectors

[Mauricio Faria de Oliveira]

The packages for libparted and kpartx don't have the patches for using multipath (actually device-mapper) on 4k-sector disks (generally: 'non 512-byte sector disks')

This causes the number of sectors of each device-mapper partition to be 1/8th (512/4096) of that expected.
These 3 errors were identified:
1) Partition sizes on multipath are 1/8th of the intended size
2) The filesystem on the underlying device is not-accessible (only in the dm mapped device)
3) grub-install fails w/ 'Unknown filesystem'

The 2 patches attached contains upstream patches from parted and multipath-tools, and fix the issues (the installation finishes without errors and the system boots).

QEMU/KVM
--------

If you don't have 4k-sector disks handy :)

An example for multipath on 4k-sector disks, with 2 underlying devices:

 $ qemu-img create -f raw disk.img 16g

 $ qemu-system-ppc64 -enable-kvm \
   -nographic -nodefaults \
   -monitor stdio -serial pty \
   -M pseries -m 2G \
   -net nic -net user \
   -drive media=cdrom,file=../../iso/vivid-server-ppc64el.iso.0408 \
   \
   -drive file=disk.img,if=none,cache=none,serial=my-serial,id=drive0 \
   -drive file=disk.img,if=none,cache=none,serial=my-serial,id=drive1 \
   -device scsi-hd,physical_block_size=4096,logical_block_size=4096,drive=drive0 \
   -device scsi-hd,physical_block_size=4096,logical_block_size=4096,drive=drive1

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "multipath-tools_non512Bsectors.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Mauricio Faria de Oliveira (mfo)]

I incorrectly mentioned LVM was also affected by the issue.

It doesn't, as the dm mappings are the LVs mapped on top of VGs, which already take sectors as 512B AFAICT.
The issue arises w/ multipath because then the partitions are mapped directly on top of the underlying devices, which have 4k sectors. Since the values being passed to libdevmapper should be in 512b sectors, if there's no multiplier of 4k/512 there, things go wrong. :)

[Mauricio Faria de Oliveira (mfo)]

Description fixed.

[Mauricio Faria de Oliveira (mfo)]

BTW, but not too important:

There's a quite repetitive verbose output during update-grub, mentioning:
  ERROR: unsupported sector size 4096 on <partition>.

It happens because os-prober calls dmraid for every partition, but it doesn't support 4k sectors, and yields to stderr.
A simple redirect quiets it up.
... I'm not sure it should be quieted up :) feel free to modify the scripts if you think so, or tell me and I can submit another patch.

 ~ # dmraid -r -c
 ERROR: unsupported sector size 4096 on /dev/sda.
 ERROR: unsupported sector size 4096 on /dev/sdb.
 ERROR: unsupported sector size 4096 on /dev/sdc.
 ERROR: unsupported sector size 4096 on /dev/sdd.
 ERROR: unsupported sector size 4096 on /dev/sde.
 ERROR: unsupported sector size 4096 on /dev/sdf.
 ERROR: unsupported sector size 4096 on /dev/sdg.
 ERROR: unsupported sector size 4096 on /dev/sdh.
 ERROR: unsupported sector size 4096 on /dev/sdi.
 ERROR: unsupported sector size 4096 on /dev/sdj.
 ERROR: unsupported sector size 4096 on /dev/sdk.
 ERROR: unsupported sector size 4096 on /dev/sdl.
 ERROR: unsupported sector size 4096 on /dev/dm-0.
 ERROR: unsupported sector size 4096 on /dev/dm-1.
 ERROR: unsupported sector size 4096 on /dev/dm-2.
 ERROR: unsupported sector size 4096 on /dev/dm-3.
 ERROR: unsupported sector size 4096 on /dev/dm-4.
 ERROR: unsupported sector size 4096 on /dev/dm-5.
 ERROR: unsupported sector size 4096 on /dev/dm-6.
 ERROR: unsupported sector size 4096 on /dev/dm-7.
 ERROR: unsupported sector size 4096 on /dev/dm-8.
 ERROR: unsupported sector size 4096 on /dev/dm-9.
 ERROR: unsupported sector size 4096 on /dev/dm-10.
 ERROR: unsupported sector size 4096 on /dev/dm-11.
 ERROR: unsupported sector size 4096 on /dev/dm-12.
 ERROR: unsupported sector size 4096 on /dev/dm-13.
 ERROR: unsupported sector size 4096 on /dev/dm-14.
 ERROR: unsupported sector size 4096 on /dev/dm-15.
 ERROR: unsupported sector size 4096 on /dev/dm-16.
 ERROR: unsupported sector size 4096 on /dev/dm-17.
 no block devices found
 ~ #

 ~ # dmraid -r -c 2>/dev/null
 no block devices found

[Mauricio Faria de Oliveira (mfo)]

Back to comment #5.
If it's possible to install LVM on top of multipath, then I think the problem shall happen too, because then the first level of mapping is multipath on top of the underlying devices (4k sectors).

[Mauricio Faria de Oliveira (mfo)]

Description updated w/ QEMU/KVM command line for 4k-sector disks.
It reproduces this bug and the fix.

 [ 0.221249] scsi 0:0:1:0: Direct-Access QEMU QEMU HARDDISK 2.1. PQ: 0 ANSI: 5
 [ 0.221382] scsi 0:0:2:0: Direct-Access QEMU QEMU HARDDISK 2.1. PQ: 0 ANSI: 5
 ...
 [ 0.254668] sd 0:0:1:0: [sda] 4194304 4096-byte logical blocks: (17.1 GB/16.0 GiB)
 ...
 [ 0.255837] sd 0:0:2:0: [sdb] 4194304 4096-byte logical blocks: (17.1 GB/16.0 GiB)

 ~ # blockdev --getss /dev/sda
 4096
 ~ # blockdev --getss /dev/sdb
 4096

 ~ # cmp /sys/block/sd[ab]/device/vpd_pg83; echo $?
 0

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.4.9-3ubuntu12

---------------
multipath-tools (0.4.9-3ubuntu12) vivid; urgency=medium

  [ Mauricio Faria de Oliveira ]
  * Support disks with non 512-byte sectors (LP: #1441930)
    - Added 0016-kpartx-non-512B-GPT.patch
    - Added 0017-kpartx-non-512B-sectors.patch

  [ Mathieu Trudel-Lapierre ]
  * debian/patches/0018-ext-part-gt-512B-sectors.patch: fix extended partition
    size for > 512B sectors. Patch cherry-picked from git commit 7202f225.
 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 13 Apr 2015 09:23:29 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package parted - 3.2-7ubuntu1

---------------
parted (3.2-7ubuntu1) vivid; urgency=medium

  [ Mauricio Faria de Oliveira ]
  * Support disks with non 512-byte sectors (LP: #1441930)
    - Added dm-512b-sectors.patch: fix partition sizes for device-mapper (multipath/lvm devices).
 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 13 Apr 2015 09:20:41 -0500

[Mauricio Faria de Oliveira (mfo)]

Thanks a bunch, Mathieu. ;-)
Hopefully this should be in tomorrow's or the day after's daily build, and test teams can take it on.

[Mauricio Faria de Oliveira (mfo)]

@mathieu-tl

Can this be backported to Trusty?

[Mauricio Faria de Oliveira (mfo)]

Attaching backports for 14.04 series.
- multipath-tools: trivial backport, only changes to packaging files.
- parted: some changes required (context lines in _dm_add_partition, and removal of _dm_resize_partition changes, as this function doesnt exist on parted version on 14.04).

[Mauricio Faria de Oliveira (mfo)]

The backports are based on the final diff applied to multipath-tools and parted, obtained from the source package pages, link 'diff from X to Y'.

[Mauricio Faria de Oliveira (mfo)]

Tested on 14.04.3 daily build.

Things go from this:

 ~ # df -h /target
 Filesystem Size Used Available Use% Mounted on
 /dev/mapper/mpath0p2 31.1G 47.9M 29.4G 0% /target

 ~ # cat /proc/swaps
 Filename Type Size Used Priority
 /dev/mapper/mpath0p3 partition 1408448 0 -1

 ~ # dmsetup table | grep ^mpath0
 mpath0: 0 554287104 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:0 1000 round-robin 0 1 1 65:144 1000
 mpath0p3: 0 2817024 linear 252:0 66468608
 mpath0p2: 0 66466560 linear 252:0 2048
 mpath0p1: 0 1792 linear 252:0 256

To this:

 ~ # df -h /target
 Filesystem Size Used Available Use% Mounted on
 /dev/mapper/mpath0p2 249.4G 59.1M 236.7G 0% /target

 ~ # cat /proc/swaps
 Filename Type Size Used Priority
 /dev/mapper/mpath0p3 partition 11268032 0 -1

 ~ # dmsetup table | grep ^mpath0
 mpath0-part2: 0 531732480 linear 252:0 16384
 mpath0-part1: 0 14336 linear 252:0 2048
 mpath0: 0 554287104 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:0 1000 round-robin 0 1 1 65:144 1000
 mpath0p3: 0 22536192 linear 252:0 531748864
 mpath0p2: 0 531732480 linear 252:0 16384
 mpath0p1: 0 14336 linear 252:0 2048
 mpath0-part3: 0 22536192 linear 252:0 531748864

 (ignore -part and p separators, this is another bug which should be backported for as well)

[Chris J Arges (arges)]

Hello Mauricio, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mathieu Trudel-Lapierre (cyphermox)]

Testing this on Trusty (and for some reason, my overlay not working), it looks as through 4k disks are already working properly in Trusty.

@Mauricio, could you please confirm this? I've prepared a patched parted in my installer-dev PPA, but it seems like it's not necessary for trusty.

[Mauricio Faria de Oliveira (mfo)]

@mathieu-tl

You're testing w/ multipath-tools from -proposed?

If that's the case and the disk is already partitioned, I expect it to work, as kpartx is patched and will use the right offsets for dmsetup.

If the disk is not partioned, thus partitioned w/ parted, then kpartx won't make a difference, as libparted creates the dm maps. Then a patched parted is necessary.

The multipath-tools / parted versions before -proposed failed on both cases for me.

Thanks!

[Mathieu Trudel-Lapierre (cyphermox)]

Let's re-test with my PPA.

[Diane Brent (drbrent)]

Will this make the 8/06/15 14.04.3 release? current course and speed.

[Brian Murray (brian-murray)]

Hello Mauricio, or anyone else affected,

Accepted parted into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/parted/2.3-19ubuntu1.14.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mathieu Trudel-Lapierre (cyphermox)]

@drbrent, provided verification passes for both the affected packages (and that, as soon as possible), there is enough time for this update to make it into 14.04.3.

parted changes needed to be vastly adapted to apply on parted 2.3. Be warned that you'll need to enable the proposed repository *in the installer* for this to be testable with the installer, which means you will need to add "apt-setup/proposed=true" to the command-line when booting the installer, or add it to the preseed file you are using. See https://wiki.ubuntu.com/Testing/EnableProposed#Installation_testing_using_-proposed

[Mauricio Faria de Oliveira (mfo)]

I can confirm the packages in -proposed for 14.04.x fix the problem.
Network installation tested with apt-setup/proposed=true.
Verification tags changed.

Thanks!

[Adam Conrad (adconrad)]

Hello Mauricio, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/multipath-tools/0.4.9-3ubuntu7.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mauricio Faria de Oliveira (mfo)]

Hello,

We can consider the new new package (local version ubuntu7.4) already tested/verified.

It just reverts an USB-related patch in the multipath device discovery function, whose changes are not exercised by/related to the 4k-sector support/functionality (which touches the kpartx partition geometry functions).

Thanks!

[Mauricio Faria de Oliveira (mfo)]

BTW, I'd like to mention I tested/exercised both multipath-tools (kpartx) and parted (libparted) patches by verifying both the already-existing partitions (kpartx) and creating new partitions (libparted) cases.
The partitions devmaps had correct number of sectors in both cases.

[Launchpad Janitor (janitor)]

This bug was fixed in the package parted - 2.3-19ubuntu1.14.04.1

---------------
parted (2.3-19ubuntu1.14.04.1) trusty; urgency=medium

  * debian/patches/dm-512b-sectors.patch: properly support 4k disks.
    (LP: #1441930)

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 23 Jul 2015 12:12:21 -0400

[Adam Conrad (adconrad)]

The verification of the Stable Release Update for parted has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package multipath-tools - 0.4.9-3ubuntu7.4

---------------
multipath-tools (0.4.9-3ubuntu7.4) trusty; urgency=medium

  * Remove 0024-ignore-usb.patch: Ignore USB devices. Verification fails
    for this fix; it needs more work.

multipath-tools (0.4.9-3ubuntu7.3) trusty; urgency=medium

  * Added debian/patches/0015-shared-lock-for-udev.patch (LP: #1431650)
  * Support disks with non 512-byte sectors (LP: #1441930)
  * Correctly write FC timeout attributes to sysfs. (LP: #1435706)
  * Ignore USB devices. (LP: #1468897)

 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 27 Jul 2015 13:48:39 -0400