linux: 4.6 kernel fails to boot on ppc64el multi-path system

[Tim Gardner]

https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/9759880/+files/linux-image-4.6.0-6-generic_4.6.0-6.7_ppc64el.deb
https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/9759880/+files/linux-image-extra-4.6.0-6-generic_4.6.0-6.7_ppc64el.deb

Provisioned modoc (ppc64el) with stock Xenial, then installed the above kernel. It eventually times out attempting to mount rootfs and drops to the busybox prompt.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1588421

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Stefan Bader (smb)]

Initial data gathered:
- The base disks seem present
- The multipath modules, too (afaiks)
  dm_round_robin
  dm_multipath
  dm_round_robin
  scsi_dh_alua
- Trying to manually create the multipath mappings fails
  somewhere related to the ALUA device handler.
[ 526.285984] sd 0:2:0:0: alua: supports implicit TPGS
[ 526.286088] sd 0:2:0:0: alua: No device descriptors found
[ 526.286135] sd 0:2:0:0: alua: Attach failed (-22)
[ 526.286192] device-mapper: table: 252:0: multipath: error attaching hardware handler

[Stefan Bader (smb)]

Comparing that to the working 4.4 kernel it looks like device handler fails to receive valid data. The info should look like this:

[ 1.653229] sd 0:2:0:0: alua: supports implicit TPGS
[ 1.653395] sd 0:2:0:0: alua: port group c339 rel port c339
[ 1.653426] sd 0:2:0:0: alua: rtpg failed with 8070002
[ 1.653459] sd 0:2:0:0: alua: port group c339 state A preferred supports TOlUSNA
[ 1.653616] sd 0:2:0:0: Attached scsi generic sg13 type 0

While the exact symptoms differ, this sounds a bit like there might be some relation to the device scan problems that are seen on zSeries (LP: #1567602).

[Stefan Bader (smb)]

Or maybe not as 4.6 seems to fix that race. Though I found one change in the ALUA handler post 4.6 which modifies the handling of missing VPD identification (which leads to the "No device descriptors found" message):

commit fe8b9534a0a0356f8a76467e2c561194bdb53c84
Author: Hannes Reinecke <email address hidden>
Date: Fri May 6 10:34:35 2016 +0200

    scsi_dh_alua: do not fail for unknown VPD identification

    Not every device will return a useable VPD identification, but still
    might support ALUA. Rather than disable ALUA support we should be
    allowing the device identification to be empty and attach individual
    ALUA device handler to each devices.

[Stefan Bader (smb)]

So I can confirm that above patch cherry picked into 4.6 allows to boot and configure multipath devices again (as long as there is not that UNIX=m thing in which case there won't be any disks to multipath on).

[Matthew Shapiro (matthewsha)]

The same error appeared in 4.6.1 for amd64 . 4.6.0 works without problem.
Matthew

[Stefan Bader (smb)]

This should be fixed in Xenial/Yakkety by now. Unfortunately as part of the backport for bug #1567602 (fixed in 4.4.0-31.50 and later).