NVMe boot drives not supported - failing in generating initramfs

Bug #1771557 reported by Guilherme G. Piccoli on 2018-05-16
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
initramfs-tools
Fix Released
Unknown
initramfs-tools (Ubuntu)
Medium
Guilherme G. Piccoli
Trusty
High
Guilherme G. Piccoli
Xenial
Medium
Unassigned

Bug Description

[Impact]

When creating the initramfs image, mkinitramfs has multiple options on how to include modules. The default (and most common) one is the "MODULES=most", which includes the majority of filesystem modules and all the block device drivers. One other option is "MODULES=dep", which tries to descend in the sysfs hierarchy and guess modules to add, with the goal of reduce the size of initramfs.

For the MODULES=dep case, the initramfs-tools hook-functions script cannot translate nvmeXnYpZ to nvmeXnY block device, so it's failing in the sysfs lookup, so it does not build the initram disk.

Upstream solution is composed of at least 2 patches (it's a series, but the 2 below are really the needed ones):

commit 3cb744c9
Author: Ben Hutchings <email address hidden>
hook-functions: Rewrite block device sysfs lookup to be generic

commit 8ac52dc0
Author: Ben Hutchings <email address hidden>
hook-functions: Include modules for all components of a multi-disk device

Instead of doing the huge backport, we added another sed substitution: currently the script has substitutions for sdX and hdX, in order to convert sda1 to sda, for example. The new substitution converts nvmeXnYpZ to nvmeXnY.
It's less intrusive than the full backport, since this is a minimal SRU to Trusty only.

[Test Case]
1. Install Trusty with rootfs in a multi-disk(md) array composed of two nvme partitions - in my tests, I've used a RAID1.

(lsblk output of my test env:

nvme0n1 259:0 0 10G 0 disk
└─nvme0n1p1 259:1 0 10G 0 part
  └─md0 9:0 0 10G 0 raid1 /
nvme1n1 259:2 0 10G 0 disk
└─nvme1n1p1 259:3 0 10G 0 part
  └─md0 9:0 0 10G 0 raid1 /
)

2. Once system is booted, modify the "/etc/initramfs-tools/initramfs.conf", replacing "MODULES=most" to "MODULES=dep".

3. Update your initramfs by running something like:
"update-initramfs -u -k <your kernel version>"

The initramfs creating procedure will fail, unless the patch from this LP is present.

[Regression Potential]
If the sed expression was somewhat broken, we could have an issue generating initiramfs when MODULES is set to "dep", even for generic block devices (like regular HDDs).

[Other Info]
* This issue is based on Debian bug #785147:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785147

* Only Trusty is affected, Xenial and late contains the full fix as mentioned above :

$ git describe --contains 8ac52dc0
v0.121_rc1~11

$ git describe --contains 3cb744c9
v0.121_rc1~11

$ rmadison initramfs-tools

 initramfs-tools | 0.103ubuntu4 | trusty | source, all
 ==> initramfs-tools | 0.103ubuntu4.10 | trusty-updates | source, all
 initramfs-tools | 0.122ubuntu8 | xenial | source, all
 initramfs-tools | 0.122ubuntu8.11 | xenial-updates | source, all
 initramfs-tools | 0.125ubuntu12 | artful | source, all
 initramfs-tools | 0.125ubuntu12.1 | artful-updates | source, all
 initramfs-tools | 0.130ubuntu3 | bionic | source, all
 initramfs-tools | 0.130ubuntu6 | cosmic | source, all

Guilherme G. Piccoli (gpiccoli) wrote :

Fixed in Debian

Changed in linux (Ubuntu):
importance: Undecided → High
milestone: none → trusty-updates
status: New → In Progress
affects: linux → initramfs-tools
Changed in initramfs-tools:
status: Unknown → Fix Released
affects: linux (Ubuntu) → initramfs-tools (Ubuntu)
Changed in initramfs-tools (Ubuntu Xenial):
importance: Undecided → Medium
status: New → Fix Released
Changed in initramfs-tools (Ubuntu Trusty):
assignee: nobody → Guilherme G. Piccoli (gpiccoli)
importance: Undecided → High
status: New → In Progress
milestone: none → trusty-updates
Changed in initramfs-tools (Ubuntu):
status: In Progress → Fix Released
importance: High → Medium
milestone: trusty-updates → none
description: updated
Guilherme G. Piccoli (gpiccoli) wrote :

This solution was suggested by Szilard Cserey and further improved by Dan Streetman - thanks both!

Eric Desrochers (slashd) on 2018-05-16
description: updated
Eric Desrochers (slashd) wrote :

Uploaded in Trusty upload queue. It is now waiting on SRU verification team approval to start building in trusty-proposed.

# Sponsoring note

[Nitpick]
- Add (LP: #1771557) in debian/changelog

[Note]
* The sed command seems pretty solid. The "*" is indeed a good idea, even if IMHO unlikely to exceed 2 digits if it happen the cmd will match any number that precedes "n", "p" and the end, even if exceeding 2 digits.
* I don't have an NVMe drive handy, but I know gpicolli did all the necessary validation pre-SRU.

[Manual verification]

Testing different nvme drive naming scenarios:

$ echo "nvme1n2p3" | sed 's/\(nvme[0-9][0-9]*n[0-9][0-9]*\)p[0-9]*/\1/g'
nvme1n2

$ echo "nvme11n22p33" | sed 's/\(nvme[0-9][0-9]*n[0-9][0-9]*\)p[0-9]*/\1/g'
nvme11n22

$ echo "nvme11n22p33" | sed 's/\(nvme[0-9][0-9]*n[0-9][0-9]*\)p[0-9]*/\1/g'
nvme11n22

$ echo "nvme111n222p333" | sed 's/\(nvme[0-9][0-9]*n[0-9][0-9]*\)p[0-9]*/\1/g'
nvme111n222

$ echo "nvme1111n2222p3333" | sed 's/\(nvme[0-9][0-9]*n[0-9][0-9]*\)p[0-9]*/\1/g'
nvme1111n2222

Hello Guilherme, or anyone else affected,

Accepted initramfs-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/initramfs-tools/0.103ubuntu4.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in initramfs-tools (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-trusty
Guilherme G. Piccoli (gpiccoli) wrote :

Thanks a lot Eric and Brian for handling the SRU process.
I just tested the packages initramfs-tools and initramfs-tools-bin version 0.103ubuntu4.11, from trusty-proposed, following the test procedure of this LP's description.

Everything is working fine, so I'll mark this as verification-done.

Cheers,

Guilherme

tags: added: verification-done verification-done-trusty
removed: verification-needed verification-needed-trusty

The verification of the Stable Release Update for initramfs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package initramfs-tools - 0.103ubuntu4.11

---------------
initramfs-tools (0.103ubuntu4.11) trusty; urgency=medium

  * hook-functions: handle md arrays with nvme disk members. (LP: #1771557)

 -- <email address hidden> (Guilherme G. Piccoli) Wed, 16 May 2018 09:47:01 -0300

Changed in initramfs-tools (Ubuntu Trusty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.