hw-detect modprobes blindly, ignoring blacklists

Bug #1602717 reported by bugproxy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
hw-detect (Ubuntu)
Fix Released
Low
Mathieu Trudel-Lapierre
Xenial
Fix Released
Low
Mathieu Trudel-Lapierre

Bug Description

[Impact]
Any system requiring specific modules to be blacklisted at install-time, when using server or alternate images.

[Test cases]
Boot the installer w/ this boot/kernel cmdline option (the alias w/ dashes is not required, but is here just in case):
    modprobe.blacklist=scsi_dh_alua,scsi-dh-alua

On the dialog for username/password, select Go Back, then Exit to shell:

The blacklist is detected/present in the config:

~ # modprobe -c | grep alua
blacklist scsi_dh_alua
blacklist scsi-dh-alua

But the module is insmod-ed anyway:

~ # modprobe -v -n scsi-dh-alua
insmod /lib/modules/4.4.0-21-generic/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko

Unless -b is used:

~ # modprobe -v -n -b scsi-dh-alua
~ #

[Regression Potential]
Failure to load kernel modules at boot time due to unexpectedly blacklisted modules would consistute a regression for this bug report.

== Comment: #0 - Mauricio Faria De Oliveira - 2016-07-11 18:47:59 ==
---Problem Description---
modprobe does not honor blacklist options in kernel cmdline unless -b is used (at least on the installer)

The modprobe.blacklist option is required as a work-around until a solution to a device-driver bug is found (IBM LTC bug 138273).

I've gone through the code of the kmod src pkg, but didn't spot anything obvious.
Passing along to the Ubuntu team.

Contact Information = Mauricio Faria de Oliveira <email address hidden> / Lekshmi C. Pillai <email address hidden>

---Steps to Reproduce---
 Boot the installer w/ this boot/kernel cmdline option (the alias w/ dashes is not required, but is here just in case):
    modprobe.blacklist=scsi_dh_alua,scsi-dh-alua

On the dialog for username/password, select Go Back, then Exit to shell:

The blacklist is detected/present in the config:

~ # modprobe -c | grep alua
blacklist scsi_dh_alua
blacklist scsi-dh-alua

But the module is insmod-ed anyway:

~ # modprobe -v -n scsi-dh-alua
insmod /lib/modules/4.4.0-21-generic/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko

Unless -b is used:

~ # modprobe -v -n -b scsi-dh-alua
~ #

Installer kernel:
[ 0.000000] Linux version 4.4.0-21-generic (buildd@bos01-ppc64el-017) (gcc version 5.3.1 20160413 (Ubuntu/IBM 5.3.1-14ubuntu2) ) #37-Ubuntu SMP Mon Apr 18 18:30:22 UTC 2016 (Ubuntu 4.4.0-21.37-generic 4.4.6)

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-143679 severity-high targetmilestone-inin16041
Changed in ubuntu:
assignee: nobody → Taco Screen team (taco-screen-team)
affects: ubuntu → linux (Ubuntu)
Revision history for this message
Tim Gardner (timg-tpi) wrote :

According to the man page modprobe is functioning as designed.

Changed in linux (Ubuntu):
status: New → Invalid
Revision history for this message
Kevin W. Rudd (kevinr) wrote :

Sorry. The original problem description and mirrored package may be a bit misleading. I believe that real issue is that the installer is *not* using "-b", so any blacklist options passed via the kernel cmdline are not being honored. Tagging the package as "debian-installer" to focus on the install-time issue of needing to be able to blacklist modules during the install. Passing blacklist options are reported to work in 16.04 Pre-GA, so it appears that something changed (either in the installer, or in modprobe handling). But, the bottom line problem we need to have solved is getting the installer to honor any blacklisting passed on the cmdline.

affects: linux (Ubuntu) → debian-installer (Ubuntu)
Changed in debian-installer (Ubuntu):
status: Invalid → New
Revision history for this message
Adam Conrad (adconrad) wrote :

A syslog from the failed installation might make it easier to hunt down where the problem might be.

Revision history for this message
Adam Conrad (adconrad) wrote :

Ahh, nevermind, I found the code in question in hw-detect. Looks like it's never respected the blacklist.

Revision history for this message
Adam Conrad (adconrad) wrote :

I assume this bug becomes less critical to you if you use the latest d-i images (20101020ubuntu451.3) I just published to xenial-updates (should hit mirrors in the next hour), since they should include the fixed kernel?

affects: debian-installer (Ubuntu) → hw-detect (Ubuntu)
summary: - modprobe does not honor blacklist options in kernel cmdline unless -b is
- used (at least on the installer)
+ hw-detect modprobes blindly, ignoring blacklists
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-07-19 18:26 EDT-------
@adconrad

> I assume this bug becomes less critical to you if you use the latest d-i
> images (20101020ubuntu451.3) I just published to xenial-updates (should hit
> mirrors in the next hour), since they should include the fixed kernel?

Can you clarify what fix is in the kernel?
I expected a fix for hw-detect actually :) (which for xenial is still in -release version)

thanks

Revision history for this message
Adam Conrad (adconrad) wrote :

There were huge updates to the scsi_dh_alua driver in the 4.4.0-30 kernel. I can't see the LTC bug you referenced, so I can't tell if said updates address your issue, but I was hopeful. :P

bugproxy (bugproxy)
affects: hw-detect (Ubuntu) → linux (Ubuntu)
Steve Langasek (vorlon)
affects: linux (Ubuntu) → hw-detect (Ubuntu)
bugproxy (bugproxy)
affects: hw-detect (Ubuntu) → linux (Ubuntu)
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-07-21 12:08 EDT-------
@adconrad,

> There were huge updates to the scsi_dh_alua driver in the 4.4.0-30 kernel.
> I can't see the LTC bug you referenced, so I can't tell if said updates
> address your issue, but I was hopeful. :P

Ok, I see. I'll ask if that helps w/ the other bug.

Nevertheless, having modprobe.blacklist working by default on the installer would be interesting.

I passed around a sed line to use modprobe -b on hw-detect, a simple workaround, but that definitely isn't convenient (neither for preseed). And since I don't know much of modprobe and its usage on the installer, I decided to reach out for you guys first.

Now that is seems to be clear, can get hw-detect be updated to use modprobe -b ?

Thanks!

Revision history for this message
Adam Conrad (adconrad) wrote :

Indeed, this is still a valid hw-detect bug/wishlist, I was just hoping the driver updates would get you over your current hump, so please do check that.

affects: linux (Ubuntu) → hw-detect (Ubuntu)
bugproxy (bugproxy)
affects: hw-detect (Ubuntu) → linux (Ubuntu)
Adam Conrad (adconrad)
affects: linux (Ubuntu) → hw-detect (Ubuntu)
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-07-25 10:24 EDT-------
@adconrad

Sure; I had asked for testing w/ a newer build, but still haven't heard any news.

bugproxy (bugproxy)
tags: added: targetmilestone-inin16042
removed: targetmilestone-inin16041
Revision history for this message
bugproxy (bugproxy) wrote : hw-detect_modprobe-b.debdiff

------- Comment on attachment From <email address hidden> 2016-08-24 12:11 EDT-------

@adconrad

It's taking too long to get a reply back from the test team, so it seems the ALUA fixes did it for them. :-P

Either way, the behavior to honor module blacklists specified via kernel cmdline is desired.
Patch attached. Would you mind to review/consider it for 16.04.x?

Thanks

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

@taco-screen-team

Can this bug be assigned to @adconrad (commented earlier on this bug) or @cyphermox (previous installer fixes in this area)?

Patch attached.

Thanks,

Steve Langasek (vorlon)
Changed in hw-detect (Ubuntu):
assignee: Taco Screen team (taco-screen-team) → Mathieu Trudel-Lapierre (cyphermox)
status: New → Triaged
importance: Undecided → Low
Changed in hw-detect (Ubuntu):
status: Triaged → In Progress
Changed in hw-detect (Ubuntu Xenial):
status: New → In Progress
importance: Undecided → Low
assignee: nobody → Mathieu Trudel-Lapierre (cyphermox)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted hw-detect into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/hw-detect/1.117ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in hw-detect (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-09-29 19:31 EDT-------
This has been verified successfully w/ the nvme driver.
Updating tags.

On the partitioner dialog, exited to shell and verified (below).

1) Enabling -proposed & blacklist options, the nvme module is not loaded and nvme devices are not present:

~ # cat /proc/cmdline
method=http://ports.ubuntu.com/ubuntu-ports/dists/xenial-updates/main/installer-ppc64el/ apt-setup/proposed=true modprobe.blacklist=nvme

~ # lsmod | grep nvme
~ #

~ # ls -1 /dev/nvme*
ls: /dev/nvme*: No such file or directory

2) Enabling -proposed but not the blacklist option, the nvme module is loaded and nvme devices are present:

~ # cat /proc/cmdline
method=http://ports.ubuntu.com/ubuntu-ports/dists/xenial-updates/main/installer-ppc64el/ apt-setup/proposed=true

~ # lsmod | grep nvme
nvme 82444 0

~ # ls -1 /dev/nvme*
/dev/nvme0
/dev/nvme0n1
/dev/nvme0n1p1

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package hw-detect - 1.117ubuntu3

---------------
hw-detect (1.117ubuntu3) yakkety; urgency=medium

  * {hw-,disk-,net}detect.sh, check-missing-firmware.sh: use 'modprobe -b'
    in order to honor module blacklisting from kernel cmdline parameters.
    (LP: #1602717)

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 29 Sep 2016 09:52:10 -0400

Changed in hw-detect (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package hw-detect - 1.117ubuntu2.1

---------------
hw-detect (1.117ubuntu2.1) xenial; urgency=medium

  * {hw-,disk-,net}detect.sh, check-missing-firmware.sh: use 'modprobe -b'
    in order to honor module blacklisting from kernel cmdline parameters.
    (LP: #1602717)

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 29 Sep 2016 09:52:10 -0400

Changed in hw-detect (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Chris J Arges (arges) wrote : Update Released

The verification of the Stable Release Update for hw-detect has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.