Enroll key whiptail prompt blocks kernel header package upgrades

[林博仁(Buo-ren Lin)]

I noticed that sometimes the system upgrade is blocked in the kernel header package's configure phase, in the process manager I found that it is blocked by the "Configuring Secure Boot" whiptail prompt started by `update-secureboot-policy --enroll-key` which doesn't surface to the terminal and hence not interactable.

Refer the attached screenshot for the process status at the time.

[Test Case]
1. Install any out-of-tree dkms packages (like bbswitch-dkms)
2. Un-enroll keys from MOK, ensure `mokutil --test-key /var/lib/shim-signed/mok/MOK.der` command returns "/var/lib/shim-signed/mok/MOK.der is not enrolled"
3. Ensure SecureBoot is enabled
4. Install any linux-headers-${version}-generic packages that is currently not installed, verify if it stucks at configurating state

[Regression potential]
Since this change will cause the output of dkms_autoinstaller to appear in the output of the kernel installs, this may surprise consumers of the dpkg log with its verbosity.

[Steve Langasek (vorlon)]

This appears to be bug #1781001, which was fixed in dkms 2.3-3ubuntu9.2 for bionic. Have you not upgraded to the current version of the dkms package from bionic-updates?

[林博仁(Buo-ren Lin) (brlin)]

@vorlon: I do currently have that version installed, but the problem is still reproduced. The reproducing upgrade:

```
Start-Date: 2019-05-04 13:24:14
Commandline: apt full-upgrade
Requested-By: brlin (12345)
Install: linux-image-4.15.0-48-generic:amd64 (4.15.0-48.51, automatic), linux-headers-4.15.0-48:amd64 (4.15.0-48.51, automatic), linux-headers-4.15.0-48-generic:amd64 (4.15.0-48.51, automatic), linux-modules-extra-4.15.0-48-generic:amd64 (4.15.0-48.51, automatic), linux-modules-4.15.0-48-generic:amd64 (4.15.0-48.51, automatic)
Upgrade: libgcc-7-dev:amd64 (7.3.0-27ubuntu1~18.04, 7.4.0-1ubuntu1~18.04), libmpx2:amd64 (8.2.0-1ubuntu2~18.04, 8.3.0-6ubuntu1~18.04), update-manager-core:amd64 (1:18.04.11.9, 1:18.04.11.10), linux-headers-generic:amd64 (4.15.0.47.49, 4.15.0.48.50), ureadahead:amd64 (0.100.0-20, 0.100.0-21), libdrm-nouveau2:amd64 (2.4.97+git1903201229.852a9d2~b~padoka0, 2.4.98+git1904260050.6a7d132~b~padoka0), libdrm-nouveau2:i386 (2.4.97+git1903201229.852a9d2~b~padoka0, 2.4.98+git1904260050.6a7d132~b~padoka0), gir1.2-gtk-3.0:amd64 (3.22.30-1ubuntu2, 3.22.30-1ubuntu3), linux-libc-dev:amd64 (4.15.0-47.50, 4.15.0-48.51), steam-launcher:amd64 (1.0.0.59, 1.0.0.61), libldap-2.4-2:amd64 (2.4.45+dfsg-1ubuntu1.1, 2.4.45+dfsg-1ubuntu1.2), libldap-2.4-2:i386 (2.4.45+dfsg-1ubuntu1.1, 2.4.45+dfsg-1ubuntu1.2), libllvm9:amd64 (1:9~svn357783-0~b~padoka0, 1:9~svn359117-0~b~padoka0), libllvm9:i386 (1:9~svn357783-0~b~padoka0, 1:9~svn359117-0~b~padoka0), libegl-mesa0:amd64 (1:19.1~git190409002900.50f3535~b~padoka0, 1:19.1~git190501171900.70da00f~b~padoka0), gnome-software-plugin-snap:amd64 (3.28.1-0ubuntu4.18.04.8, 3.28.1-0ubuntu4.18.04.9), libsystemd0:amd64 (237-3ubuntu10.19, 237-3ubuntu10.21), libsystemd0:i386 (237-3ubuntu10.19, 237-3ubuntu10.21), libgtk-3-common:amd64 (3.22.30-1ubuntu2, 3.22.30-1ubuntu3), linux-image-generic:amd64 (4.15.0.47.49, 4.15.0.48.50), libgtk-3-0:amd64 (3.22.30-1ubuntu2, 3.22.30-1ubuntu3), libobjc4:amd64 (8.2.0-1ubuntu2~18.04, 8.3.0-6ubuntu1~18.04), cpp-7:amd64 (7.3.0-27ubuntu1~18.04, 7.4.0-1ubuntu1~18.04), libglapi-mesa:amd64 (1:19.1~git190409002900.50f3535~b~padoka0, 1:19.1~git190501171900.70da00f~b~padoka0), libglapi-mesa:i386 (1:19.1~git190409002900.50f3535~b~padoka0, 1:19.1~git190501171900.70da00f~b~padoka0), gcc-8-base:amd64 (8.2.0-1ubuntu2~18.04, 8.3.0-6ubuntu1~18.04), gcc-8-base:i386 (8.2.0-1ubuntu2~18.04, 8.3.0-6ubuntu1~18.04), flatpak:amd64 (1.3.1-flatpak1~bionic, 1.3.3-1flatpak1~bionic), binutils:amd64 (2.30-21ubuntu1~18.04, 2.30-21ubuntu1~18.04.1), cpp:amd64 (4:7.3.0-3ubuntu2.1, 4:7.4.0-1ubuntu2.2), language-selector-common:amd64 (0.188.1, 0.188.2), linux-signed-image-generic:amd64 (4.15.0.47.49, 4.15.0.48.50), libitm1:amd64 (8.2.0-1ubuntu2~18.04, 8.3.0-6ubuntu1~18.04), libflatpak0:amd64 (1.3.1-flatpak1~bionic, 1.3.3-1flatpak1~bionic), gnome-software:amd64 (3.28.1-0ubuntu4.18.04.8, 3.28.1-0ubuntu4.18.04.9), g++:amd64 (4:7.3.0-3ubuntu2.1, 4:7.4.0-1ubuntu2.2), gtk-3-examples:amd64 (3.22.30-1ubuntu2, 3.22.30-1ubuntu3), google-chrome-stable:amd64 (73.0.3683.103-1, 74.0.3729.131-1), ubuntu-standard:amd64 (1.417, 1.417.1), libopenconnect5:amd64 (7.08-3, 7.08-3ubuntu0.18.04.1), console-setup-linux:amd64 (1.178ubunt...

[Steve Langasek (vorlon)]

When this happens, it would be good to get the output of lsof for the debconf frontend process as well as the whiptail process, to check where stdin and stdout are connected.

[Steve Langasek (vorlon)]

ok based on the process tree I've identified the source of the trouble, /etc/kernel/header_postinst.d/dkms invokes dkms_autoinstaller with >/dev/null. So this is a similar symptom to other bugs that have been fixed, but is a separate unfixed bug.

Note that a workaround for this problem is to run 'sudo update-secureboot-policy --enroll-key' outside of apt.

[Julian Andres Klode (juliank)]

xenial I don't know what to do: It's all fine importing the patch into the package, but as soon as I add the changelog entry to debian/changelog, building fails with

dpkg-source: info: using patch list from debian/patches/series
patching file dkms_common.postinst
Hunk #1 FAILED at 146.
1 out of 1 hunk FAILED
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch '666023.patch' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/666023.patch/ --reject-file=- < dkms.orig.axFxWt/debian/patches/666023.patch subprocess returned exit status 1

[Julian Andres Klode (juliank)]

Now it's fine, maybe changelog entry was broken and dpkg's error handling was crazy.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dkms - 2.6.1-4ubuntu4

---------------
dkms (2.6.1-4ubuntu4) eoan; urgency=medium

  * Do not invoke dkms_autoinstaller from /etc/kernel/header_postinst.d/dkms
    with redirection to /dev/null (LP: #1827697), this caused debconf dialog
    to not be shown.

 -- Julian Andres Klode <email address hidden> Fri, 17 May 2019 12:56:47 +0200

[Timo Aaltonen (tjaalton)]

Hello 林博仁(Buo-ren, or anyone else affected,

Accepted dkms into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dkms/2.6.1-4ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello 林博仁(Buo-ren, or anyone else affected,

Accepted dkms into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dkms/2.3-3ubuntu11.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello 林博仁(Buo-ren, or anyone else affected,

Accepted dkms into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dkms/2.3-3ubuntu9.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello 林博仁(Buo-ren, or anyone else affected,

Accepted dkms into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dkms/2.2.0.3-2ubuntu11.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Julian Andres Klode (juliank)]

The regressions in rdep autopkgtests are not caused by this upload, but also present in other runs:

xenial:

Regression in autopkgtest for dahdi-linux (s390x)
Regression in autopkgtest for iscsitarget (armhf)

Still can't actually verify the bug, as I can't trigger it.

[林博仁(Buo-ren Lin) (brlin)]

I have applied the updates on bionic and can no longer reproduce it.

[林博仁(Buo-ren Lin) (brlin)]

@juliank

To trigger the bug:

1. Install any out-of-tree dkms packages (like bbswitch-dkms)
2. Un-enroll keys from MOK, ensure `mokutil --test-key /var/lib/shim-signed/mok/MOK.der` command returns "/var/lib/shim-signed/mok/MOK.der is not enrolled"
3. Ensure SecureBoot is enabled
4. Install any linux-headers-${version}-generic packages that is currently not installed, verify if it stucks at configurating state

[Julian Andres Klode (juliank)]

You have not stated the versions tested in bionic, hence the verification is not valid. Can you check that and state which versions you tested?

One test run needs about 30 mins to an hour (install ubuntu in a VM, then try to reproduce), so if I can avoid one, that'd be great :)

[Julian Andres Klode (juliank)]

disco: I reproduced the bug by installing linux headers -17-generic which hang and then upgrading dkms to 2.6.1-4ubuntu2.1 and installing linux-headers -16-generic (purging and reinstalling the 17 headers did nothing it seems, so, don't care ..) which had a working prompt.

To "revoke" the key, run mokutil --revoke-import after a fresh setup and install of the module, before installing the header package.

[Julian Andres Klode (juliank)]

cosmic: Installed kernel header, got the hang; Updated dkms to -ubuntu11.1 from -ubuntu11; installed different kernel header, got prompt. Verifed.

[林博仁(Buo-ren Lin) (brlin)]

@juliank
The reproduced dkms version on bionic is 2.3-3ubuntu9.2, while the verified version is 2.3-3ubuntu9.3.

[Julian Andres Klode (juliank)]

@buo-ren-lin thanks!

xenial: I verified that ubuntu11.6 was broken, and ubuntu11.7 is fine in the same way as the others.

[Julian Andres Klode (juliank)]

xenial:

> Regression in autopkgtest for dahdi-linux (s390x): test log
> Regression in autopkgtest for iscsitarget (armhf): test log

I looked at those and they seem to be failing a lot for other uploads too, so do not seem to be regressions.

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for dkms has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dkms - 2.3-3ubuntu9.3

---------------
dkms (2.3-3ubuntu9.3) bionic; urgency=medium

  * Do not invoke dkms_autoinstaller from /etc/kernel/header_postinst.d/dkms
    with redirection to /dev/null (LP: #1827697), this caused debconf dialog
    to not be shown.

 -- Julian Andres Klode <email address hidden> Fri, 17 May 2019 12:56:47 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package dkms - 2.3-3ubuntu11.1

---------------
dkms (2.3-3ubuntu11.1) cosmic; urgency=medium

  * Do not invoke dkms_autoinstaller from /etc/kernel/header_postinst.d/dkms
    with redirection to /dev/null (LP: #1827697), this caused debconf dialog
    to not be shown.

 -- Julian Andres Klode <email address hidden> Fri, 17 May 2019 12:56:47 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package dkms - 2.6.1-4ubuntu2.1

---------------
dkms (2.6.1-4ubuntu2.1) disco; urgency=medium

  * Do not invoke dkms_autoinstaller from /etc/kernel/header_postinst.d/dkms
    with redirection to /dev/null (LP: #1827697), this caused debconf dialog
    to not be shown.

 -- Julian Andres Klode <email address hidden> Fri, 17 May 2019 12:56:47 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package dkms - 2.2.0.3-2ubuntu11.7

---------------
dkms (2.2.0.3-2ubuntu11.7) xenial; urgency=medium

  * Do not invoke dkms_autoinstaller from /etc/kernel/header_postinst.d/dkms
    with redirection to /dev/null (LP: #1827697), this caused debconf dialog
    to not be shown.

 -- Julian Andres Klode <email address hidden> Fri, 17 May 2019 13:21:05 +0200

[R T (roadrunner2)]

The fix here breaks installing custom kernels, at least on xenial (dkms (2.2.0.3-2ubuntu11.7) xenial): the redirection of stdout was to there to fix a problem with dkms output contaminating debconf's input - see #292606 for details. To be specific, during package install (either via dpkg or apt-get) I'm now getting the following error from /usr/sbin/update-grub-legacy-ec2 which is run from /etc/kernel/postinst.d/x-grub-legacy-ec2 (after running /etc/kernel/postinst.d/dkms):

20 Unsupported command "*" (full line was " * dkms: running auto installation service for kernel ...") received from confmodule.