ASUS Z170-A, shim 15+1533136590.3beb971-0ubuntu1: grub menu not showing with both SecureBoot and CSM enabled after update

[frank]

Started with Ubuntu Version 14.04 I always had SecureBoot enabled and also CSM (compatibility support module) enabled in BIOS of Motherboard ASUS Z170-A, BIOS 3802 03/15/2018.
On October, 13th i did the updates that UpdateManager showed in the morning.

After rebooting in the evening, afterwards the BIOS splash screen there was only a black screen. No grub menu, no booting in SecureBoot mode.
I tried different repairs (tool boot-repair, reinstalling grub, installing signed kernel transitional packages and so on), but nothing would help.

At the end I found out out, that switching CSM to disabled "solved" the problem and it is now reproducible: Switching CSM on, no secure boot, switching off, secure boot works.

But I did not wont to switch CSM off.

I also tried to get some help at ubuntuforum. But no luck.

See here
https://ubuntuforums.org/showthread.php?t=2404150

Since today I think it's a bug in shim or grub, because at the morning of Oct, 13th these have been updated.

Regards

Frank

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: ubuntu-release-upgrader-core 1:18.04.27
ProcVersionSignature: Ubuntu 4.15.0-38.41-lowlatency 4.15.18
Uname: Linux 4.15.0-38-lowlatency x86_64
ApportVersion: 2.20.9-0ubuntu7.4
Architecture: amd64
CrashDB: ubuntu
CurrentDesktop: XFCE
Date: Wed Oct 24 20:36:17 2018
InstallationDate: Installed on 2016-04-03 (933 days ago)
InstallationMedia: Ubuntu-Studio 14.04.4 LTS "Trusty Tahr" - Release amd64 (20160217.1)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=de_DE
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
SourcePackage: ubuntu-release-upgrader
Symptom: release-upgrade
UpgradeStatus: Upgraded to bionic on 2018-09-24 (30 days ago)
VarLogDistupgradeTermlog:
---
ProblemType: Bug
ApportVersion: 2.20.9-0ubuntu7.4
Architecture: amd64
CurrentDesktop: XFCE
Dependencies:

DistroRelease: Ubuntu 18.04
InstallationDate: Installed on 2016-04-03 (935 days ago)
InstallationMedia: Ubuntu-Studio 14.04.4 LTS "Trusty Tahr" - Release amd64 (20160217.1)
Package: shim 15+1533136590.3beb971-0ubuntu1
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.15.0-38.41-lowlatency 4.15.18
Tags: bionic
Uname: Linux 4.15.0-38-lowlatency x86_64
UpgradeStatus: Upgraded to bionic on 2018-09-24 (32 days ago)
UserGroups: adm audio cdrom dialout dip lpadmin plugdev sambashare sudo vboxusers
_MarkForUpload: True
---
ProblemType: Bug
.proc.sys.kernel.moksbstate_disabled: Error: [Errno 2] Datei oder Verzeichnis nicht gefunden: '/proc/sys/kernel/moksbstate_disabled'
ApportVersion: 2.20.9-0ubuntu7.4
Architecture: amd64
CurrentDesktop: XFCE
DistroRelease: Ubuntu 18.04
EFIBootMgr:
 BootCurrent: 0001
 Timeout: 1 seconds
 BootOrder: 0001,0000
 Boot0000* Windows Boot Manager HD(2,GPT,ae4c669f-cdb2-4172-8148-b15621f6b33c,0xe1800,0x32000)/File(\EFI\MICROSOFT\BOOT\BOOTMGFW.EFI)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}....................
 Boot0001* ubuntu HD(2,GPT,ae4c669f-cdb2-4172-8148-b15621f6b33c,0xe1800,0x32000)/File(\EFI\UBUNTU\SHIMX64.EFI)
EFITables:
 Okt 31 22:22:51 Zuse2016 kernel: efi: EFI v2.50 by American Megatrends
 Okt 31 22:22:51 Zuse2016 kernel: efi: ESRT=0x8b1add98 ACPI=0x8a217000 ACPI 2.0=0x8a217000 SMBIOS=0x8b1ab000 SMBIOS 3.0=0x8b1aa000
 Okt 31 22:22:51 Zuse2016 kernel: secureboot: Secure boot enabled
 Okt 31 22:22:51 Zuse2016 kernel: esrt: Reserving ESRT space from 0x000000008b1add98 to 0x000000008b1addd0.
InstallationDate: Installed on 2016-04-03 (941 days ago)
InstallationMedia: Ubuntu-Studio 14.04.4 LTS "Trusty Tahr" - Release amd64 (20160217.1)
Package: shim-signed 1.37~18.04.2+15+1533136590.3beb971-0ubuntu1
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.15.0-38.41-lowlatency 4.15.18
SecureBoot: 6 0 0 0 1
Tags: bionic
Uname: Linux 4.15.0-38-lowlatency x86_64
UpgradeStatus: Upgraded to bionic on 2018-09-24 (37 days ago)
UserGroups: adm audio cdrom dialout dip lpadmin plugdev sambashare sudo vboxusers
_MarkForUpload: True

[Steve Langasek (vorlon)]

Please run apport-collect 1799767 on the affected system, so we can gather the information that is relevant to the shim package, rather than for ubuntu-release-upgrader.

[frank (franknfurter)]

apport information

[frank (franknfurter)]

apport information

[Steve Langasek (vorlon)]

apologies, it looks like we may only have the hook for the shim-signed source package. Could you please run apport-collect 1799767 once more?

[frank (franknfurter)]

Output of apport-collect now is
...
Package shim-signed not installed and no hook available, ignoring
Gtk-Message: 21:00:40.534: GtkDialog mapped without a transient parent. This is discouraged
...

And yes, it's true, at the moment it is not installed and the behaviour is as I described first.
But I'm sure, that the behaviour was the same when shim-signed had been installed some days ago. If you want, I can install it, try reeboot with CSM enabled and with CSM disabled and tell, what I have found out then.

[frank (franknfurter)]

apport information

[frank (franknfurter)]

apport information

[frank (franknfurter)]

apport information

[frank (franknfurter)]

apport information

[Steve Langasek (vorlon)]

Ok. Crucially, this shows that you have the 1.37~18.04.2+15+1533136590.3beb971-0ubuntu1 version of shim-signed installed, which was released on October 11 to bionic-updates; superseding version 1.34.9.2 which was there previously.

Why was the shim-signed package not installed? This is the package that \EFI\UBUNTU\SHIMX64.EFI is installed from, so for it to be absent suggests something was wrong with your upgrade process. Are you sure that the \EFI\UBUNTU\SHIMX64.EFI which was installed on your EFI System Partition at the time you encountered this problem was the one from shim-signed 1.37~18.04.2+15+1533136590.3beb971-0ubuntu1 ? (E.g.: if you disable CSM again now, does the problem reappear?)

[Steve Langasek (vorlon)]

Since your dist-upgrade logs also show that you had shim-signed 1.34.9.2 installed since September 23 and the problem only occurred for you after the upgrade on October 11, I'm provisionally marking this as a regression-update due to the apparent connection to shim-signed 1.37~18.04.2+1533136590.3beb971-0ubuntu1.

[frank (franknfurter)]

Dear Steve,

thanks for your reply. When you wanted me to apport-collect 1799767 once more, shim-signed wasn't installed because I had mad some experiments to solve the problem by myself. So I installed it again to do apport-collect 1799767 once more, but the effect is still the same.

And to clearify that: When I enable CSM in BIOS there is no secure boot but when I disable it, I have secure boot. And as a news: I tried the day before yesterday to set CSM to "auto" and SecureBoot is working with that configuration as well. Only setting to "CSM=enabled" the system doesn't boot in secure mode.

You asked if I am sure, that it was "shim-signed 1.37~18", but I can not tell, because normally I do not look at the details, when an update will be installed. I just install it when Update-manager means i should. I trust Ubuntu in that point.

I normally install the updates close to the time when they are published. Maybe some days later.

[Steve Langasek (vorlon)]

Could you please test the following:

- boot with CSM disabled
- run sudo mokutil --set-verbosity true
- reboot to the firmware and enable CSM
- boot Ubuntu
- report here any messages that appear on the console at boot time (take a picture and upload, or transcribe, etc)

After this, you can run 'sudo mokutil --set-verbosity false' to restore the previous behavior.

[Steve Langasek (vorlon)]

Advice from @cyphermox is to actually record the screen rather than taking a picture, in case the messages appear/disappear quickly.

[frank (franknfurter)]

Oh my god, that is really a strange behavin. As you said, I entered that mokutil command and rebooted the system, entered BIOS. I enabled CSM, saved the config and rebooted. In the attachment you see the message from shim after BIOS splash screen. But what happened then?
Suddenly I saw the grub menu and after 10 secs ubuntu starts automagically in secure boot mode. I didn't believe that. Thought I had made a mistake. So I switched off verbosity and rebooted. But, and that's surprising, I did not saw any grub menu, just the black screen as before.
So I had to boot into BIOS, set CSM to "auto" and reboot. And I did it again, just to verify this. I switched verbosity on, rebooted, enabled CSM, rebooted again and I have seen the grub menu. Ubuntu starts automagically in secure boot mode.

Conclusion:
CSM enabled and mokutil verbosity true-> grub menu and SecureBoot
CSM enabled and mokutil verbosity false-> Black screen

Further info: Yesterday I run an update that has been listed in update manager. Here you see the apt history:
Start-Date: 2018-11-02 21:20:21
Commandline: aptdaemon role='role-commit-packages' sender=':1.73'
Upgrade: libparted2:amd64 (3.2-20, 3.2-20ubuntu0.1), libparted-fs-resize0:amd64 (3.2-20, 3.2-20ubuntu0.1), parted:amd64 (3.2-20, 3.2-20ubuntu0.1), mokutil:amd64 (0.3.0-0ubuntu5, 0.3.0+1538710437.fb6250f-0ubuntu2~18.04.1), secureboot-db:amd64 (1.1, 1.4~ubuntu0.18.04.1)
End-Date: 2018-11-02 21:20:29

So what about updates at the moment. Is it ok for you if I still run these updates or should I wait with this. You know, I always want to install security patches.