UEFI secure boot fails after 14.04 to 16.04 upgrade

[Stefan Bader]

I did a release upgrade from fully upgraded Trusty/14.04.x to Xenial/16.04 today (amd64). There was no indication of any problems during the upgrade. Only oddly asking to disable secure boot on the shim level again (already had done this on Trusty). Also I had the proposed pocket enabled in Trusty before doing the upgrade (update-manager).
After reboot I get a textual error message that "image verification has failed" and I am presented with a menu to select a different UEFI element (this is a Lenovo x230).
I can disable secure boot in the BIOS and am then able to boot.
Not sure this is related to the issue but from the system booted without secure boot I tried to run sbverify and it returns the same error for all EFI binaries I tried:

# sbverify shimx64.efi
warning: data remaining[1170360 vs 1289424]: gaps between PE/COFF sections?
PKCS7 verification failed
140313718134424:error:21075075:PKCS7 routines:PKCS7_verify:certificate verify error:pk7_smime.c:336:Verify error:unable to get local issuer certificate
Signature verification failed

If there is any other info that is needed, let me know. Or/and if there are any steps to resolve the issue, let me know, too.

[Stefan Bader (smb)]

Oh, one thing to add: looking at the versions of shim and shim-signed, those were beyond the versions for Xenial, rather Yakkety. Maybe because of proposed being enabled before (it is turned off now after the upgrade. I manually downgraded both now to Xenial(updates), still sbverify fails with the same error.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1652147/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Stefan Bader (smb)]

Looks like the sbverify command failing is not relevant. Retrying to boot with secure boot enabled in BIOS _after_ I downgraded shim/shim-signed to the versions that are in the archive for Xenial(updates pocket) was successful and brought up mok manager once, allowing to again disable secure boot at that stage (since I still need to use some DKMS packages and want to try test kernels).
So the main problem were those incorrect versions:

shim 0.9+1474479173.6c180c6-0ubuntu1 -> 0.8-0ubuntu2
shim-signed 1.21.4~14.04.1+0.9+1474479173.6c180c6-0ubuntu1 -> 1.19~16.04.1+0.8-0ubuntu2

[Steve Langasek (vorlon)]

dist-upgrading with -proposed enabled gives unsupportable results as it pulls in all packages that have not yet completed SRU verification. See also https://lists.ubuntu.com/archives/ubuntu-release/2016-October/003950.html ff.

[Stefan Bader (smb)]

Would it not be really nice if update-manager would then disable proposed before that? Also I just did another upgrade and had disabled proposed *before*. Still I end up with the same non-xenial versions of shim and shim-signed.

[Brian Murray (brian-murray)]

ubuntu-release-upgrade should disable -proposed to prevent situations like this. Please include your the file /var/log/dist-upgrade/main.log which should contain information about your upgrade from 14.04 to 16.04.

DistUpgradeController.py contains the following code:

 653 # Disable proposed on upgrade to a development release.
 654 if (not entry.disabled and self.options
 655 and self.options.devel_release == True and
 656 "%s-proposed" % self.fromDist in entry.dist):
 657 logging.debug("upgrade to development release, disabling proposed")
 658 entry.dist = "%s-proposed" % self.toDist
 659 entry.comment += _("Not for humans during development stage of release %s") % self.toDist
 660 entry.disabled = True
 661 continue

So you'd also see the above comment in /etc/apt/sources.list.

[Steve Langasek (vorlon)]

On Tue, Jan 03, 2017 at 05:19:24PM -0000, Brian Murray wrote:
> ubuntu-release-upgrade should disable -proposed to prevent situations
> like this. Please include your the file /var/log/dist-upgrade/main.log
> which should contain information about your upgrade from 14.04 to 16.04.
>
> DistUpgradeController.py contains the following code:
>
> 653 # Disable proposed on upgrade to a development release.
> 654 if (not entry.disabled and self.options
> 655 and self.options.devel_release == True and
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So we in fact only disable it on upgrades to the devel release, not on
upgrade to a new stable release.

> 656 "%s-proposed" % self.fromDist in entry.dist):
> 657 logging.debug("upgrade to development release, disabling proposed")
> 658 entry.dist = "%s-proposed" % self.toDist
> 659 entry.comment += _("Not for humans during development stage of release %s") % self.toDist
> 660 entry.disabled = True
> 661 continue
>
> So you'd also see the above comment in /etc/apt/sources.list.
>
> ** Changed in: shim (Ubuntu)
> Importance: Undecided => High
>
> --
> You received this bug notification because you are subscribed to shim in
> Ubuntu.
> https://bugs.launchpad.net/bugs/1652147
>
> Title:
> UEFI secure boot fails after 14.04 to 16.04 upgrade
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1652147/+subscriptions

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Stefan Bader (smb)]

In sources.list on the laptop proposed is disabled now. I think I did not manually change this. So it seems that it got disabled during upgrade. Which would make sense with my second upgrade where I disabled proposed manually before the release upgrade. The weird thing is anyway where those versions come from. Using rmadison they do not show and I thought they should.

/etc/apt/sources.list:
...
# deb http://de.archive.ubuntu.com/ubuntu/ xenial-proposed universe main restricted multiverse #Not for humans during development stage of release xenial

#> rmadison shim-signed | grep xenial
shim-signed | 1.12 | xenial | source
shim-signed | 1.12+0.8-0ubuntu2 | xenial | amd64
shim-signed | 1.19~16.04.1 | xenial-updates | source
shim-signed | 1.19~16.04.1+0.8-0ubuntu2 | xenial-updates | amd64

[Steve Langasek (vorlon)]

On Thu, Jan 05, 2017 at 09:03:04AM -0000, Stefan Bader wrote:
> In sources.list on the laptop proposed is disabled now. I think I did
> not manually change this. So it seems that it got disabled during
> upgrade.

Check timestamps?

> Which would make sense with my second upgrade where I disabled proposed
> manually before the release upgrade. The weird thing is anyway where
> those versions come from.

There was a withdrawn SRU of shim-signed.

[Stefan Bader (smb)]

Timestamps are of no use since I did other modifications.

Ok, I think I now know what happened. I had proposed enabled in Trusty (in my case to have those act as canaries for updates). So I got those new versions of shim/shim-signed back then. And together with grub2 (or maybe kernel) this was somehow working in Trusty. Then those updates got removed from the archive but not replaced by newer versions. So the release-upgrade actually did *not* update those two packages. And now in the Xenial environment they actually break boot completely.

The problem I can see is that many people have proposed enabled at some point when they are asked to verify bugs. And IIRC the instructions do not tell them to turn off proposed after that. So this might happen to more people we think it could.

The work-around for me: disable secure boot in bios, boot, downgrade shim/shim-signed, reboot, enable secure boot in bios again.

[Andy Whitcroft (apw)]

Confirming the version installed was the withdrawn shim SRU from trusty, still installed in the xenial system.