IPL on z15 always performed regardless of the secure-boot related settings

[bugproxy]

Description will follow

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-01-22 05:29 EDT-------
addl. description:
"Operating system messages" output

Test on a z15 LPAR: Checking the combinations of the /etc/zipl.conf "secure" keyword and the HMC "Enable Secure Boot for Linux" option on the HMC SCSI load panel.

Result: the system always performs a successful IPL regardless of the
settings of the zipl.conf "secure" keyword and and the HMC "Enable Secure Boot
for Linux" option.

Problem: No IPL should be performed for the combination of "secure=0" in /etc/zipl.conf and the selection of "Enable Secure Boot for Linux" option in the HMC SCSI load panel.

Scenario
--------

root@t35lp36:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu Focal Fossa (development branch)"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@t35lp36:~# uname -a
Linux t35lp36 5.4.0-9-generic #12-Ubuntu SMP Mon Dec 16 22:31:38 UTC 2019 s390x s390x s390x GNU/Linux

Setting secure=0 in /etc/zipl.conf

root@t35lp36:~# cat /etc/zipl.conf
[defaultboot]
defaultmenu = menu
secure=0

:menu
target = /boot
1 = ubuntu
2 = old
default = 1
prompt = 1
timeout = 10

[ubuntu]
target = /boot
image = /boot/vmlinuz
ramdisk = /boot/initrd.img
parameters = root=UUID=dc6b7633-49f0-4095-8c35-678cbc212ca5 crashkernel=196M

[old]
target = /boot
image = /boot/vmlinuz.old
ramdisk = /boot/initrd.img.old
parameters = root=UUID=dc6b7633-49f0-4095-8c35-678cbc212ca5 crashkernel=196M
optional = 1

root@t35lp36:~# zipl -V
Using config file '/etc/zipl.conf'
Run /lib/s390-tools/zipl_helper.device-mapper /boot
Target device information
Device..........................: fd:00 *)
Partition.......................: fd:01
Device name.....................: dm-0
Device driver name..............: device-mapper
Type............................: disk partition
Disk layout.....................: SCSI disk layout *)
Geometry - start................: 2048 *)
File system block size..........: 4096
Physical block size.............: 512 *)
Device size in physical blocks..: 37746688
*) Data provided by script.
Building bootmap in '/boot'
Building menu 'menu'
Adding #1: IPL section 'ubuntu' (default)
initial ramdisk...: /boot/initrd.img
signature for.....: /lib/s390-tools/stage3.bin
kernel image......: /boot/vmlinuz
signature for.....: /boot/vmlinuz
kernel parmline...: 'root=UUID=dc6b7633-49f0-4095-8c35-678cbc212ca5 crashkernel=196M'
component address:
heap area.......: 0x00002000-0x00005fff
stack area......: 0x0000f000-0x0000ffff
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x000091ff
kernel image....: 0x00010000-0x007d7fff
parmline........: 0x007d9000-0x007d91ff
initial ramdisk.: 0x007e0000-0x01a73bff
Adding #2: IPL section 'old'
initial ramdisk...: /boot/initrd.img.old
signature for.....: /lib/s390-tools/stage3.bin
kernel image......: /boot/vmlinuz.old
signature for.....: /boot/vmlinuz.old
kernel parmline...: 'root=UUID=dc6b76...

[Frank Heimes (fheimes)]

Changing affected package (from kernel to) s390-tools, according to comment #2 and the mentioned patch there.

[Launchpad Janitor (janitor)]

This bug was fixed in the package s390-tools - 2.12.0-0ubuntu1

---------------
s390-tools (2.12.0-0ubuntu1) focal; urgency=medium

  * New upstream release, plus cherrypick patches from master.
    LP: #1860574 LP: #1860531 LP: #1859018 LP: #1853308

 -- Dimitri John Ledkov <email address hidden> Wed, 29 Jan 2020 22:27:05 +0000

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-02-03 07:03 EDT-------
Retest by doing an "apt upgrade" which provided

root@t35lp36:~# apt list --installed s390-tools
Listing... Done
s390-tools/focal,now 2.12.0-0ubuntu1 s390x [installed]
root@t35lp36:~#

When trying to IPL with the HMC setting "Enable Secure Boot for Linux" selected, the system did not start. HMC messages are:

Preparing system.
Starting system.

System version 8.

Watchdog enabled.
Running 'ZBootLoader' version '1.0.0' level 'D41C.D41C_0013'.

ZBootLoader 2.0.0.

MLOLOA6269050E Secure IPL: Execute entry does not point to the beginning of a si
gned component on device HBA=0.0.1900, WWPN=500507630B01C320, LUN=40504047000000
00.

IPL failed.

With the setting "secure=1" in /etc/zipl.conf, the zipl command yields the following output:

root@t35lp36:/# zipl -V
Using config file '/etc/zipl.conf'
Run /lib/s390-tools/zipl_helper.device-mapper /boot
Target device information
Device..........................: fd:00 *)
Partition.......................: fd:01
Device name.....................: dm-0
Device driver name..............: device-mapper
Type............................: disk partition
Disk layout.....................: SCSI disk layout *)
Geometry - start................: 2048 *)
File system block size..........: 4096
Physical block size.............: 512 *)
Device size in physical blocks..: 37746688
*) Data provided by script.
Building bootmap in '/boot'
Building menu 'menu'
Adding #1: IPL section 'ubuntu' (default)
initial ramdisk...: /boot/initrd.img
kernel image......: /boot/vmlinuz
kernel parmline...: 'root=UUID=dc6b7633-49f0-4095-8c35-678cbc212ca5 crashkernel=196M'
component address:
heap area.......: 0x00002000-0x00005fff
stack area......: 0x0000f000-0x0000ffff
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x000091ff
kernel image....: 0x00010000-0x007d8fff
parmline........: 0x007da000-0x007da1ff
initial ramdisk.: 0x007e0000-0x01a883ff
Adding #2: IPL section 'old'
initial ramdisk...: /boot/initrd.img.old
kernel image......: /boot/vmlinuz.old
kernel parmline...: 'root=UUID=dc6b7633-49f0-4095-8c35-678cbc212ca5 crashkernel=196M'
component address:
heap area.......: 0x00002000-0x00005fff
stack area......: 0x0000f000-0x0000ffff
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x000091ff
kernel image....: 0x00010000-0x007d7fff
parmline........: 0x007d9000-0x007d91ff
initial ramdisk.: 0x007e0000-0x01a865ff
Preparing boot device: dm-0.
Detected SCSI PCBIOS disk layout.
Writing SCSI master boot record.
Syncing disks...
Done.
root@t35lp36:/#

In this output there is no line that reads "signature for ..." which leads to the assumption that the Ubuntu kernel is not (yet) signed.

root@t35lp36:~# uname -a
Linux t35lp36 5.4.0-12-generic #15-Ubuntu SMP Tue Jan 21 17:56:00 UTC 2020 s390x s390x s390x GNU/Linux
root@t35lp36:~#

I tried the former kernel 5.4.0-9-generic too but the result is just the same.

So the fix cannot be tested with the "as-is" distro kernels.

[Frank Heimes (fheimes)]

Did you also do a test based on a cleanly installed system, rather than after an upgrade?
If not, would you mind giving it a try?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-02-04 04:02 EDT-------
(In reply to comment #17)
> Did you also do a test based on a cleanly installed system, rather than
> after an upgrade?
> If not, would you mind giving it a try?

Retried the test with a new Ubuntu20.04 installation.
This install already had s390-tools 2.12 in place.

root@t35lp36:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu Focal Fossa (development branch)"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
root@t35lp36:~#

root@t35lp36:~# uname -a
Linux t35lp36 5.4.0-12-generic #15-Ubuntu SMP Tue Jan 21 17:56:00 UTC 2020 s390x s390x s390x GNU/Linux

root@t35lp36:~# apt list s390-tools
Listing... Done
s390-tools/focal,now 2.12.0-0ubuntu1 s390x [installed]

root@t35lp36:~# cat /etc/zipl.conf
[defaultboot]
defaultmenu = menu
secure=1

:menu
target = /boot
1 = ubuntu
2 = old
default = 1
prompt = 1
timeout = 10

[ubuntu]
target = /boot
image = /boot/vmlinuz
ramdisk = /boot/initrd.img
parameters = root=UUID=a631ceb7-59fc-4450-8ff7-208b91bd22c1 crashkernel=196M

[old]
target = /boot
image = /boot/vmlinuz.old
ramdisk = /boot/initrd.img.old
parameters = root=UUID=a631ceb7-59fc-4450-8ff7-208b91bd22c1 crashkernel=196M
optional = 1

root@t35lp36:~# zipl -V
Using config file '/etc/zipl.conf'
Run /lib/s390-tools/zipl_helper.device-mapper /boot
Target device information
Device..........................: fd:00 *)
Partition.......................: fd:01
Device name.....................: dm-0
Device driver name..............: device-mapper
Type............................: disk partition
Disk layout.....................: SCSI disk layout *)
Geometry - start................: 2048 *)
File system block size..........: 4096
Physical block size.............: 512 *)
Device size in physical blocks..: 37746688
*) Data provided by script.
Building bootmap in '/boot'
Building menu 'menu'
Adding #1: IPL section 'ubuntu' (default)
initial ramdisk...: /boot/initrd.img
kernel image......: /boot/vmlinuz
kernel parmline...: 'root=UUID=a631ceb7-59fc-4450-8ff7-208b91bd22c1 crashkernel=196M'
component address:
heap area.......: 0x00002000-0x00005fff
stack area......: 0x0000f000-0x0000ffff
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x000091ff
kernel image....: 0x00010000-0x007d8fff
parmline........: 0x007da000-0x007da1ff
initial ramdisk.: 0x007e0000-0x01a8f5ff
Adding #2: IPL section 'old'
initial ramdisk...: /boot/initrd.img.old
kernel image......: /boot/vmlinuz.old
kernel parmline...: 'root=UUID=a631ceb7-59fc-4450-8ff7-208b91bd22c1 crashkernel=196M'
component address:
heap area.......: 0x00002000-0x00005fff
stack area......: 0x0000f000-0x0000ffff
internal loader.: 0x0000a000-0x0000dfff
parameters......: 0x00009000-0x000091ff
kernel image....: 0x00010000-0x007d8fff
parmline........: 0x007da000-0x007da1ff
initial ramdisk.: 0x007e0000-0x01a8f5ff
Preparing boot device: dm-0.
Detected SCSI PCBIOS...

[Dimitri John Ledkov (xnox)]

In above logs zipl.conf is shown to be like this:

root@t35lp36:~# cat /etc/zipl.conf
[defaultboot]
defaultmenu = menu
secure=1

:menu
target = /boot
1 = ubuntu
2 = old
default = 1
prompt = 1
timeout = 10

However, for me, setting secure like that has never worked.

Instead i had to set secure=1 on the ':menu' portion of the zipl.conf file, i.e.

root@t35lp36:~# cat /etc/zipl.conf
[defaultboot]
defaultmenu = menu

:menu
target = /boot
1 = ubuntu
2 = old
default = 1
prompt = 1
timeout = 10
secure=1

Can it be that this is leading to incorrect testing?

Also I wanted to make sure you have the right kernel installed.

Can you please doublecheck output for all of the below commands is the same for you?

$ dpkg-query -W linux-image-5.4.0-12-generic
linux-image-5.4.0-12-generic 5.4.0-12.15

$ sudo md5sum /boot/vmlinuz /boot/vmlinuz-5.4.0-12-generic
6e2c2d81d3fa1d50bd3b30f12085554b /boot/vmlinuz
6e2c2d81d3fa1d50bd3b30f12085554b /boot/vmlinuz-5.4.0-12-generic

$ grep vmlinuz /var/lib/dpkg/info/linux-image-5.4.0-12-generic.md5sums
6e2c2d81d3fa1d50bd3b30f12085554b boot/vmlinuz-5.4.0-12-generic

To double check that signature is present on /boot/vmlinuz you can use the extract-module-sig.pl from the linux source tree scripts directly and then run something like this:

$ sudo perl linux/scripts/extract-module-sig.pl -d /boot/vmlinuz
Read 8163896 bytes from module file
Found magic number at 8163896
Found PKCS#7/CMS encapsulation
Found 528 bytes of signature [3082020c06092a864886f70d010702a0]
0 0 2 0 0 528

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-02-06 03:44 EDT-------
Retested with the secure entry moved to the menu section:

[defaultboot]
defaultmenu = menu

:menu
target = /boot
1 = ubuntu
2 = old
default = 1
prompt = 1
timeout = 10
secure=1
.
.

root@t35lp36:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu Focal Fossa (development branch)"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@t35lp36:~# uname -a
Linux t35lp36 5.4.0-12-generic #15-Ubuntu SMP Tue Jan 21 17:56:00 UTC 2020 s390x s390x s390x GNU/Linux

root@t35lp36:~# apt list s390-tools
Listing... Done
s390-tools/focal,now 2.12.0-0ubuntu1 s390x [installed]
root@t35lp36:~#

With the new placement of the "secure" keyword, secure boot works as expected:

(1) IPL always possible with the "Enable secure boot for Linux" HMC checkbox
disabled for secure=1/0/auto. /sys/firmware/ipl/secure shows value 0 after IPL.

(2) IPL successful with the "Enable secure boot for Linux" HMC checkbox
enabled for secure=1/auto. /sys/firmware/ipl/secure shows value 1 after IPL.

(3) No IPL with the "Enable secure boot for Linux" checkbox enabled for secure=0.
Console messages in this case

Preparing system.
Starting system.
System version 8.
Watchdog enabled.
Running 'ZBootLoader' version '1.0.0' level 'D41C.D41C_0013'.
ZBootLoader 2.0.0.
MLOLOA6269050E Secure IPL: Execute entry does not point to the beginning of a signed component on device HBA=0.0.1900, WWPN=500507630B01C320, LUN=4050404700000000.
IPL failed.

But for the secure IPLs (2) the console shows about 1800 messages (or more)
that look like:

[ 2.485469] Lockdown: swapper/0: use of tracefs is restricted; see man kernel_lockdown.7
[ 2.485471] Could not create tracefs 'available_events' entry

with occasional intersections like these:

[ 2.487994] ------------[ cut here ]------------
[ 2.487995] Could not register function stat for cpu 0
[ 2.488004] WARNING: CPU: 0 PID: 1 at kernel/trace/ftrace.c:987 ftrace_init_tracefs_toplevel+0x160/0x1b8
[ 2.488005] Modules linked in:
[ 2.488007] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.0-12-generic #15-Ubuntu
[ 2.488008] Hardware name: IBM 8561 T01 703 (LPAR)
[ 2.488009] Krnl PSW : 0704f00180000000 00000000c886b0d0 (ftrace_init_tracefs_toplevel+0x160/0x1b8)
[ 2.488011] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:3 PM:0 RI:0 EA:3
[ 2.488013] Krnl GPRS: 000000000000000a 00000000c8794110 000000000000002a 0000000000000001
[ 2.488014] 0000000000000f3b 000000007fe06000 0000000000000000 00000000c88fedb8
[ 2.488015] 00000000c8958000 0000000000000000 00000000f1081e70 0000000000000000
[ 2.488015] 00000000f093b300 00000000f19d2000 00000000c886b0cc 000003e00000bcd8
[ 2.488020] Krnl Code: 00000000c886b0c0: c020ffeb5dd3 larl %r2,00000000c85d6c66
00000000c886b0c6: c0e5ff9a87e5 brasl %r14,00000000c7bbc090
#00000000c8...

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2020-02-06 03:43 EDT-------

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-02-06 03:53 EDT-------
The correct behaviour documented currently in the man-pages, will be implemented once the addl. bugzilla: https://bugzilla.linux.ibm.com/show_bug.cgi?id=183663 / LP = https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/1861994
will be applied.

After integration a short restest should be done.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-02-06 07:32 EDT-------
After testing, it seems to be that the behaviour of the settings need addl. fixes... Stay tuned.

------- Comment From <email address hidden> 2020-02-06 07:35 EDT-------
OK, to be more precise. You can have the secure= entry in any of the following places:

- in a configuration section:

Example:
[test]
target = /boot
image = /boot/image-test
parmfile = /boot/parmfile-test
secure = 1

- in the menu section:

Example:
:menu1
target = /boot
1 = linux
2 = test
default = 1
prompt = 1
timeout = 0
secure = 1

- in the defaultboot section with defaultauto
when NOT using a menu but generating an auto menu
(this can be seen as a replacement for the :menu section)

Example:
[defaultboot]
defaultauto
target = /boot
default = 1
secure = 1

You are right, the man page is misleading as it combines the [defaultboot] section containing a defaultmenu with the secure= entry. This does not work and we should fix the man page here.

I do not completely understand the following question:

> With multiple menus in zipl.conf: how does zipl -S work?

There is only one active menu in the zipl.conf

But nevertheless "zipl -S" should overwrite every setting from the zipl.conf regardless where secure= is specified.

Also combinations of the various places are possible. The most significant entry wins.
The expected significance of the different positions should be:

"zipl-S " > menu section or "defaultauto" section > configuration section

Hope this clarifies the situation a bit more.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-04-02 07:23 EDT-------
IBM Bugzilla status-> closed, Fix with updates man-pages.

[Frank Heimes (fheimes)]

Changing the Eoan entry to Invalid, since Eoan reached it's EOL: https://lists.ubuntu.com/archives/ubuntu-announce/2020-July/000258.html