Got Segmentation fault when enable zipl -V on the guest.

[[e-mail]]

I am test ubuntu secure execution on Ubuntu22.04 s390x on DPM mode.

Host secure execution enablement is OK.

root@a257seubut:/home/ubuntu# cat /proc/cmdline
root=/dev/disk/by-id/dm-uuid-part1-mpath-20017380030bb1668 prot_virt=1
root@a257seubut:/home/ubuntu# dmesg | grep ultravisor
[ 0.119067] prot_virt: Reserving 130MB as ultravisor base storage
root@a257seubut:/home/ubuntu#

And I have create some Ubuntu guest on it.
But when I enable guest execution, I got this Segmentation fault error.

ubuntu@a257ubtu20seon04:~$ cat parmfile
root=/dev/disk/by-path/ccw-0.0.0000-part1 swiotlb=262144

ubuntu@a257ubtu20seon04:~$ cat /etc/zipl.conf
[defaultboot]
defaultmenu = menu

:menu
target = /boot
1 = ubuntu
3 = secure
default = 3
prompt = 1
timeout = 10
[secure]
target = /boot
image = /boot/secure-linux
[ubuntu]
target = /boot
image = /boot/vmlinuz
ramdisk = /boot/initrd.img
parameters = root=/dev/disk/by-path/ccw-0.0.0000-part1

ubuntu@a257ubtu20seon04:~$ sudo genprotimg -i /boot/vmlinuz -r /boot/initrd.img -p parmfile -k HKD-3932-02860A8.crt --no-verify -o /boot/secure-linux -V
WARNING: host-key document verification is disabled. Your workload is not secured.
      kernel: 0x000000015000 ( 8085504 / 8081976 Bytes)
    parmline: 0x0000007cb000 ( 4096 / 57 Bytes)
     ramdisk: 0x0000007cc000 ( 26640384 / 26639555 Bytes)
     stage3b: 0x000002134000 ( 8192 / 5716 Bytes)
     stage3a: 0x000000010000 ( 20480 / 20480 Bytes)

ubuntu@a257ubtu20seon04:~$ sudo zipl -V
Looking for components in '/lib/s390-tools'
Using config file '/etc/zipl.conf'
[ 1326.923001] User process fault: interruption code 0010 ilc:3 in zipl[1000000+26000]
[ 1326.923012] Failing address: 0000000000000000 TEID: 0000000000000800
[ 1326.923014] Fault in primary space mode while using user ASCE.
[ 1326.923017] AS:0000000085de01c7 R3:0000000085260007 S:0000000000000020
Segmentation fault
ubuntu@a257ubtu20seon04:~$

ubuntu@a257seubut:~$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
ubuntu@a257seubut:~$

Could you help to take a look for this? If you need more info, please let me know.

[[e-mail] (liwbj)]

I found this syslog

Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612939] User process fault: interruption code 0010 ilc:3 in zipl[1000000+26000]
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612949] Failing address: 0000000000000000 TEID: 0000000000000800
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612951] Fault in primary space mode while using user ASCE.
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612954] AS:00000000852301c7 R3:00000000827bc007 S:0000000000000020
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612958] CPU: 1 PID: 1888 Comm: zipl Not tainted 5.15.0-53-generic #59-Ubuntu
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612961] Hardware name: IBM 3932 LA2 A00 (KVM/Linux)
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612962] User PSW : 0705000180000000 0000000001008938
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612963] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:1 AS:0 CC:0 PM:0 RI:0 EA:3
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612966] User GPRS: 0000000000000000 0000000000000000 0000000000000000 000000000000000f
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612967] 0000000000000380 0000000000000000 0000000001b28040 000003ff00000000
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612968] 0000000000000068 0000000001b28250 000003ff85c70720 0000000000000001
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612969] 0000000001b28040 0000000001b28250 00000000010088de 000003ffe7f7f1a0
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] User Code: 000000000100892a: a72b0010 aghi %r2,16
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] 000000000100892e: e70000300081 vfene %v0,%v0,%v0,0,3
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] #0000000001008934: a784000c brc 8,000000000100894c
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] >0000000001008938: e70210004006 vl %v0,0(%r2,%r1),4
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] 000000000100893e: a72b0010 aghi %r2,16
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] 0000000001008942: e70000300081 vfene %v0,%v0,%v0,0,3
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] 0000000001008948: a774fff8 brc 7,0000000001008938
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612976] 000000000100894c: e75000070021 vlgv %r5,%v0,7,0
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612987] Last Breaking-Event-Address:
Nov 19 01:18:15 a257ubtu20seon04 kernel: [ 5473.612987] [<0000000001008a2a>] 0x1008a2a
Nov 19 01:18:55 a257ubtu20seon04 kernel: [ 5513.312528] User process fault: interruption code 0010 ilc:3 in zipl[1000000+26000]
Nov 19 01:18:55 a257ubtu20seon04 kernel: [ 5513.312541] Failing address: 0000000000000000 TEID: 0000000000000800
Nov 19 01:18:55 a257ubtu20seon04 kernel: [ 5513.312542] Fault in primary space mode while using user ASCE.
Nov 19 01:18:55 a257ubtu20seon04 kernel: [ 5513.312545] AS:00000000825441c7 R3:0000000081fa0007 S:0000000000000020
Nov 19 01:18:55 a257ubtu20seon04 kernel: [ 5513.312549] CPU: 0 PID: 1904 Comm: zipl Not tainted 5.15.0-53-generic #59-Ubuntu
Nov 19 01:18:55 a257ubtu20seon04 ker...

[Frank Heimes (fheimes)]

Hello liwbj, thanks for the report.
Would you please also share the out put of:
apt-cache policy s390-tools
and
zipl --version

It does seem to behave like expected in non SE environments.

I think this ticket need to be reverse mirrored to IBM development...

[[e-mail] (liwbj)]

ubuntu@a257ubtu20seon04:~$ sudo zipl -V
[sudo] password for ubuntu:
Looking for components in '/lib/s390-tools'
Using config file '/etc/zipl.conf'
[255787.244767] User process fault: interruption code 0010 ilc:3 in zipl[1000000+26000]
[255787.244778] Failing address: 0000000000000000 TEID: 0000000000000800
[255787.244780] Fault in primary space mode while using user ASCE.
[255787.244783] AS:0000000081d541c7 R3:0000000083590007 S:0000000000000020
Segmentation fault
ubuntu@a257ubtu20seon04:~$
ubuntu@a257ubtu20seon04:~$ apt-cache policy s390-tools
s390-tools:
  Installed: 2.20.0-0ubuntu3.1
  Candidate: 2.20.0-0ubuntu3.1
  Version table:
 *** 2.20.0-0ubuntu3.1 500
        500 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main s390x Packages
        100 /var/lib/dpkg/status
     2.20.0-0ubuntu3 500
        500 http://ports.ubuntu.com/ubuntu-ports jammy/main s390x Packages
ubuntu@a257ubtu20seon04:~$ zipl --version
zipl: zSeries Initial Program Loader version 2.20.0-build-20220623
Copyright IBM Corp. 2001, 2017
ubuntu@a257ubtu20seon04:~$ date
Tue Nov 22 12:59:46 AM UTC 2022
ubuntu@a257ubtu20seon04:~$

[Frank Heimes (fheimes)]

Looking at this again, there is a small chance that it's related to LP#1974109.

So would you please try the s390-tools version from this PPA:
https://launchpad.net/~fheimes/+archive/ubuntu/lp1974109+lp1959987+lp1990520+lp1990524+lp1996069+lp1996477
respectively:
https://launchpad.net/~fheimes/+archive/ubuntu/lp1974109+lp1959987+lp1990520+lp1990524+lp1996069+lp1996477/+packages

Since this version fixes several issues.

[[e-mail] (liwbj)]

Yeah, updated to 2.20.0-0ubuntu3.2. It works fine now.
Thank you very much, Frank.

add-apt-repository deb https://ppa.launchpadcontent.net/fheimes/lp1974109+lp1959987+lp1990520+lp1990524+lp1996069+lp1996477/ubuntu jammy main

root@a257ubtu20seon04:/etc# apt upgrade -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
.....
Preparing to unpack .../s390-tools_2.20.0-0ubuntu3.2_s390x.deb ...
Unpacking s390-tools (2.20.0-0ubuntu3.2) over (2.20.0-0ubuntu3.1) ...
Preparing to unpack .../s390-tools-signed_2.20.0-0ubuntu3.2_s390x.deb ...
Unpacking s390-tools-signed (2.20.0-0ubuntu3.2) over (2.20.0-0ubuntu3.1) ...
Setting up s390-tools-signed (2.20.0-0ubuntu3.2) ...
Setting up s390-tools (2.20.0-0ubuntu3.2) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for initramfs-tools (0.140ubuntu13) ...
update-initramfs: Generating /boot/initrd.img-5.15.0-53-generic
Using config file '/etc/zipl.conf'
Building bootmap in '/boot'

root@a257ubtu20seon04:~# zipl -V
Looking for components in '/lib/s390-tools'
Using config file '/etc/zipl.conf'
Target device information
  Device..........................: fc:00
  Partition.......................: fc:01
  Device name.....................: vda
  Device driver name..............: virtblk
  Type............................: disk partition
  Disk layout.....................: SCSI disk layout
  Geometry - start................: 2048
  File system block size..........: 4096
  Physical block size.............: 512
  Device size in physical blocks..: 41938944
Building bootmap in '/boot'
Building menu 'menu'
Adding #1: IPL section 'ubuntu'
  initial ramdisk...: /boot/initrd.img
  kernel image......: /boot/vmlinuz
  kernel parmline...: 'root=/dev/disk/by-path/ccw-0.0.0000-part1'
  component address:
    heap area.......: 0x00002000-0x00005fff
    stack area......: 0x0000f000-0x0000ffff
    internal loader.: 0x0000a000-0x0000dfff
    parameters......: 0x00009000-0x000091ff
    kernel image....: 0x00010000-0x007c4fff
    parmline........: 0x007c6000-0x007c61ff
    initial ramdisk.: 0x007e0000-0x02147fff
    environment blk.: 0x007d6000-0x007d6fff
Adding #3: IPL section 'secure' (default)
  kernel image......: /boot/secure-linux
  component address:
    heap area.......: 0x00002000-0x00005fff
    stack area......: 0x0000f000-0x0000ffff
    internal loader.: 0x0000a000-0x0000dfff
    parameters......: 0x00009000-0x000091ff
    kernel image....: 0x00010000-0x02145fff
    environment blk.: 0x02156000-0x02156fff
Environment block content:
Preparing boot device: vda (0000).
Detected SCSI PCBIOS disk layout.
Writing SCSI master boot record.
Syncing disks...
Done.
root@a257ubtu20seon04:~#

[Frank Heimes (fheimes)]

Okay, great - and thanks for testing!.
This new version is currently in the process to be rolled out (we call it SRU process).
Will soon land in jammy-proposed and later in jammy-updates.

I'm marking this as duplicate of LP#1974109,
since the fix for this ticket obviously also fixes this bug.