System: IBM Z15 z/VM with shared CEX7C adapters
OS: Ubuntu 16.04.6 LTS ( 4.4.0-165-generic kernel ) with latest updates
Shared CEX7C adapters are not displayed on Ubuntu even though APAR 66266 had been installed onto the unterlying z/VM system.
Details
=======
Defined shared CEX7C CCA adapters to provide cryptographic accelerators based on CCA cards to a z/VM guest system running Ubuntu 16.04.6 LTS.
The adapters display all right under vm or when running vmcp commands under Linux.
lszcrypt -VVV does not display any adapter.
We observed that zcrypt_cex4 was not automatically loaded via dependency by modprobe ap. Explicitly loading by modprobe zcrypt_cex4 did not change card availability.
Please investigate.
Thanks.
Terminal output
==============
root@system:/sys/bus/ap/devices/card01# ls -l
total 0
-r--r--r-- 1 root root 4096 Oct 8 17:51 ap_functions
-r--r--r-- 1 root root 4096 Oct 8 17:51 depth
-r--r--r-- 1 root root 4096 Oct 8 17:51 hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 interrupt
-r--r--r-- 1 root root 4096 Oct 8 17:51 modalias
-r--r--r-- 1 root root 4096 Oct 8 17:51 pendingq_count
drwxr-xr-x 2 root root 0 Oct 8 17:51 power
-r--r--r-- 1 root root 4096 Oct 8 17:51 raw_hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 request_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 requestq_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 reset
lrwxrwxrwx 1 root root 0 Oct 8 17:51 subsystem -> ../../../bus/ap
-rw-r--r-- 1 root root 4096 Oct 8 17:50 uevent
# lszcrypt -V // < No output displayed >
# vmcp q v crypto
AP 001 CEX7C Domain 001 shared online
---uname output---
Linux system 4.4.0-164-generic #192-Ubuntu SMP Fri Sep 13 12:01:28 UTC 2019 s390x s390x s390x GNU/Linux
Machine Type = IBM Type: 8561 Model: 403 T01
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1.) Define shared CEX7 CCA cards to z/VM Guest
2.) boot up Ubuntu 16.04.6 LTS
3.) modprobe ap
4.) lszcrypt -VVV
Stack trace output:
no
Oops output:
no
System Dump Info:
The system is not configured to capture a system dump.
Device driver error code:
N/A
*Additional Instructions for <email address hidden>:
-Attach sysctl -a output output to the bug.
lszcrypt returns with
# lszcrypt -VVV ; echo RC=$?
RC=0
After investigating here a little ...
Ubuntu 16.04 has only toleration support for CEX6 and no support for CEX7.
Here is a patch which maps cex7 cards to cex5 cards.
Have a look into - it is just a 2 line code change which
extends the toleration patch for cex6 (mapped to cex5)
by the cex7 card - also mapped to cex5.
Code compiles and I've tested the kernel on a z15 with
lots of cex6 and cex7 cards - works fine.
System: IBM Z15 z/VM with shared CEX7C adapters
OS: Ubuntu 16.04.6 LTS ( 4.4.0-165-generic kernel ) with latest updates
Shared CEX7C adapters are not displayed on Ubuntu even though APAR 66266 had been installed onto the unterlying z/VM system.
Details
=======
Defined shared CEX7C CCA adapters to provide cryptographic accelerators based on CCA cards to a z/VM guest system running Ubuntu 16.04.6 LTS.
The adapters display all right under vm or when running vmcp commands under Linux.
lszcrypt -VVV does not display any adapter.
We observed that zcrypt_cex4 was not automatically loaded via dependency by modprobe ap. Explicitly loading by modprobe zcrypt_cex4 did not change card availability.
Please investigate.
Thanks.
Terminal output /sys/bus/ ap/devices/ card01# ls -l
==============
root@system:
total 0
-r--r--r-- 1 root root 4096 Oct 8 17:51 ap_functions
-r--r--r-- 1 root root 4096 Oct 8 17:51 depth
-r--r--r-- 1 root root 4096 Oct 8 17:51 hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 interrupt
-r--r--r-- 1 root root 4096 Oct 8 17:51 modalias
-r--r--r-- 1 root root 4096 Oct 8 17:51 pendingq_count
drwxr-xr-x 2 root root 0 Oct 8 17:51 power
-r--r--r-- 1 root root 4096 Oct 8 17:51 raw_hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 request_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 requestq_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 reset
lrwxrwxrwx 1 root root 0 Oct 8 17:51 subsystem -> ../../../bus/ap
-rw-r--r-- 1 root root 4096 Oct 8 17:50 uevent
# lszcrypt -V // < No output displayed >
# vmcp q v crypto
AP 001 CEX7C Domain 001 shared online
root@system: /sys/bus/ ap/devices/ card01# cat hwtype /sys/bus/ ap/devices/ card01# cat raw_hwtype
13
root@system:
13
# lsmod s390,sha1_ s390,sha512_ s390
Module Size Used by
ap 36864 0
ghash_s390 16384 0
prng 16384 0
aes_s390 20480 0
des_s390 16384 0
des_generic 28672 1 des_s390
sha512_s390 16384 0
qeth_l2 53248 1
sha256_s390 16384 0
sha1_s390 16384 0
sha_common 16384 3 sha256_
qeth 151552 1 qeth_l2
vmur 20480 0
ccwgroup 20480 1 qeth
dm_multipath 36864 0
zfcp 143360 0
dasd_eckd_mod 118784 8
qdio 73728 3 qeth,zfcp,qeth_l2
scsi_transport_fc 86016 1 zfcp
dasd_mod 135168 5 dasd_eckd_mod
# modprobe zcrypt_cex4 cex4,zcrypt_ api
...
zcrypt_cex4 16384 0
zcrypt_api 36864 1 zcrypt_cex4
ap 36864 2 zcrypt_
...
Contact Information = <email address hidden>
---uname output---
Linux system 4.4.0-164-generic #192-Ubuntu SMP Fri Sep 13 12:01:28 UTC 2019 s390x s390x s390x GNU/Linux
Machine Type = IBM Type: 8561 Model: 403 T01
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1.) Define shared CEX7 CCA cards to z/VM Guest
2.) boot up Ubuntu 16.04.6 LTS
3.) modprobe ap
4.) lszcrypt -VVV
Stack trace output:
no
Oops output:
no
System Dump Info:
The system is not configured to capture a system dump.
Device driver error code:
N/A
*Additional Instructions for <email address hidden>:
-Attach sysctl -a output output to the bug.
lszcrypt returns with
# lszcrypt -VVV ; echo RC=$?
RC=0
After investigating here a little ...
Ubuntu 16.04 has only toleration support for CEX6 and no support for CEX7.
Here is a patch which maps cex7 cards to cex5 cards.
Have a look into - it is just a 2 line code change which
extends the toleration patch for cex6 (mapped to cex5)
by the cex7 card - also mapped to cex5.
Code compiles and I've tested the kernel on a z15 with
lots of cex6 and cex7 cards - works fine.