Activity log for bug #1848173

Date Who What changed Old value New value Message
2019-10-15 10:09:25 bugproxy bug added bug
2019-10-15 10:09:27 bugproxy tags architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046
2019-10-15 10:09:28 bugproxy attachment added CEX7 toleration patch for Ubuntu 16.04 https://bugs.launchpad.net/bugs/1848173/+attachment/5297314/+files/s390-zcrypt-CEX7-toleration-support.patch
2019-10-15 10:09:30 bugproxy ubuntu: assignee Skipper Bug Screeners (skipper-screen-team)
2019-10-15 10:09:33 bugproxy affects ubuntu linux (Ubuntu)
2019-10-15 10:12:40 Frank Heimes bug task added ubuntu-z-systems
2019-10-15 10:12:47 Frank Heimes ubuntu-z-systems: status New Triaged
2019-10-15 10:12:55 Frank Heimes ubuntu-z-systems: importance Undecided Medium
2019-10-15 19:13:49 Frank Heimes summary Ubuntu16.04.6 - shared CEX7C cards defined in z/VM guest not established by zcrypt device driver Ubuntu 16.04.6 - shared CEX7C cards defined in z/VM guest not established by zcrypt device driver
2019-10-16 10:45:46 Frank Heimes summary Ubuntu 16.04.6 - shared CEX7C cards defined in z/VM guest not established by zcrypt device driver Ubuntu 16.04.6 - Shared CEX7C cards defined in z/VM guest not established by zcrypt device driver
2019-10-16 13:06:11 Frank Heimes description System: IBM Z15 z/VM with shared CEX7C adapters OS: Ubuntu 16.04.6 LTS ( 4.4.0-165-generic kernel ) with latest updates Shared CEX7C adapters are not displayed on Ubuntu even though APAR 66266 had been installed onto the unterlying z/VM system. Details ======= Defined shared CEX7C CCA adapters to provide cryptographic accelerators based on CCA cards to a z/VM guest system running Ubuntu 16.04.6 LTS. The adapters display all right under vm or when running vmcp commands under Linux. lszcrypt -VVV does not display any adapter. We observed that zcrypt_cex4 was not automatically loaded via dependency by modprobe ap. Explicitly loading by modprobe zcrypt_cex4 did not change card availability. Please investigate. Thanks. Terminal output ============== root@system:/sys/bus/ap/devices/card01# ls -l total 0 -r--r--r-- 1 root root 4096 Oct 8 17:51 ap_functions -r--r--r-- 1 root root 4096 Oct 8 17:51 depth -r--r--r-- 1 root root 4096 Oct 8 17:51 hwtype -r--r--r-- 1 root root 4096 Oct 8 17:51 interrupt -r--r--r-- 1 root root 4096 Oct 8 17:51 modalias -r--r--r-- 1 root root 4096 Oct 8 17:51 pendingq_count drwxr-xr-x 2 root root 0 Oct 8 17:51 power -r--r--r-- 1 root root 4096 Oct 8 17:51 raw_hwtype -r--r--r-- 1 root root 4096 Oct 8 17:51 request_count -r--r--r-- 1 root root 4096 Oct 8 17:51 requestq_count -r--r--r-- 1 root root 4096 Oct 8 17:51 reset lrwxrwxrwx 1 root root 0 Oct 8 17:51 subsystem -> ../../../bus/ap -rw-r--r-- 1 root root 4096 Oct 8 17:50 uevent # lszcrypt -V // < No output displayed > # vmcp q v crypto AP 001 CEX7C Domain 001 shared online root@system:/sys/bus/ap/devices/card01# cat hwtype 13 root@system:/sys/bus/ap/devices/card01# cat raw_hwtype 13 # lsmod Module Size Used by ap 36864 0 ghash_s390 16384 0 prng 16384 0 aes_s390 20480 0 des_s390 16384 0 des_generic 28672 1 des_s390 sha512_s390 16384 0 qeth_l2 53248 1 sha256_s390 16384 0 sha1_s390 16384 0 sha_common 16384 3 sha256_s390,sha1_s390,sha512_s390 qeth 151552 1 qeth_l2 vmur 20480 0 ccwgroup 20480 1 qeth dm_multipath 36864 0 zfcp 143360 0 dasd_eckd_mod 118784 8 qdio 73728 3 qeth,zfcp,qeth_l2 scsi_transport_fc 86016 1 zfcp dasd_mod 135168 5 dasd_eckd_mod # modprobe zcrypt_cex4 ... zcrypt_cex4 16384 0 zcrypt_api 36864 1 zcrypt_cex4 ap 36864 2 zcrypt_cex4,zcrypt_api ... Contact Information = Christian.Rund@de.ibm.com ---uname output--- Linux system 4.4.0-164-generic #192-Ubuntu SMP Fri Sep 13 12:01:28 UTC 2019 s390x s390x s390x GNU/Linux Machine Type = IBM Type: 8561 Model: 403 T01 ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1.) Define shared CEX7 CCA cards to z/VM Guest 2.) boot up Ubuntu 16.04.6 LTS 3.) modprobe ap 4.) lszcrypt -VVV Stack trace output: no Oops output: no System Dump Info: The system is not configured to capture a system dump. Device driver error code: N/A *Additional Instructions for Christian.Rund@de.ibm.com: -Attach sysctl -a output output to the bug. lszcrypt returns with # lszcrypt -VVV ; echo RC=$? RC=0 After investigating here a little ... Ubuntu 16.04 has only toleration support for CEX6 and no support for CEX7. Here is a patch which maps cex7 cards to cex5 cards. Have a look into - it is just a 2 line code change which extends the toleration patch for cex6 (mapped to cex5) by the cex7 card - also mapped to cex5. Code compiles and I've tested the kernel on a z15 with lots of cex6 and cex7 cards - works fine. SRU Justification: ================== [Impact] * Ubuntu 16.04.6 systems on z15 with crypto CEX7C adapters under z/VM cannot see and make use of their hw crypto resources. * The patch/backport adds CEX7 toleration support (by mapping it to CEX5) to kernel 4.4. [Fix] * Backport: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1848173/+attachment/5297314/+files/s390-zcrypt-CEX7-toleration-support.patch [Test Case] * Define a z/VM guest with 'apvirt' (hardware crypto adapter virtualization) having CryptoExpress 7S adapters attached to z/VM LPAR. * Use lszcrypt command (ideally lszcrypt -VVV) from the s390-tools package to list the detected and available hardware crypto resources. * Canonical can only do a toleration test: IBM needs to do the functional test (due to hardware availability). [Regression Potential] * The regression potential can be considered as moderate since this is purely s390x specific * and limited to CryptoExpress 7S (CEX7) adapter cards * and again if they running under z/VM (on z15) with 'apvirt' configured for the guest. * and again only with 16.04.6's kernel 4.4. [Other Info] * The patch was already applied, kernel compiled and things tested on z15 und z/VM. __________ System: IBM Z15 z/VM with shared CEX7C adapters OS: Ubuntu 16.04.6 LTS ( 4.4.0-165-generic kernel ) with latest updates Shared CEX7C adapters are not displayed on Ubuntu even though APAR 66266 had been installed onto the unterlying z/VM system. Details ======= Defined shared CEX7C CCA adapters to provide cryptographic accelerators based on CCA cards to a z/VM guest system running Ubuntu 16.04.6 LTS. The adapters display all right under vm or when running vmcp commands under Linux. lszcrypt -VVV does not display any adapter. We observed that zcrypt_cex4 was not automatically loaded via dependency by modprobe ap. Explicitly loading by modprobe zcrypt_cex4 did not change card availability. Please investigate. Thanks. Terminal output ============== root@system:/sys/bus/ap/devices/card01# ls -l total 0 -r--r--r-- 1 root root 4096 Oct 8 17:51 ap_functions -r--r--r-- 1 root root 4096 Oct 8 17:51 depth -r--r--r-- 1 root root 4096 Oct 8 17:51 hwtype -r--r--r-- 1 root root 4096 Oct 8 17:51 interrupt -r--r--r-- 1 root root 4096 Oct 8 17:51 modalias -r--r--r-- 1 root root 4096 Oct 8 17:51 pendingq_count drwxr-xr-x 2 root root 0 Oct 8 17:51 power -r--r--r-- 1 root root 4096 Oct 8 17:51 raw_hwtype -r--r--r-- 1 root root 4096 Oct 8 17:51 request_count -r--r--r-- 1 root root 4096 Oct 8 17:51 requestq_count -r--r--r-- 1 root root 4096 Oct 8 17:51 reset lrwxrwxrwx 1 root root 0 Oct 8 17:51 subsystem -> ../../../bus/ap -rw-r--r-- 1 root root 4096 Oct 8 17:50 uevent # lszcrypt -V // < No output displayed > # vmcp q v crypto AP 001 CEX7C Domain 001 shared online root@system:/sys/bus/ap/devices/card01# cat hwtype 13 root@system:/sys/bus/ap/devices/card01# cat raw_hwtype 13 # lsmod Module Size Used by ap 36864 0 ghash_s390 16384 0 prng 16384 0 aes_s390 20480 0 des_s390 16384 0 des_generic 28672 1 des_s390 sha512_s390 16384 0 qeth_l2 53248 1 sha256_s390 16384 0 sha1_s390 16384 0 sha_common 16384 3 sha256_s390,sha1_s390,sha512_s390 qeth 151552 1 qeth_l2 vmur 20480 0 ccwgroup 20480 1 qeth dm_multipath 36864 0 zfcp 143360 0 dasd_eckd_mod 118784 8 qdio 73728 3 qeth,zfcp,qeth_l2 scsi_transport_fc 86016 1 zfcp dasd_mod 135168 5 dasd_eckd_mod # modprobe zcrypt_cex4 ... zcrypt_cex4 16384 0 zcrypt_api 36864 1 zcrypt_cex4 ap 36864 2 zcrypt_cex4,zcrypt_api ... Contact Information = Christian.Rund@de.ibm.com ---uname output--- Linux system 4.4.0-164-generic #192-Ubuntu SMP Fri Sep 13 12:01:28 UTC 2019 s390x s390x s390x GNU/Linux Machine Type = IBM Type: 8561 Model: 403 T01 ---Debugger--- A debugger is not configured ---Steps to Reproduce---  1.) Define shared CEX7 CCA cards to z/VM Guest 2.) boot up Ubuntu 16.04.6 LTS 3.) modprobe ap 4.) lszcrypt -VVV Stack trace output:  no Oops output:  no System Dump Info:   The system is not configured to capture a system dump. Device driver error code:  N/A *Additional Instructions for Christian.Rund@de.ibm.com: -Attach sysctl -a output output to the bug. lszcrypt returns with # lszcrypt -VVV ; echo RC=$? RC=0 After investigating here a little ... Ubuntu 16.04 has only toleration support for CEX6 and no support for CEX7. Here is a patch which maps cex7 cards to cex5 cards. Have a look into - it is just a 2 line code change which extends the toleration patch for cex6 (mapped to cex5) by the cex7 card - also mapped to cex5. Code compiles and I've tested the kernel on a z15 with lots of cex6 and cex7 cards - works fine.
2019-10-16 13:06:47 Frank Heimes linux (Ubuntu): status New In Progress
2019-10-16 13:06:50 Frank Heimes ubuntu-z-systems: assignee Frank Heimes (frank-heimes)
2019-10-16 13:06:54 Frank Heimes ubuntu-z-systems: status Triaged In Progress
2019-10-17 16:05:59 Kleber Sacilotto de Souza nominated for series Ubuntu Xenial
2019-10-17 16:05:59 Kleber Sacilotto de Souza bug task added linux (Ubuntu Xenial)
2019-10-17 16:06:09 Kleber Sacilotto de Souza linux (Ubuntu Xenial): status New In Progress
2019-10-17 16:06:52 Kleber Sacilotto de Souza linux (Ubuntu): status In Progress Invalid
2019-10-21 03:40:04 Khaled El Mously linux (Ubuntu Xenial): status In Progress Fix Committed
2019-10-21 05:23:42 Frank Heimes ubuntu-z-systems: status In Progress Fix Committed
2019-10-22 16:04:51 Ubuntu Kernel Bot tags architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046 architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046 verification-needed-xenial
2019-10-23 12:29:36 Frank Heimes tags architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046 verification-needed-xenial architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046 verification-done-xenial
2019-11-12 22:33:58 Launchpad Janitor linux (Ubuntu Xenial): status Fix Committed Fix Released
2019-11-12 22:33:58 Launchpad Janitor cve linked 2018-12207
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-0154
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-0155
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-11135
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-15098
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17052
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17053
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17054
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17055
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17056
2019-11-12 22:33:58 Launchpad Janitor cve linked 2019-17666
2019-11-12 23:05:46 Andrew Cloke ubuntu-z-systems: status Fix Committed Fix Released