[uacc-0623] hisi_sec2 fail to alloc uacce

[Fred Kimmy]

SRU Justification
=================

[Impact]
* Users with HiSilicon Security Engine (SEC) version 2 controller fail to allocate Uacce.

[Fix]
* upstream 183b60e005975d3c84c22199ca64a9221e620fb6 crypto: hisilicon/qm - modify the uacce mode check

[Test Plan]
* Deploy Ubuntu Focal with HWE 5.13 kernel to a Kunpeng920 chip with HiSilicon Security Engine (SEC) version 2 controller
* Boot the system
* 'dmesg | grep "fail to alloc uacce"' and you will see complains from hisi_sec2. For example: [ 27.015484] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)

[Where problems could occur]
* The regression can be considered as low, since it is HiSilicon crypto system specific

====== Original Bug Description ======

[Bug Description]

ubuntu 20.04.2 boot system and fail to alloc uacce (-22)

[Steps to Reproduce]
1) boot ubuntu 20.04.2 system
2) dmesg | grep fail

[Actual Results]
[ 27.000086] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.006515] hisi_sec2 0000:b6:00.0: Failed to enable PASID
[ 27.012043] hisi_sec2 0000:b6:00.0: Adding to iommu group 45
[ 27.015484] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)

[Expected Results]
no fail
[Reproducibility]
100%

[Additional information]
(Firmware version, kernel version, affected hardware, etc. if required):

[Resolution]
this following patches will solve this bug.

commit f8408d2b79b834f79b6c578817e84f74a85d2190
Author: Kai Ye <email address hidden>
Date: Tue Jan 5 14:16:42 2021 +0800

    crypto: hisilicon - add ZIP device using mode parameter

    Add 'uacce_mode' parameter for ZIP, which can be set as 0(default) or 1.
    '0' means ZIP is only registered to kernel crypto, and '1' means it's
    registered to both kernel crypto and UACCE.

    Signed-off-by: Kai Ye <email address hidden>
    Reviewed-by: Zhou Wang <email address hidden>
    Reviewed-by: Zaibo Xu <email address hidden>
    Signed-off-by: Herbert Xu <email address hidden>

commit bedd04e4aa1434d2f0f038e15bb6c48ac36876e1
Author: Kai Ye <email address hidden>
Date: Tue Jan 5 14:16:43 2021 +0800

    crypto: hisilicon/hpre - register HPRE device to uacce

commit 34932a6033be3c0088935c334e4dc5ad43dcb0cc
Author: Kai Ye <email address hidden>
Date: Tue Jan 5 14:16:44 2021 +0800

    crypto: hisilicon/sec - register SEC device to uacce

[Taihsiang Ho (tai271828)]

Patch f8408d2b79b834f79b6c578817e84f74a85d2190 :
    - shows up since upstream v5.12-rc1-dontuse tag
    - ubuntu-focal hwe 5.11 kernel tree could cherry-pick this patch cleanly.

Next:
    - Verify and nail down the reproducer

[Taihsiang Ho (tai271828)]

Successfully reproduced. Reproducing rate: 1 out of 1.[1]
btw focal GA kernel does not has such error messages.[2]

[1]

ubuntu@scobee:~$ uname -a; dmesg | grep fail
Linux scobee 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:13:01 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
[ 8.289936] cma: cma_alloc: alloc failed, req-size: 4096 pages, ret: -12
[ 8.346853] acpi PNP0A08:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.348502] acpi PNP0A08:02: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.350243] acpi PNP0A08:03: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.352192] acpi PNP0A08:04: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.354245] acpi PNP0A08:05: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.364407] acpi PNP0A08:07: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.366055] acpi PNP0A08:08: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.367616] acpi PNP0A08:09: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.369603] acpi PNP0A08:0a: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 8.371717] acpi PNP0A08:0b: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 18.189114] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.202784] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.216457] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.244041] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.257710] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.271383] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.298846] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.312514] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.326179] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.353636] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 21.256047] dw_spi_mmio HISI0173:00: DMA init failed
[ 21.267497] dw_spi_mmio HISI0173:00: DMA init failed
[ 23.595842] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 25.859215] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 32.804988] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.833621] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.909952] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.916375] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.934443] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 32.935833] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.936277] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 32.947732] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.955618] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 32.959619] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.960888] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 32.972760] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 32.977988] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 32.992290] arm_spe_pmu: probe of arm,spe-v1 failed with error -1
[ 32.993764] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
ubuntu@scobee:~$ lsb_release -a
No LSB modu...

[Taihsiang Ho (tai271828)]

Next step:
    - verify the reproducer with more trial to nail down the reproducing rate
    - build the focal hwe kernel with the upsteam patch f8408d2b79b834f79b6c578817e84f74a85d2190

[Taihsiang Ho (tai271828)]

- My current reproducing rate is 3 out of 3.
- The focal hwe kernel cherry-picks the upstream patch did not resolve the error message. The tree could be found here https://code.launchpad.net/~taihsiangho/ubuntu/+source/linux/+git/focal/+ref/hwe-5.11-lp1933301-crypto

Next:
    - revisit the kernel tree, the patch and the reproducer.

[Taihsiang Ho (tai271828)]

Ah, I missed some more patches described in the bug report. Let me cherry-pick and rebuild the kernel again.

Additionally, sometimes cma messages do not really mean something is wrong https://bugs.launchpad.net/kunpeng920/+bug/1878520/comments/2 let's keep in mind this point of view.

[Taihsiang Ho (tai271828)]

The patches could be clean cherry-picked with focal hwe 5.11 kernel tree. However, by merging the patches do not resolve the error messages[1]. If you are interested in the kernel deb merged the patches and you may download the kernel via https://people.canonical.com/~taihsiang/lp1933301/deb-03+lp1933301tai3.tar.gz (untar and dpkg -i *.deb to install the kernel)

[1]
[ 33.408534] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 33.429453] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 33.435405] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)

[Taihsiang Ho (tai271828)]

Additional information of comment#6 : the corresponding modules should be loaded.

ubuntu@scobee:~$ lsmod | grep -e uacce -e zip -e hpre -e sec
hisi_hpre 36864 0
hisi_zip 36864 0
hisi_sec2 53248 0
hisi_qm 69632 3 hisi_sec2,hisi_zip,hisi_hpre
authenc 16384 1 hisi_sec2
uacce 20480 4 hisi_sec2,hisi_qm,hisi_zip,hisi_hpre

ubuntu@scobee:~$ lsb_release -a ; uname -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
Linux scobee 5.11.0-22-generic #23~20.04.1+lp1933301tai3 SMP Thu Jul 8 12:38:12 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

[Taihsiang Ho (tai271828)]

kreiken could also reproduce this issue with focal hwe.

ubuntu@kreiken:~$ dmesg | grep fail | grep hisi
[ 35.171762] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 35.188874] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 35.212729] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 35.221600] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 35.230007] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 35.251963] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
ubuntu@kreiken:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
ubuntu@kreiken:~$ uname -a
Linux kreiken 5.8.0-63-generic #71~20.04.1-Ubuntu SMP Thu Jul 15 17:46:44 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@kreiken:~$

[Taihsiang Ho (tai271828)]

The mainline 5.13.7 could still reproduce this issue. You may download the pre-built mainline 5.13.7 deb from here https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.13.7/

===

ubuntu@kreiken:~$ sudo dmesg | grep fail | grep hisi
[ 40.093841] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 40.105529] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 40.119650] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 40.138267] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 40.204472] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 40.234359] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
ubuntu@kreiken:~$ uname -a ; lsb_release -a
Linux kreiken 5.13.7-051307-generic #202107310732 SMP Sat Jul 31 07:54:56 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
ubuntu@kreiken:~$

[Taihsiang Ho (tai271828)]

Both the mainline v5.12 and v5.13 could reproduce the issue[1].

[1]

ubuntu@kreiken:~$ uname -a; lsb_release -a; sudo dmesg | grep fail | grep hisi
Linux kreiken 5.12.0-051200-generic #202104252130 SMP Sun Apr 25 21:33:16 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
[ 36.399996] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 36.437650] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 36.439113] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 36.467654] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 36.482807] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 36.531349] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
ubuntu@kreiken:~$

ubuntu@kreiken:~$ uname -a; lsb_release -a; sudo dmesg | grep fail | grep hisi
Linux kreiken 5.13.0-051300-generic #202106272333 SMP Sun Jun 27 23:36:23 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
[ 35.497954] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 35.543293] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 36.057044] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 36.058494] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 36.093783] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 36.108451] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
ubuntu@kreiken:~$

[Taihsiang Ho (tai271828)]

Please note the three commits show up in the upstream mainline kernel tree since v5.12 (v5.12-rc1-dontuse).

[Taihsiang Ho (tai271828)]

@Fred

The commits you provided do not resolve the issue. May you confirm that they are the target commits, or anything else we need?

[Fred Kimmy (kongzizaixian)]

this following patchset will slove this issue, can you varity it?

https://lkml.org/lkml/2021/7/12/3516

[Taihsiang Ho (tai271828)]

Next step:
1. tai can try the patch https://lkml.org/lkml/2021/7/12/3516
2. Fred will help the patches to be upstreamed.

[Taihsiang Ho (tai271828)]

Patches suggested in comment#15 are upstreamed since v5.15-rc1 . They are

- 8c09e896cef8d908dd9a20a9f2a5c3fcb9799de3
- 8304a3a199eeb3e6434fc855ac072e7374d0c490

in the upstream kernel tree.

[Taihsiang Ho (tai271828)]

The issue could be reproduced with the current hwe kernel, and the current focal hwe kernel tree could cherry-pick the commits in comment#17 cleanly.

Next: build and verify the cherry-picked kernel on kreiken

[1]

ubuntu@kreiken:~$ uname -a; lsb_release -a; lsmod | grep -e uacce -e zip -e hpre -e sec; sudo dmesg | grep fail | grep hisi
Linux kreiken 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:46 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
hisi_zip 32768 0
hisi_sec2 53248 0
hisi_hpre 32768 0
hisi_qm 69632 3 hisi_sec2,hisi_zip,hisi_hpre
uacce 20480 2 hisi_qm,hisi_zip
authenc 16384 1 hisi_sec2
[ 34.922091] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 34.936231] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 34.966511] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 34.970904] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 34.984718] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 35.012844] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)

[Taihsiang Ho (tai271828)]

The kernel built based on the latest focal-hwe with the patches in comment#17 seems NOT to resolve the error messages.

ubuntu@kreiken:~$ uname -a; lsb_release -a; lsmod | grep -e uacce -e zip -e hpre -e sec; sudo dmesg | grep fail | grep hisi
Linux kreiken 5.11.0-38-generic #42~20.04.1+lp1933301tai1 SMP Tue Oct 12 05:33:29 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
hisi_hpre 32768 0
hisi_zip 32768 0
hisi_sec2 53248 0
hisi_qm 69632 3 hisi_sec2,hisi_zip,hisi_hpre
authenc 16384 1 hisi_sec2
uacce 20480 2 hisi_qm,hisi_zip
[ 35.616700] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 35.652961] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 35.677191] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 35.687192] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 35.706518] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 35.708386] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)

[Taihsiang Ho (tai271828)]

@Fred

Comparing the kernel with/without the patches[1][2], some error messages still present and some didn't. I attached the syslog for more details. Fred, would you mind verifying if the syslog delivers what you want, or some more patches to resolve the remaining error messages are expected?

[1] without the patches in comment#17

ubuntu@kreiken:~$ dmesg | grep hisi_sec2 ; uname -a
[ 35.291450] hisi_sec2 0000:76:00.0: Failed to enable PASID
[ 35.347369] hisi_sec2 0000:76:00.0: Adding to iommu group 53
[ 35.349487] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 35.349501] hisi_sec2 0000:76:00.0: enabling device (0000 -> 0002)
[ 35.369709] hisi_sec2 0000:76:00.0: SMMU Opened, the iommu type = 3
[ 35.377772] hisi_sec2 0000:b6:00.0: Failed to enable PASID
[ 35.384833] hisi_sec2 0000:b6:00.0: Adding to iommu group 56
[ 35.387016] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 35.387033] hisi_sec2 0000:b6:00.0: enabling device (0000 -> 0002)
[ 35.402929] hisi_sec2 0000:b6:00.0: SMMU Opened, the iommu type = 3
Linux kreiken 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:46 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

[2] with the patches in comment#17

ubuntu@kreiken:~$ dmesg | grep hisi_sec2 ; uname -a
[ 35.755052] hisi_sec2 0000:76:00.0: Adding to iommu group 51
[ 35.762616] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 35.762647] hisi_sec2 0000:76:00.0: enabling device (0000 -> 0002)
[ 35.772899] hisi_sec2 0000:76:00.0: SMMU Opened, the iommu type = 3
[ 35.781704] hisi_sec2 0000:b6:00.0: Adding to iommu group 52
[ 35.789265] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 35.789295] hisi_sec2 0000:b6:00.0: enabling device (0000 -> 0002)
[ 35.799282] hisi_sec2 0000:b6:00.0: SMMU Opened, the iommu type = 3
Linux kreiken 5.11.0-38-generic #42~20.04.1+lp1933301tai1 SMP Tue Oct 12 05:33:29 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

[Taihsiang Ho (tai271828)]

Confirmed with Fred. "fail to alloc uacce (-22)" is indeed what we want to resolve rather than "Failed to enable PASID".

Next step:
Both tai and Fred will try the latest upsteam kernel to verify if the patches in comment#17 landed in v5.15-rc indeed fixes "fail to alloc uacce (-22)".

[Taihsiang Ho (tai271828)]

Mainline v5.15rc5 does not resolve "fail to alloc uacce".

ubuntu@kreiken:~$ uname -a; lsb_release -a; lsmod | grep -e uacce -e zip -e hpre -e sec; sudo dmesg | grep fail | grep hisi
Linux kreiken 5.15.0-051500rc5-generic #202110102131 SMP Mon Oct 11 01:34:08 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
hisi_hpre 53248 0
hisi_zip 36864 0
ecdh_generic 16384 1 hisi_hpre
libcurve25519_generic 40960 1 hisi_hpre
hisi_sec2 61440 0
ecc 36864 2 ecdh_generic,hisi_hpre
hisi_qm 94208 3 hisi_sec2,hisi_zip,hisi_hpre
authenc 16384 1 hisi_sec2
uacce 20480 4 hisi_sec2,hisi_qm,hisi_zip,hisi_hpre
[ 41.307508] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 41.318091] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 41.392228] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 41.397621] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 41.399170] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 41.403381] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)

[Andrew Cloke (andrew-cloke)]

Marking as "low" as this does not cause any data loss or kernel panic.

[Fred Kimmy (kongzizaixian)]

ubuntu 20.04.3 boot fail log
root@ubuntu:~# dmesg | grep -i Fail
[ 13.044886] acpi PNP0A08:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 13.131546] acpi PNP0A08:02: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 13.261251] acpi PNP0A08:03: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 13.440636] acpi PNP0A08:04: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 13.916500] acpi PNP0A08:05: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 14.621801] acpi PNP0A08:07: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 14.709989] acpi PNP0A08:08: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 14.843923] acpi PNP0A08:09: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 15.028596] acpi PNP0A08:0a: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 15.247959] acpi PNP0A08:0b: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 16.770964] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.798512] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.812220] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.825929] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.853470] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.867198] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.880922] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.908516] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.922275] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 16.936032] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 18.759082] dw_spi_mmio HISI0173:00: DMA init failed
[ 21.849263] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 24.228987] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 27.376743] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.415382] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.416089] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.416118] hisi_hpre 0000:79:00.0: Failed to enable PASID
[ 27.421870] hisi_zip 0000:75:00.0: Failed to enable PASID
[ 27.428306] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.439204] hisi_sec2 0000:76:00.0: Failed to enable PASID
[ 27.477215] hisi_hpre 0000:79:00.0: fail to alloc uacce (-22)
[ 27.477536] hisi_zip 0000:75:00.0: fail to alloc uacce (-22)
[ 27.477580] hisi_sec2 0000:76:00.0: fail to alloc uacce (-22)
[ 27.483622] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.513425] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.519938] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.519967] hisi_hpre 0000:b9:00.0: Failed to enable PASID
[ 27.527736] hisi_zip 0000:b5:00.0: Failed to enable PASID
[ 27.559025] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.565507] hisi_sec2 0000:b6:00.0: Failed to enable PASID
[ 27.618338] hisi_hpre 0000:b9:00.0: fail to alloc uacce (-22)
[ 27.628473] hisi_zip 0000:b5:00.0: fail to alloc uacce (-22)
[ 27.628665] hisi_sec2 0000:b6:00.0: fail to alloc uacce (-22)
[ 27.710207] arm_spe_pmu: probe of arm,spe-v1 failed with error -1
root@ubuntu:~# uname ...

[Fred Kimmy (kongzizaixian)]

linus kernel 5.16rc3 version fix this bug.
root@ubuntu:~# dmesg | grep fail
[ 18.616768] dw_spi_mmio HISI0173:00: DMA init failed
[ 20.463886] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 23.128220] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 33.571242] arm_spe_pmu: probe of arm,spe-v1 failed with error -1
root@ubuntu:~# uname -a
Linux ubuntu 5.16.0-rc3-acc+ #3 SMP Tue Dec 7 11:42:57 CST 2021 aarch64 aarch64 aarch64 GNU/Linux

refer to this following linus kernel patch:

8c09e896cef8 PCI: Allow PASID on fake PCIe devices without TLP prefixes
8304a3a199ee PCI: Set dma-can-stall for HiSilicon chips
8de8d4fe7d5a crypto: hisilicon/qm - fix the uacce mmap failed
8bb765271ade crypto: hisilicon/qm - support the userspace task resetting
183b60e00597 crypto: hisilicon/qm - modify the uacce mode check

[Taihsiang Ho (tai271828)]

Thanks Fred! Let me try to merge them.

[Taihsiang Ho (tai271828)]

(elaboration in an audio call) xinwei: "cma_alloc" error seems only caused by Ubuntu kernel, and irreproducible with “newer” ubuntu kernel

[Taihsiang Ho (tai271828)]

This error message[1] is suggested in comment#27 to have a further investigation.

[1]
[ 27.376743] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.415382] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 27.416089] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12

[dann frazier (dannf)]

Note that cma_alloc errors are usually not critical. The path that allocates memory w/ cma_alloc will fall back and allocate memory a different way. You can bump up available cma memory on the kernel command line to avoid those messages, and confirm if the other issues persist.

[Taihsiang Ho (tai271828)]

Firstly let me reproduce the issue again with maas ubuntu focal hwe:

ubuntu@saenger:~$ dmesg | grep -i Fail
[ 3.413183] cma: cma_alloc: alloc failed, req-size: 4096 pages, ret: -12
[ 13.689928] dw_spi_mmio HISI0173:00: DMA init failed
[ 13.742613] cma: cma_alloc: alloc failed, req-size: 24 pages, ret: -12
[ 13.754442] cma: cma_alloc: alloc failed, req-size: 24 pages, ret: -12
[ 13.772834] cma: cma_alloc: alloc failed, req-size: 4 pages, ret: -12
[ 13.785092] cma: cma_alloc: alloc failed, req-size: 2 pages, ret: -12
[ 13.798934] cma: cma_alloc: alloc failed, req-size: 2 pages, ret: -12
[ 13.808322] cma: cma_alloc: alloc failed, req-size: 256 pages, ret: -12
[ 13.889647] cma: cma_alloc: alloc failed, req-size: 16 pages, ret: -12
[ 13.913513] cma: cma_alloc: alloc failed, req-size: 255 pages, ret: -12
[ 13.932184] cma: cma_alloc: alloc failed, req-size: 255 pages, ret: -12
[ 13.945286] cma: cma_alloc: alloc failed, req-size: 255 pages, ret: -12
[ 27.249874] hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)
[ 27.250061] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.256601] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.263134] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.269698] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.276230] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.282785] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.289317] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.295871] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.302402] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.308955] cma: cma_alloc: alloc failed, req-size: 36 pages, ret: -12
[ 27.319751] hisi_sec2 0000:b5:00.0: fail to alloc uacce (-22)
ubuntu@saenger:~$ uname -a; lsb_release -a
Linux saenger 5.11.0-41-generic #45~20.04.1-Ubuntu SMP Wed Nov 10 10:22:58 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[Taihsiang Ho (tai271828)]

Please note 21.04 is reaching EOL soon a.k.a. on Jan 20 2022. I may investigate how 5.13 hwe kernel goes as well.

[Taihsiang Ho (tai271828)]

Ubuntu 21.04 (Hirsute Hippo) reaches End of Life on January 20 2022

https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html

[Taihsiang Ho (tai271828)]

The following commits fail to have clean cherry-pick in:

5.11 focal hwe tree:
183b60e005975d3c84c22199ca64a9221e620fb6 crypto: hisilicon/qm - modify the uacce mode check
8bb765271aded24ca724a39701c6e686234c7020 crypto: hisilicon/qm - support the userspace task resetting

5.13 focal hwe tree:
8bb765271aded24ca724a39701c6e686234c7020 crypto: hisilicon/qm - support the userspace task resetting

Next step:
Having a look of these failed patches and see if they relates to our goal.

[Taihsiang Ho (tai271828)]

- The latest Focal HWE kernel, Focal hwe 5.11.0-43, is still able to reproduce this issue
- The mainline kernel 5.16.0rc5 does not reproduce the error message[1]

[1]

ubuntu@segers:~$ sudo dmesg | grep -i fail
[ 13.825973] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.274444] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.650976] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 19.682985] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.058964] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
ubuntu@segers:~$ uname -a
Linux segers 5.16.0-051600rc5-generic #202112121931 SMP Mon Dec 13 00:35:59 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

[Taihsiang Ho (tai271828)]

With the following patches, we can resolve the error message "hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)" with segers 5.13 hwe tree[1]:

055c66e6fcba crypto: hisilicon/qm - modify the uacce mode check
ac7fd87d4869 crypto: hisilicon/qm - fix the uacce mmap failed
0ecad886c74e PCI: Set dma-can-stall for HiSilicon chips
49a6a1a89ad0 PCI: Allow PASID on fake PCIe devices without TLP prefixes

Next step:
Having a look of these patches to pick up the ones that really fix the issue.

[1]
ubuntu@segers:~$ cat collect.sh
uname -a
lsb_release -a
echo
dmesg | grep -i fail
echo

echo
echo ==================
dmesg | grep "fail to alloc uacce"
echo ==================

ubuntu@segers:~$ cat 5.13.0-22-d20211217t025549~055c66e6fcba-RESOLVED.log
Linux segers 5.13.0-22-generic #22~20.04.1~d20211217t025549~055c66e6fcba SMP Fri Dec 17 02:56:4 aarch64 aarch64 aarch64 GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[ 3.424477] cma: cma_alloc: reserved: alloc failed, req-size: 4096 pages, ret: -12
[ 11.595409] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.609990] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.624567] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.653068] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.667647] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.682225] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.710601] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.725173] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.739743] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.768137] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 13.942749] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.489486] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.863784] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 19.895829] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.259751] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 22.877577] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 22.885394] cma: cma_alloc: reserved: alloc failed, req-size: 16 pages, ret: -12
[ 22.902554] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 22.910275] cma: cma_alloc: reserved: alloc failed, req-size: 16 pages, ret: -12

[Taihsiang Ho (tai271828)]

Succeeding comment#29 and comment#36, and enlarging cma size larger e.g. 1G, could remove more error messages[1].

[1] append cma=1G in the kernel cmdline
ubuntu@segers:~$ dmesg | grep -i cma
[ 0.000000] cma: Failed to reserve 1024 MiB
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.11.0-43-generic root=UUID=e70c736e-0fc9-4459-9e23-d18d2fafb1bb ro sysrq_always_enabled cma=1G
[ 0.000000] Memory: 131355008K/134213440K available (15744K kernel code, 3572K rwdata, 13344K rodata, 7744K init, 1147K bss, 2858432K reserved, 0K cma-reserved)
ubuntu@segers:~$ dmesg | grep -i fail
[ 0.000000] cma: Failed to reserve 1024 MiB
[ 14.105493] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.733645] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.998866] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 21.050609] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 21.411963] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 23.600179] hisi_sec2 0000:75:00.0: Failed to enable PASID
[ 23.606891] hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)
[ 23.620062] hisi_sec2 0000:b5:00.0: Failed to enable PASID
[ 23.628695] hisi_sec2 0000:b5:00.0: fail to alloc uacce (-22)
ubuntu@segers:~$ uname -a
Linux segers 5.11.0-43-generic #47~20.04.2-Ubuntu SMP Mon Dec 13 11:10:13 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@segers:~$

[Taihsiang Ho (tai271828)]

5.13 hwe with cma=128M and patch 3b7e7c389812 looks good[1].

[1]

ubuntu@segers:~$ uname -a; dmesg | grep -i fail ; echo; dmesg | grep -i cma
Linux segers 5.13.0-22-generic #22~20.04.1~d20220104t205635~3b7e7c389812 SMP Tue Jan 4 20:57:29 aarch64 aarch64 aarch64 GNU/Linux
[ 13.835200] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.189114] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.551451] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 19.583481] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.962821] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 22.561053] hisi_sec2 0000:75:00.0: Failed to enable PASID
[ 22.588807] hisi_sec2 0000:b5:00.0: Failed to enable PASID

[ 0.000000] cma: Reserved 128 MiB at 0x0000000078000000
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.13.0-22-generic root=UUID=e70c736e-0fc9-4459-9e23-d18d2fafb1bb ro sysrq_always_enabled cma=128M
[ 0.000000] Memory: 131221096K/134213440K available (15744K kernel code, 3654K rwdata, 13604K rodata, 7936K init, 1153K bss, 2861272K reserved, 131072K cma-reserved)

[Taihsiang Ho (tai271828)]

5.13 focal hwe kernel tree With the following patches and cma parameters, we can remove all error messages we did not like:

- 183b60e00597 crypto: hisilicon/qm - modify the uacce mode check
- 8c09e896cef8 PCI: Allow PASID on fake PCIe devices without TLP prefixes
- cma=128M

The corresponding messages:
ubuntu@segers:~$ cat collect.sh
uname -a
lsb_release -a

echo
dmesg | grep -i cma
echo

echo
dmesg | grep -i fail
echo

echo
echo ==================
dmesg | grep "fail to alloc uacce"
echo ==================

ubuntu@segers:~$ ./collect.sh
Linux segers 5.13.0-22-generic #22~20.04.1~d20220106t133928~c8dfe75d4593 SMP Thu Jan 6 13:40:22 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[ 0.000000] cma: Reserved 128 MiB at 0x0000000078000000
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.13.0-22-generic root=UUID=e70c736e-0fc9-4459-9e23-d18d2fafb1bb ro sysrq_always_enabled cma=128M
[ 0.000000] Memory: 131221092K/134213440K available (15744K kernel code, 3654K rwdata, 13604K rodata, 7936K init, 1153K bss, 2861276K reserved, 131072K cma-reserved)

[ 13.827504] dw_spi_mmio HISI0173:00: DMA init failed
[ 13.856392] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.429931] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.792187] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 19.824213] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.190966] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1

==================
==================
ubuntu@segers:~$

[Taihsiang Ho (tai271828)]

With 5.11 focal hwe kernel tree, "183b60e00597 crypto: hisilicon/qm - modify the uacce mode check" could not be cherry-picked cleanly.

However, with the following patch and conditions, we can remove some more error messages:

- 8c09e896cef8 PCI: Allow PASID on fake PCIe devices without TLP prefixes
- cma=128M

ubuntu@segers:~$ cat collect.sh
uname -a
lsb_release -a

echo
dmesg | grep -i fail
echo

echo
echo ==================
dmesg | grep -i cma
echo ==================
echo

echo
echo ==================
dmesg | grep "fail to alloc uacce"
echo ==================
echo

echo
echo ==================
dmesg | grep -i PASID
echo ==================
echo

ubuntu@segers:~$ ./collect.sh
Linux segers 5.11.0-44-generic #48~20.04.2~d20220106t152459~458ca06c1773 SMP Thu Jan 6 15:25:51 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[ 14.109315] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.707211] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.069487] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 20.111577] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.473453] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 23.168785] hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)
[ 23.180802] hisi_sec2 0000:b5:00.0: fail to alloc uacce (-22)

==================
[ 0.000000] cma: Reserved 128 MiB at 0x0000000078000000
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.11.0-44-generic root=UUID=e70c736e-0fc9-4459-9e23-d18d2fafb1bb ro sysrq_always_enabled cma=128M
[ 0.000000] Memory: 131223928K/134213440K available (15744K kernel code, 3572K rwdata, 13344K rodata, 7744K init, 1147K bss, 2858440K reserved, 131072K cma-reserved)
==================

==================
[ 23.168785] hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)
[ 23.180802] hisi_sec2 0000:b5:00.0: fail to alloc uacce (-22)
==================

==================
==================

ubuntu@segers:~$

[Taihsiang Ho (tai271828)]

@Fred, are you ok if we only submit/merge the patch of the commit 183b60e00597 (crypto: hisilicon/qm - modify the uacce mode check) to resolve "fail to alloc uacce" with impish (5.13). Do you think it makes sense for you?

In my opinion, the EOL of 21.04 (5.11) is coming soon (see comment31), and 5.11 does not manage to have clean cherry-pick of the patch 183b60e00597 (see comment33). Submitting the patch to 5.11 does not help that much for this issue with 20.04 hwe kernel.

Besides, I checked the other patches in comment25, and it seems that they are irrelevant to the removal of "fail to alloc uacce" error message (even more, some of them could not be cherry-picked cleanly) (please correct me if I am wrong). The kernel team may decline our patch if they are confused and question about irrelevant patches. Thus I would prefer to merge only 183b60e00597 to move forward faster and more smoothly.

Next step:
Prepare SRU for the commit 183b60e00597 with impish (5.13) if Fred is happy with the approach.

[Taihsiang Ho (tai271828)]

Jammy will release a few months later and we are aware of the development activities of Jammy 5.15 kernel. The patch needs to land Jammy as well to prevent user's dist-upgrade regression.

My investigation of the patch, 183b60e00597 crypto: hisilicon/qm - modify the uacce mode check, has clean cherry-pick with Jammy 5.15 master-next, and fixes the issue[1].

Next:
Process to merge 183b60e00597 with Jammy 5.15 kernel along with impish 5.13 (for up-coming focal-hwe)

[1] Before applying the patch (master-next branch, 5.15.13):

ubuntu@segers:~$ sudo ./collect.sh
Linux segers 5.15.0-16-generic #16~d20220111t131721~7ec137fc2603 SMP Tue Jan 11 13:18:16 UTC 20 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[ 3.408872] cma: cma_alloc: reserved: alloc failed, req-size: 4096 pages, ret: -12
[ 11.594241] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.636486] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.678612] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.720721] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.762846] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.805032] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.847216] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.889401] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.970515] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 12.067413] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 13.974371] dw_spi_mmio HISI0173:00: DMA init failed
[ 19.448332] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 19.820834] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 19.852904] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.224761] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[ 22.956763] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 22.966698] cma: cma_alloc: reserved: alloc failed, req-size: 16 pages, ret: -12
[ 22.982319] hisi_sec2 0000:75:00.0: fail to alloc uacce (-22)
[ 22.990846] cma: cma_alloc: reserved: alloc failed, req-size: 4 pages, ret: -12
[ 22.999914] cma: cma_alloc: reserved: alloc failed, req-size: 16 pages, ret: -12
[ 23.010710] hisi_sec2 0000:b5:00.0: fail to alloc uacce (-22)

==================
[ 0.000000] cma: Reserved 32 MiB at 0x000000007e000000
[ 0.000000] Memory: 131307604K/134213440K available (17280K kernel code, 4572K rwdata, 13820K rodata, 9600K init, 1182K bss, 2873068K reserved, 32768K cma-reserved)
[ 3.408872] cma: cma_alloc: reserved: alloc failed, req-size: 4096 pages, ret: -12
[ 11.594241] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.636486] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.678612] cma: cma_alloc: reserved: alloc failed, req-size: 256 pages, ret: -12
[ 11.720721] cma: cma_alloc: reserved: alloc fai...

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1933301

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Taihsiang Ho (tai271828)]

Submitted the patch to the kernel team mailing list https://lists.ubuntu.com/archives/kernel-team/2022-January/127109.html for both Impish and Jammy.

[Taihsiang Ho (tai271828)]

The patch is applied https://lists.ubuntu.com/archives/kernel-team/2022-January/127138.html

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the linux/5.13.0-29.32 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-impish' to 'verification-done-impish'. If the problem still exists, change the tag 'verification-needed-impish' to 'verification-failed-impish'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Taihsiang Ho (tai271828)]

impish 5.13.0-30 verification passed.

ubuntu@segers:~$ uname -a; lsb_release -a; sudo dmesg | grep "fail to alloc uacce"
Linux segers 5.13.0-30-generic #33-Ubuntu SMP Fri Feb 4 17:05:14 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 21.10
Release: 21.10
Codename: impish

[Taihsiang Ho (tai271828)]

Kernel re-spun. 5.13.0-30 is expected to be released to -updates by 2022/Feb/17[1].

[1] https://lists.ubuntu.com/archives/kernel-sru-announce/2022-February/000198.html

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the linux-gcp-5.13/5.13.0-1015.18~20.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Taihsiang Ho (tai271828)]

Focal 5.13.0-30-generic #33~20.04.1-Ubuntu verification passed.

ubuntu@segers:~$ uname -a; lsb_release -a; sudo dmesg | grep "fail to alloc uacce"
Linux segers 5.13.0-30-generic #33~20.04.1-Ubuntu SMP Mon Feb 7 14:28:43 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.13.0-30.33

---------------
linux (5.13.0-30.33) impish; urgency=medium

  * impish/linux: 5.13.0-30.33 -proposed tracker (LP: #1960055)

  * systemd/248.3-1ubuntu8.2 ADT test failure with linux/5.13.0-29.32
    (LP: #1960034)
    - Revert "block: avoid to quiesce queue in elevator_init_mq"
    - Revert "blk-mq: cancel blk-mq dispatch work in both blk_cleanup_queue and
      disk_release()"

linux (5.13.0-29.32) impish; urgency=medium

  * impish/linux: 5.13.0-29.32 -proposed tracker (LP: #1959238)

  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/2022.01.31)

  * CVE-2022-22942
    - SAUCE: drm/vmwgfx: Fix stale file descriptors on failed usercopy

  * CVE-2022-0330
    - drm/i915: Flush TLBs before releasing backing store

  * Impish update: upstream stable patchset 2022-01-05 (LP: #1956508)
    - ACPI: Get acpi_device's parent from the parent field
    - USB: serial: option: add Telit LE910S1 0x9200 composition
    - USB: serial: option: add Fibocom FM101-GL variants
    - usb: dwc2: gadget: Fix ISOC flow for elapsed frames
    - usb: dwc2: hcd_queue: Fix use of floating point literal
    - usb: dwc3: gadget: Ignore NoStream after End Transfer
    - usb: dwc3: gadget: Check for L1/L2/U3 for Start Transfer
    - usb: dwc3: gadget: Fix null pointer exception
    - net: nexthop: fix null pointer dereference when IPv6 is not enabled
    - usb: chipidea: ci_hdrc_imx: fix potential error pointer dereference in probe
    - usb: typec: fusb302: Fix masking of comparator and bc_lvl interrupts
    - usb: hub: Fix usb enumeration issue due to address0 race
    - usb: hub: Fix locking issues with address0_mutex
    - binder: fix test regression due to sender_euid change
    - ALSA: ctxfi: Fix out-of-range access
    - ALSA: hda/realtek: Add quirk for ASRock NUC Box 1100
    - ALSA: hda/realtek: Fix LED on HP ProBook 435 G7
    - media: cec: copy sequence field for the reply
    - Revert "parisc: Fix backtrace to always include init funtion names"
    - HID: wacom: Use "Confidence" flag to prevent reporting invalid contacts
    - staging/fbtft: Fix backlight
    - staging: greybus: Add missing rwsem around snd_ctl_remove() calls
    - staging: rtl8192e: Fix use after free in _rtl92e_pci_disconnect()
    - fuse: release pipe buf after last use
    - xen: don't continue xenstore initialization in case of errors
    - xen: detect uninitialized xenbus in xenbus_init
    - KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB
    - tracing/uprobe: Fix uprobe_perf_open probes iteration
    - tracing: Fix pid filtering when triggers are attached
    - mmc: sdhci-esdhc-imx: disable CMDQ support
    - mmc: sdhci: Fix ADMA for PAGE_SIZE >= 64KiB
    - mdio: aspeed: Fix "Link is Down" issue
    - powerpc/32: Fix hardlockup on vmap stack overflow
    - PCI: aardvark: Deduplicate code in advk_pcie_rd_conf()
    - PCI: aardvark: Implement re-issuing config requests on CRS response
    - PCI: aardvark: Simplify initialization of rootcap on virtual bridge
    - PCI: aardvark: Fix link training
    - proc/vmcore: fix clearing user buffer by properly using clear_user()
...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.15.0-22.22

---------------
linux (5.15.0-22.22) jammy; urgency=medium

  * jammy/linux: 5.15.0-22.22 -proposed tracker (LP: #1960290)

 -- Paolo Pisati <email address hidden> Tue, 08 Feb 2022 10:48:49 +0100