Regression for net:vrf-xfrm-tests.sh with 5.15 kernel on ARM64 node scobee-kernel

Bug #2039816 reported by Po-Hsu Lin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
New
Undecided
Unassigned
linux (Ubuntu)
In Progress
Undecided
Ike Panhc
Jammy
In Progress
Undecided
Ike Panhc

Bug Description

Issue found on ARM64 node scobee-kernel with:
 * J-5.15.0-86.95 lowlatency
 * 5.15.0-85.95~20.04.2 generic
 * F-5.15.0-86.95~20.04.1 lowlatency
 * F-5.15.0-87.96~20.04.1 lowlatency-64k

Test failed with:
$ sudo ./vrf-xfrm-tests.sh

No qdisc on VRF device
TEST: IPv4 no xfrm policy [ OK ]
TEST: IPv6 no xfrm policy [ OK ]
TEST: IPv4 xfrm policy based on address [FAIL]
TEST: IPv6 xfrm policy based on address [FAIL]
TEST: IPv6 xfrm policy with VRF in selector [ OK ]
TEST: IPv4 xfrm policy with xfrm device [FAIL]
TEST: IPv6 xfrm policy with xfrm device [FAIL]

netem qdisc on VRF device
TEST: IPv4 no xfrm policy [ OK ]
TEST: IPv6 no xfrm policy [ OK ]
TEST: IPv4 xfrm policy based on address [FAIL]
TEST: IPv6 xfrm policy based on address [FAIL]
TEST: IPv6 xfrm policy with VRF in selector [ OK ]
TEST: IPv4 xfrm policy with xfrm device [FAIL]
TEST: IPv6 xfrm policy with xfrm device [FAIL]

Tests passed: 6
Tests failed: 8

And this issue does not exist with the following combination:
* F-generic-5.15.0-86.96~20.04.1 howzit-kernel
* F-generic-5.15.0-86.96~20.04.1 wright-kernel
* F-generic-64k-5.15.0-85.95~20.04.2 kopter-kernel
* F-lowlatency-5.15.0-88.98~20.04.1 howzit-kernel
* F-lowlatency-5.15.0-85.94 starmie-kernel
* F-lowlatency-64k-5.15.0-85.94~20.04.1 howzit-kernel
* J-lowlatency-64k-5.15.0-85.94 starmie-kernel
* J-lowlatency-64k-5.15.0-86.95 howzit-kernel

So it looks like this is hardware related.

And the cause seems to be commit cb43c60 (" selftests: net: vrf-xfrm-tests: change authentication and encryption algos"), which lands on the Jammy tree since:
 * Ubuntu-5.15.0-85.95
 * Ubuntu-lowlatency-5.15.0-85.94

With this commit reverted, this test can pass on node scobee-kernel with 5.15.0-87-lowlatency-64k

Po-Hsu Lin (cypressyew)
summary: - Regression for net:vrf-xfrm-tests.sh with F-hwe-5.15 lowlatency on
- scobee-kernel
+ Regression for net:vrf-xfrm-tests.sh with 5.15 kernel on scobee-kernel
summary: - Regression for net:vrf-xfrm-tests.sh with 5.15 kernel on scobee-kernel
+ Regression for net:vrf-xfrm-tests.sh with 5.15 kernel on ARM64 node
+ scobee-kernel
tags: added: 5.15 arm64 focal jammy ubuntu-kernel-selftests
description: updated
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

With 5.15.0-84-generic-64k on scobee-kernel, with test case patched with commit cb43c60 it is still failing. Without that commit the test will pass.

It shows this is related to commit cb43c60, not the kernel function itself.

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 2039816

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in linux (Ubuntu Jammy):
status: New → Incomplete
Ike Panhc (ikepanhc)
Changed in linux (Ubuntu Jammy):
assignee: nobody → Ike Panhc (ikepanhc)
Changed in linux (Ubuntu):
assignee: nobody → Ike Panhc (ikepanhc)
Revision history for this message
Ike Panhc (ikepanhc) wrote :

Interested. I can reproduce this issue with latest vrf-xfrm-tests.sh on 5.15.0-87.97 but can not reproduce with linux v6.3 vrf-xfrm-tests.sh.

This might be a testcase issue.

Revision history for this message
Ike Panhc (ikepanhc) wrote :

Checked with vrf-xfrm-tests.sh in mainline kernel. With 5.15.0-87.97 kernel v6.3 passes and v6.4 failed.

Revision history for this message
Ike Panhc (ikepanhc) wrote :

Between v6.3 and v6.4 some algorithms has been replaced.

commit cb43c60e64ca67fcc9d23bd08f51d2ab8209d9d7
Author: Magali Lemes <email address hidden>
Date: Tue Jun 13 09:32:21 2023 -0300

    selftests: net: vrf-xfrm-tests: change authentication and encryption algos

    The vrf-xfrm-tests tests use the hmac(md5) and cbc(des3_ede)
    algorithms for performing authentication and encryption, respectively.
    This causes the tests to fail when fips=1 is set, since these algorithms
    are not allowed in FIPS mode. Therefore, switch from hmac(md5) and
    cbc(des3_ede) to hmac(sha1) and cbc(aes), which are FIPS compliant.

Revision history for this message
Ike Panhc (ikepanhc) wrote :

Using 5.15.0-25.25 and the results are the same, vrf-xfrm-tests.sh in linux v6.3 passes and v6.4 fails. Good news is this is not because of any kernel upgrade, and the next step is to look if any function missing/broken in Ubuntu 5.15 kernels.

Revision history for this message
Ike Panhc (ikepanhc) wrote :

After `rmmod hisi_sec2`, 5.15.0-87.97 kernel passes all tests. In conclusion, after vrf-xfrm-tests algorithms switched since v6.4 kernel, it fails with hisi_sec2 loaded.

Revision history for this message
Ike Panhc (ikepanhc) wrote :

Try ‘s/sha1/md5’ and ‘s/aes/des3_ede’ on vrf-xfrm-tests.sh, either way it can passes. I will read more about this test and find out how it fails.

Bug against kunpeng920 is also filed. https://bugs.launchpad.net/kunpeng920/+bug/2043365

Changed in linux (Ubuntu Jammy):
status: Incomplete → In Progress
Changed in linux (Ubuntu):
status: Incomplete → In Progress
Revision history for this message
Magali Lemes do Sacramento (magalilemes) wrote :

This seems to be related to the way hisi_sec2 implements aead, specifically with authenc(hmac(sha1),cbc(aes)), authenc(hmac(sha256),cbc(aes)) and authenc(hmac(sha512),cbc(aes)). Other aead algorithms seem to work fine as they don't rely on this module.

Notice that net:xfrm_policy.sh also fails similarly on this node, as it also uses aes and sha1. See LP #2011414.
When running either net:vrf-xfrm-tests.sh or net:xfrm_policy.sh, we get the same output from dmesg:
`hisi_sec2 0000:76:00.0: flag[3], icv[2]`, showing that something is off with the integrity check value.

Po-Hsu Lin (cypressyew)
tags: added: sru-20231030
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.