xfrm_policy.sh in net from ubuntu_kernel_selftests failed with "expected ping to .254 to fail" on Groovy/Hirsute

[Po-Hsu Lin]

Issue found on 5.8.0-1010.10-aws

Test failed with:
 # selftests: net: xfrm_policy.sh
 # PASS: policy before exception matches
 # FAIL: expected ping to .254 to fail (exceptions)
 # PASS: direct policy matches (exceptions)
 # PASS: policy matches (exceptions)
 # FAIL: expected ping to .254 to fail (exceptions and block policies)
 # PASS: direct policy matches (exceptions and block policies)
 # PASS: policy matches (exceptions and block policies)
 # FAIL: expected ping to .254 to fail (exceptions and block policies after hresh changes)
 # PASS: direct policy matches (exceptions and block policies after hresh changes)
 # PASS: policy matches (exceptions and block policies after hresh changes)
 # FAIL: expected ping to .254 to fail (exceptions and block policies after hthresh change in ns3)
 # PASS: direct policy matches (exceptions and block policies after hthresh change in ns3)
 # PASS: policy matches (exceptions and block policies after hthresh change in ns3)
 # FAIL: expected ping to .254 to fail (exceptions and block policies after htresh change to normal)
 # PASS: direct policy matches (exceptions and block policies after htresh change to normal)
 # PASS: policy matches (exceptions and block policies after htresh change to normal)
 # PASS: policies with repeated htresh change
 ok 12 selftests: net: xfrm_policy.sh

This is not a regression, it can be found in 5.8.0-1007.7-aws as well.

[Po-Hsu Lin (cypressyew)]

Oh I mis-read the test output, looks like it's FAIL as it's expected to FAIL,
perhaps this should be modified to PASS, as eventually this has failed and pass with the rule.

[Po-Hsu Lin (cypressyew)]

Re-open this, with the return value issue (bug 1909647) fixed, this is still failing on Groovy (5.8.0-49.55).

[Po-Hsu Lin (cypressyew)]

This issue can be found on Hirsute 5.11.0-13 as well.

[Guilherme G. Piccoli (gpiccoli)]

Another instance of this failure (detected on G/kvm) would be:

DEBUG| [stdout] # selftests: net: vrf-xfrm-tests.sh
DEBUG| [stdout] # Error: Unknown device type.
DEBUG| [stdout] # Cannot find device "red"
DEBUG| [stdout] # Error: argument "red" is wrong: Invalid VRF

DEBUG| [stdout] # netem qdisc on VRF device
DEBUG| [stdout] # TEST: IPv4 no xfrm policy [FAIL]
DEBUG| [stdout] # TEST: IPv6 no xfrm policy [FAIL]
DEBUG| [stdout] # TEST: IPv4 xfrm policy based on address [FAIL]
DEBUG| [stdout] # TEST: IPv6 xfrm policy based on address [FAIL]
DEBUG| [stdout] # TEST: IPv6 xfrm policy with VRF in selector [FAIL]
DEBUG| [stdout] # Error: Unknown device type.
DEBUG| [stdout] # Error: argument "red" is wrong: Not a valid VRF name
DEBUG| [stdout] #
DEBUG| [stdout] # Cannot find device "xfrm0"
DEBUG| [stdout] # Cannot find device "xfrm0"
DEBUG| [stdout] # TEST: IPv4 xfrm policy with xfrm device [FAIL]
DEBUG| [stdout] # TEST: IPv6 xfrm policy with xfrm device [FAIL]
DEBUG| [stdout] # Cannot find device "xfrm0"
DEBUG| [stdout] #
DEBUG| [stdout] # Tests passed: 0
DEBUG| [stdout] # Tests failed: 14
DEBUG| [stdout] not ok 41 selftests: net: vrf-xfrm-tests.sh # exit=1

[Guilherme G. Piccoli (gpiccoli)]

All of these issues seems to be related with missing Kconfigs on linux-kvm, for example:

# grep -nrIi vrf config-5.8.0-50-generic
2984:CONFIG_NET_VRF=m
# grep -nrIi vrf config-5.8.0-1026-kvm
1916:# CONFIG_NET_VRF is not set

# grep -nrIi xfrm_interface config-5.8.0-50-generic
1084:CONFIG_XFRM_INTERFACE=m
# grep -nrIi xfrm_interface config-5.8.0-1026-kvm
903:# CONFIG_XFRM_INTERFACE is not set

[Guilherme G. Piccoli (gpiccoli)]

Observed on G/KVM, cycle sru-20210510 .

[Guilherme G. Piccoli (gpiccoli)]

Observed on H/KVM, cycle sru-20210510 .

[Kelsey Steele (kelsey-steele)]

Also seen on Hirsute/linux 5.11.0-18.19

[Guilherme G. Piccoli (gpiccoli)]

Observed on G/KVM, cycle sru-20210531 .

[Kleber Sacilotto de Souza (kleber-souza)]

Found with hirsute/linux 5.11.0-24.25.

[Tim Gardner (timg-tpi)]

Seen on hirsute/linux-aws 5.11.0-1021.22

[Krzysztof Kozlowski (krzk)]

Found also on 2021.11.29/impish/linux-oracle/5.13.0-1011.13