performance degradation on BlueField-2 5.15.0-1044-bluefield

[William Tu]

* Explain the bug(s)

We run a simple CPU-only xor application.
test.c
------
int main(void) {
        long i;
        int count = 0;
        for (i=0; i< 10000000000; i++) {
                count = count ^ i;
        }
        return 0;
}
tested on BlueField-2 and found consistent performance degradation.

* Performance

install from debian kernel packages:
/auto/sw/integration/bluefield_build/kernels/Ubuntu22.04-kernels/*.deb

$ gcc test.c; time ./a.out

kernel 5.15.0-1042-bluefield: 39sec

kernel 5.15.0-1043-bluefield (1043): 39sec

kernel 5.15.0-1044-bluefield: 49 sec

kernel 5.15.0-1048-bluefield: 49 sec

* Performance-2
However, If we download the kernel source and compile, then we don't see the difference
/boot/vmlinuz-5.15.160 (1049): 49sec

/boot/vmlinuz-5.15.160-20576-gba1f73c15047 (1043): 49sec

also test other versions
 bf-1024
 bf-1030
 bf-1044
 bf-1049
1024, 1030, 1044, 1049 all shows the same performance, 49 sec

[William Tu (wtu)]

root@c-237-155-120-p88-00-0-bf2:/boot# diff config-5.15.0-1043-bluefield config-5.15.0-1044-bluefield
3c3
< # Linux/arm64 5.15.0-1043-bluefield Kernel Configuration
---
> # Linux/arm64 5.15.0-1044-bluefield Kernel Configuration
37c37
< CONFIG_VERSION_SIGNATURE="Ubuntu 5.15.0-1043.45-bluefield 5.15.149"
---
> CONFIG_VERSION_SIGNATURE="Ubuntu 5.15.0-1044.46-bluefield 5.15.152"
1663d1662
< CONFIG_NET_SCH_CBQ=m
1666d1664
< CONFIG_NET_SCH_ATM=m
1678d1675
< CONFIG_NET_SCH_DSMARK=m
2154a2152
> # CONFIG_PCI_EPF_VNTB is not set