Valgrind regression tests

[Pjfloyd]

Fisrst off, how to reproduce. I'm testing on a Pi 5.

The two patches below fix 4 testcase failures.

Firstly you will need a patched libarmmem

git clone https://github.com/bavison/arm-mem.git

Apply patch2 from https://bugs.launchpad.net/raspbian/+bug/2051392

Build Valgrind from source

git clone https://sourceware.org/git/valgrind.git

Apply the patch in https://bugs.kde.org/show_bug.cgi?id=398569

./autogen.sh
./configure --host=armv8-unknown-linux
make
make -i check

A few of the testcases fail to build, which is why -i was required. I've applied a few fixes upstream so this may not be necessary. No harm in leaving it.

Make sure that you have libc-dbg and libstdc++ debug (something like libstdc++6-12-dbg, depending on your version of GCC g++).

Use the debug version of libstdc++:
export LD_LIBRARY_PATH=/lib/arm-linux-gnueabihf/debug

Run the regression tests:
LD_PRELOAD=$HOME/scratch/arm-mem/libarmmem-v7l.so make -i regtest

(modify the above path to point to where you built libarmmem)

[Pjfloyd (pjfloyd)]

And here are the results (for reference, low single digits failures are what I'd consider normal).

== 720 tests, 30 stderr failures, 6 stdout failures, 2 stderrB failures, 2 stdoutB failures, 0 post failures ==
gdbserver_tests/mcblocklistsearch (stderrB)
gdbserver_tests/mcinfcallWSRU (stderrB)
gdbserver_tests/nlcontrolc (stdoutB)
gdbserver_tests/nlvgdbsigqueue (stdoutB)
memcheck/tests/atomic_incs (stdout)
memcheck/tests/atomic_incs (stderr)
memcheck/tests/dw4 (stderr)
memcheck/tests/leak-cases-full (stderr)
memcheck/tests/leak-cases-summary (stderr)
memcheck/tests/leak-cycle (stderr)
memcheck/tests/leak-segv-jmp (stderr)
memcheck/tests/leak-tree (stderr)
memcheck/tests/leak_cpp_interior (stderr)
memcheck/tests/lks (stderr)
memcheck/tests/origin5-bz2 (stderr)
memcheck/tests/partiallydefinedeq (stderr)
memcheck/tests/sem (stderr)
memcheck/tests/sh-mem-random (stdout)
memcheck/tests/sh-mem-random (stderr)
memcheck/tests/supp_unknown (stderr)
memcheck/tests/varinfo2 (stderr)
memcheck/tests/varinfo3 (stderr)
memcheck/tests/varinfo4 (stderr)
memcheck/tests/varinfo5 (stderr)
memcheck/tests/varinfo6 (stderr)
memcheck/tests/varinforestrict (stderr)
helgrind/tests/bug392331 (stderr)
helgrind/tests/getaddrinfo (stderr)
helgrind/tests/hg05_race2 (stderr)
helgrind/tests/tc20_verifywrap (stderr)
drd/tests/annotate_trace_memory (stderr)
drd/tests/annotate_trace_memory_xml (stderr)
drd/tests/tc19_shadowmem (stderr)
none/tests/arm/neon64 (stdout)
none/tests/arm/v8crypto_t (stdout)
none/tests/arm/v8crypto_t (stderr)
none/tests/arm/v8fpsimd_t (stdout)
none/tests/arm/v8fpsimd_t (stderr)
none/tests/arm/vcvt_fixed_float_VFP (stdout)
none/tests/arm/vcvt_fixed_float_VFP (stderr)

So far I've only looked at a few. The 'none' tests are all related to the build failures.

The leak ones are likely to be semi-spurious - it's hard to make leak detection very robust as memcheck will search memory (and registers) for pointers to unfreed memory. That can cause mis-categorization from the view of the testcase.

The varinfo ones look like problems with DWARF debuginfo.

getaddrinfo, I might need to add that to the suppressions. Debugging with /usr/lib/arm-linux-gnueabihf/libnss_mdns4_minimal.so.2 debuginfo would help.

[Pjfloyd (pjfloyd)]

gdbserver tests.

mcblocklistsearch is based on memcheck/tests/leak-tree which is also failing, for now I assume that fixing the leak detection will also fix the gdbserver tests

3 other are related to regtest filters.

Fixed upstream. 27 to go.

[Pjfloyd (pjfloyd)]

I've fixed a few things upstream, and now get

== 720 tests, 20 stderr failures, 4 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==

Two things aren't simple to deal with. The varinfo failures are due to not being able to read the debuginfo related to local variables. I don't know if that is a Valgrind bug or a GCC/ARM limitation. It's not a big problem, just makes the error messages a bit less clear. helgrind/tests/hg05_race2 is in the same boat.

There seems to be some issue with the 'newarray' leak detection heuristic. Again not too serious, it just changes how detected leaks get categorized.

The above two are the main ones that affect users.

I still have a number of failures related to CPU features. I need to improve the autoconf configure script to improve that, and also add compiler switches on a case by case basis for specific tests.

I need to look at memcheck/tests/sem to see exactly which bit of memory is triggering the spurious looking error.

The other errors all look fairly cosmetic.

[Pjfloyd (pjfloyd)]

== 721 tests, 10 stderr failures, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==
memcheck/tests/dw4 (stderr)
memcheck/tests/supp_unknown (stderr)
memcheck/tests/varinfo2 (stderr)
memcheck/tests/varinfo4 (stderr)
memcheck/tests/varinfo5 (stderr)
memcheck/tests/varinfo6 (stderr)
memcheck/tests/varinforestrict (stderr)
helgrind/tests/bug392331 (stderr)
helgrind/tests/hg05_race2 (stderr)
helgrind/tests/tc20_verifywrap (stderr)

That's about as good as I can get. There's at least one more upstream fix since my previous post here.