In some test cases, during execution of the test several distinct build processes generate executables with the same name, overwriting older versions. (E.g. a file may be compiled as C first and as C++ later, or a file may be compiled with different compile options). When running the testsuite remotely, executables need to be copied over to the target before running the test. However, there is a cache that omits copying files that were already copied if their (1-second resolution) time stamp remains the same.
This generates a race condition in that in some tests, multiple versions of an executable are generated within the same second. If that happens, the second copy is omitted, and a wrong version of the executable is tested. This can cause spurious failures, or some tests to be missed silently.
To fix this, either the test cases need to be changed to use different executable file names, or else the caching mechanism needs to be enhanced to not solely rely on a time stamp (but e.g. a hash).
The consensus on the list was that test executable names should always be different in any case (to simplify manual debugging of a failing case). I've checked in the following two patches to mainline to fix this:
http://
http://