Aria unit tests fail at ma_test2 line 316

[Elena Stepanova]

Initially reported on maria-discuss at launchpad:
https://lists.launchpad.net/maria-discuss/msg00601.html

[16:34:05] ../storage/maria/unittest/ma_test_all-t .....................
73/830
Failed test '../storage/maria/unittest/../ma_test2 -s -htmp -m10000
-e4096 -K ' (loop iteration 8.) at line 316
(errcode: 256, test: 83)

etc.

The failure is only observed with -O2 (not -O1)

[Paolo-pedroni (paolo-pedroni)]

There is (and it's been there for a few months already) a new version (5.1.60) of mariadb on the mariadb site. An ebuild (even in the overlay) would be nice. I don't know (and I can't check at the moment) if a simple rename would do the trick.

[Brian (grknight)]

MariaDB 5.1.62 is now in the overlay.
A patch from mysql-extras needs a change so please use the updated info.

[kripton (kripton-r)]

MariaDB 5.1.62 is now in official tree as unstable :)
RESOLVED FIXED?

[Robin H. Johnson (robbat2)]

Please run the testsuites. I'm not 100% happy with it.

[Paolo-pedroni (paolo-pedroni)]

With FEATURES="test -usersandbox" I get these errors:
[13:56:10] mysys/waiting_threads-t ..................................... ok 4 ms
[13:56:10] mysys/base64-t .............................................. ok 154 ms
[13:56:10] mysys/lf-t .................................................. ok 155 ms
[13:56:10] mysys/my_atomic-t ........................................... ok 62 ms
[13:56:10] mysys/bitmap-t .............................................. ok 289 ms
[13:56:10] strings/strings-t ........................................... ok 1 ms
[13:56:10] ../storage/maria/unittest/ma_pagecache_consist_64kHC-t ...... ok 8 ms
[13:56:10] ../storage/maria/unittest/ma_pagecache_consist_64k-t ........ ok 3 ms
[13:56:10] ../storage/maria/unittest/ma_test_loghandler_nologs-t ....... ok 299 ms
[13:56:11] ../storage/maria/unittest/ma_test_loghandler_noflush-t ...... ok 19 ms
[13:56:11] ../storage/maria/unittest/ma_pagecache_consist_1k-t ......... ok 8 ms
[13:56:11] ../storage/maria/unittest/ma_test_loghandler_long-t ......... ok 3861 ms
[13:56:15] ../storage/maria/unittest/ma_pagecache_consist_1kHC-t ....... ok 7 ms
[13:56:15] ../storage/maria/unittest/ma_test_loghandler_readonly-t ..... ok 55 ms
[13:56:15] ../storage/maria/unittest/ma_test_loghandler_purge-t ........ ok 256 ms
[13:56:15] ../storage/maria/unittest/ma_test_loghandler_max_lsn-t ...... ok 269 ms
[13:56:15] ../storage/maria/unittest/ma_test_loghandler_first_lsn-t .... ok 21 ms
[13:56:15] ../storage/maria/unittest/ma_pagecache_consist_64kRD-t ...... ok 33304 ms
[13:56:49] ../storage/maria/unittest/ma_pagecache_consist_1kRD-t ....... ok 443 ms
[13:56:49] ../storage/maria/unittest/ma_pagecache_single_1k-t .......... ok 333 ms
[13:56:49] ../storage/maria/unittest/ma_test_loghandler_multiflush-t ... ok 5572 ms
[13:56:55] ../storage/maria/unittest/ma_test_loghandler_pagecache-t .... ok 22 ms
[13:56:55] ../storage/maria/unittest/ma_test_loghandler_multithread-t .. ok 29585 ms
[13:57:25] ../storage/maria/unittest/ma_test_loghandler_multigroup-t ... ok 27926 ms
[13:57:53] ../storage/maria/unittest/ma_test_loghandler-t .............. ok 4067 ms
[13:57:57] ../storage/maria/unittest/ma_pagecache_rwconsist2_1k-t ...... ok 6 ms
[13:57:57] ../storage/maria/unittest/ma_pagecache_rwconsist_1k-t ....... ok 6 ms
[13:57:57] ../storage/maria/unittest/ma_pagecache_consist_64kWR-t ...... ok 43644 ms
[13:58:40] ../storage/maria/unittest/ma_pagecache_consist_1kWR-t ....... ok 1317 ms
[13:58:42] ../storage/maria/unittest/ma_pagecache_single_8k-t .......... ok 1846 ms
[13:58:44] ../storage/maria/unittest/ma_pagecache_single_64k-t ......... ok 8 ms
[13:58:44] ../storage/maria/unittest/ma_control_file-t ................. ok 1 ms
[13:58:44] ../storage/maria/unittest/trnman-t .......................... ok 687 ms
[13:58:44] ../storage/maria/unittest/ma_test_all-t ..................... 77/830
Failed test '../storage/maria/unittest/../ma_test2 -s -htmp -m10000 -e4096 -K ' (loop iteration 8.) at line 316
(errcode: 256, test: 83)
[13:58:44] ../storage/ma...

[Paolo-pedroni (paolo-pedroni)]

Nope, same errors on a real filesystem as well. :-(

[Brian (grknight)]

(In reply to comment #4)
> With FEATURES="test -usersandbox" I get these errors:
[snip]
>
> Test Summary Report
> -------------------
> ../storage/maria/unittest/ma_test_all-t (Wstat: 256 Tests:
> 830 Failed: 15)
> Failed tests: 83, 85, 87, 242, 244, 246, 401, 403, 405
> 560, 562, 564, 719, 721, 723
> Non-zero exit status: 1
> Files=35, Tests=145284, 267 wallclock secs ( 8.93 usr 0.30 sys + 138.85
> cusr 56.43 csys = 204.51 CPU)
> Result: FAIL
> Failed 1/35 test programs. 15/145284 subtests failed.
>
> Could it be because I have /var/tmp/portage on a tmpfs?

I've discovered that ma_test_all-t fails with -O2 or -Os in C(XX)FLAGS set.

They work fine if -O1 is set.

I've pointed this out on the MariaDB mailing list, but not sure if it's a MariaDB bug or gcc optimization issue.

@Robin: Do we need to force -O1 for tests only or should we always do it?

[kripton (kripton-r)]

I get the same test failures but they are not new since I get exactly the same test cases failing with version 5.1.50.

[Elena Stepanova (elenst)]

From what I see, ma_test2 has the array key3[5000], used in a loop which sets

n3=rnd(5000);
and then reads
key3[n3] == ...

while rnd(uint max_value) is
return (uint) ((rand() & 32767)/32767.0*max_value);

which means with some luck it can return max_value itself.

Luck happens, rnd returns 5000, key3[n3] reads something entirely wrong (in my case the value of 'silent' variable which is int and set to 1 as the unit test is launched with -s), the entire condition check with key3[n3] gets twisted, the bogus error occurs.

The following change fixes the problem for me, although maybe there are more elegant solutions.

=== modified file 'storage/maria/ma_test2.c'
--- storage/maria/ma_test2.c 2011-02-25 12:55:40 +0000
+++ storage/maria/ma_test2.c 2012-04-20 10:05:45 +0000
@@ -259,7 +259,7 @@
   for (i=0 ; i < recant ; i++)
   {
     ulong blob_length;
- n1=rnd(1000); n2=rnd(100); n3=rnd(5000);
+ n1=rnd(1000); n2=rnd(100); n3=rnd(4999);
     sprintf((char*) record,"%6d:%4d:%8d:Pos: %4d ",n1,n2,n3,write_count);
     int4store(record+STANDARD_LENGTH-4,(long) i);
     fix_length(record,(uint) STANDARD_LENGTH+rnd(60));

[Brian (grknight)]

More info about ma_test_all-t that may be causing random test failures:
https://lists.launchpad.net/maria-discuss/msg00608.html

[Brian (grknight)]

Created attachment 310111
Fix index errors in the ma_test2 unit test binary

This patch fixes the errors on ma_test2 during unit tests.
It appears randomly due to the rnd function returning a random number between and including 0 to 5000. Since there are 5000 entries in the array, if rnd() pulls 5000, it's out of bounds.

When applied, my test box passes all tests with the following parameters.
FEATURES="test userpriv -usersandbox" USE='-cluster -latin1 embedded extraengine perl ssl community'

This also applies and fixes on mariadb-5.2.12 and mariadb-5.3.6.

5.2.12 only 1 test fails (plugins.unix_socket) due to Portage stripping $USER.
5.3.6 has the same test failure due to Portage

[Brian Evans (grknight-scent-team)]

I've attached a patch which helped me resolve the random test failures.

There is a second spot for a possible index out of range that was not previously reported.

[Brian (grknight)]

Upstream bug on the test failure:
https://bugs.launchpad.net/maria/+bug/986261

[Brian (grknight)]

Patch is now InCVS and will ship with the next mysql-extras release.

[Jorge Manuel B. S. Vicetto (jmbsvicetto)]

*** Bug 423467 has been marked as a duplicate of this bug. ***