MySQL 8.0 in Ubuntu ignores MTR test results, allowing faulty packages to users

[Otto Kekäläinen]

I came across https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/2019203 where MySQL was crashing for users of armhf.

This issue was actually fully detectable in the post-build mysql-test-run. Looking at build logs of https://launchpad.net/ubuntu/+source/mysql-8.0/8.0.33-0ubuntu2 anybody can see the exact crash of bug #2019203 in https://launchpadlibrarian.net/665338538/buildlog_ubuntu-mantic-armhf.mysql-8.0_8.0.33-0ubuntu2_BUILDING.txt.gz

```
RULES.override_dh_auto_test
touch builddir/mysql-test/skiplist
# Tests that are known to be unstable on all platforms are skipped
# http://bugs.mysql.com/bug.php?id=83340
echo "main.xa_prepared_binlog_off : BUG#00000 - unstable test" >> builddir/mysql-test/skiplist
echo "main.mysql_client_test : BUG#100274 - unstable test" >> builddir/mysql-test/skiplist
echo "main.type_float : BUG#92375 - fails on ppc64el. Ref https://bugs.mysql.com/bug.php?id=92375" >> builddir/mysql-test/skiplist
echo "main.type_newdecimal : BUG#92375 - Same as above" >> builddir/mysql-test/skiplist
echo "main.type_ranges : BUG#92375 - Same as above" >> builddir/mysql-test/skiplist
# https://bugs.mysql.com/bug.php?id=86608
echo "main.mysqlpump_basic : BUG#00000 - needs openssl with zlib" >> builddir/mysql-test/skiplist
# Test is broken for 32bit. Fixed upstream, so remove in 8.0.12+
echo "main.window_functions_explain : BUG#00000 - broken on i386" >> builddir/mysql-test/skiplist
# New test in 8.0.26, needs investigation
echo "main.slow_log : BUG#00000 - broken" >> builddir/mysql-test/skiplist
# Test is broken for 32bit.
echo "main.index_merge_myisam : BUG#00000 - broken on i386" >> builddir/mysql-test/skiplist
# Test is broken on Mantic
echo "main.derived_condition_pushdown : BUG#00000 - broken on mantic" >> builddir/mysql-test/skiplist
# Skip replication tests since they are timing sensitive and may
# result in false positives.
cd builddir/mysql-test && ./mtr --suite-timeout=600 --testcase-timeout=60 --report-unstable-tests --parallel=4 --skip-rpl --suite=main --force --skip-test-list=./skiplist || true ;
Logging: /<<PKGBUILDDIR>>/mysql-test/mysql-test-run.pl --suite-timeout=600 --testcase-timeout=60 --report-unstable-tests --parallel=4 --skip-rpl --suite=main --force --skip-test-list=./skiplist
2023-05-09T14:58:46Z UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
BuildID[sha1]=195a631af53dd4598a337c119fb58728b7bc1c74
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x100000
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x2b) [0x1bec0bc]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(print_fatal_signal(int)+0x29d) [0x10c1ba2]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(handle_fatal_signal+0x71) [0x10c1c9e]
/lib/arm-linux-gnueabihf/libc.so.6(+0x2d750) [0xf6d49750]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(memory::Aligned_atomic<long>::Aligned_atomic()+0x51) [0x199b49a]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(Delegate::Delegate(unsigned int)+0x3b) [0x199b6c4]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(delegates_init()+0x37) [0x199b804]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(+0x8ec92c) [0xf2392c]
/<<PKGBUILDDIR>>/builddir/runtime_output_directory/mysqld(mysqld_main(int, char**)+0x1959) [0xf2822a]
/lib/arm-linux-gnueabihf/libc.so.6(+0x1d7da) [0xf6d397da]
/lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x5d) [0xf6d3987e]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
mysql-test-run: *** ERROR: Could not find version of MySQL
```

Nobody ever looked at this, because the build is configured to ignore test results.

It should be fixed to actually fail if the mysql-test-run fails.

[Athos Ribeiro (athos-ribeiro)]

I suppose this bug is about re-enabling the skipped tests then, is this correct?

[Robie Basak (racb)]

I suspect the tests got disabled because they resulted in too many false positives. I wonder if this is still the case. Maybe we could enable specific tests rather than enabling them all and then having to constantly deal with disabling false positives.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mysql-8.0 - 8.0.35-1ubuntu1

---------------
mysql-8.0 (8.0.35-1ubuntu1) noble; urgency=medium

  * Removed patch no longer needed with 8.0.35
    - debian/patches/revert-be8348a7.patch: fix crash on startup on armhf
  * Enable test suite on armhf (LP: #2024276)
    - debian/rules: check for failures in armhf tests
    - debian/patches/suppress_armhf_test_warning.patch: suppress warning
      about the CYCLE timer not being available on armhf. (LP: #1999075)
    - debian/patches/disable_64bits_time_armhf.patch: specify armhf doesn't
      have a 64bit timer in mysql-test/include/have_64bits_time.inc.

 -- Marc Deslauriers <email address hidden> Fri, 10 Nov 2023 07:58:18 +0200