slow XtraDB shutdown due to 10 second sleep in purge thread

[Kristian Nielsen]

Shutting down XtraDB can take up to 10 seconds, even when there is no actual
work to do during shutdown.

An easy way to see this is to run a quick innodb test in the mysql-test-run.pl
test suite:

    ./mtr innodb-consistent

This takes >10 seconds, even though the actual test runs instantaneous.

The problem is this code in srv/srv0srv.c:

    srv_purge_thread(...)
    {
        ulint sleep_ms= 10000; /* initial: 10 sec. */
        ...

    loop:
        if (srv_shutdown_state > 0) {
            ...
        }
        os_thread_sleep( sleep_ms * 1000 );
        ...
        goto loop;

So right after startup, the purge thread sleeps for 10 seconds, so server
cannot shut down for at least 10 seconds after startup.

Of course, in a production server, the purge thread sleep could have adjusted
itself to less than 10 seconds, and a busy server anyway will probably need
much more than 10 seconds to shut down cleanly.

But for development and testing, it is annoying to have to wait for 10 seconds
for shutdown.

Suggested fix:

Change the os_thread_sleep() into a sleep that can be interrupted by
shutdown. For example a wait with os_event_wait_time() on an event created
specifically for this. (Note that currently os_event_wait_time() waits
infinitely on non-windows, but it looks easy to fix this using
pthread_cond_timedwait()). Then signal the purge thread sleep to wakeup early
during server shutdown.

[Kristian Nielsen (knielsen)]

Work-around: --innodb-fast-shutdown=2, if that is acceptable (which it probably often is during test/development).

[Kristian Nielsen (knielsen)]

Just tested running the full test suite on my laptop (release build):

(cd mysql-test && time ./mtr --mem --parallel=3 --force)
    -> 18:16.60elapsed

(cd mysql-test && time ./mtr --mem --parallel=3 --force --mysqld=--loose-innodb-fast-shutdown=2)
    -> 11:14.47elapsed

So that's 38% faster ... so fixing this does seem rather worth-while.

[Kristian Nielsen (knielsen)]

Here is a patch for this.

It greatly speeds up server shutdown for tests (though --innodb-fast-shutdown=2 is still a couple seconds faster).

[Kristian Nielsen (knielsen)]

I updated the patch, attached. I will push this patch into MariaDB, hopefully it can be included
upstream in XtraDB.

[Kristian Nielsen (knielsen)]

I used PMP to find some more sleeps in the XtraDB shutdown code. This additional
patch, which goes on top of the one attached before, fixes the remaining cases.

With this, server shutdown during mtr tests is very close to --innodb-fast-shutdown-2.

[Kristian Nielsen (knielsen)]

Attaching BSD copyright license for the patches submitted to this bug report.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1834