Comment 8 for bug 1108035

Revision history for this message
Jay Janssen (jay-janssen) wrote : Re: [Bug 1108035] PXC innodb_buffer_pool_restore_at_startup and node joining cluster io resources race condition

The buffer pool is clearly getting restored in both places:

On Feb 18, 2013, at 6:24 PM, Ilias Bertsimas <email address hidden> wrote:

> I can confirm that innodb_blocking_buffer_pool_restore=ON solves the
> issue but with a weird side-effect. Galera tries to restore the
> wsrep_position and with that option enabled it spends a lot of time on
> that part:
>
> 130205 13:10:38 mysqld_safe Starting mysqld daemon with databases from /data/mysql
> 130205 13:10:38 mysqld_safe WSREP: Running position recovery with --log_error=/tmp/tmp.nci5tZ9xZF
> 130205 13:16:37 mysqld_safe WSREP: Recovered position 402367df-5fd0-11e2-0800-58b321ec9eec:518510084

The --wsrep_recover starts Innodb using our standard my.cnf, and it allocates and loads the full buffer pool (without logging as it seems).

>
> Almost 6 minutes spent above when without the innodb_blocking_buffer_pool_restore=ON it is pretty much instant.
> It is not that it restores the buffer pool pages at that point as the log indicates later on:
>
> 130205 13:16:45 InnoDB: Restoring buffer pool pages from ib_lru_dump
> 130205 13:22:52 InnoDB: Completed reading buffer pool pages (requested: 3178478, read: 3178076)
>
> What do you think ?

Again, Innodb blocks until the buffer pool is reloaded, again incurring the 6 minutes penalty (which just happens to be the time it takes your LRU dump to reload). This is what I would expect with innodb_blocking_buffer_pool_restore=ON

So, here's the issues as I see them:
- The mandatory wsrep_recover step has some interesting side-effects. Why do we even force run that? If we must do that, can we ignore certain variables in the config? (Huge buffer pools can slow this down, I've noticed too)
- LRU restore obviously has some side effect on cluster operation, but it's not clear why that should be the case. This is probably the real bug here.

Jay Janssen, MySQL Consulting Lead, Percona
http://about.me/jay.janssen
Percona Live in Santa Clara, CA April 22nd-25th 2013
http://www.percona.com/live/mysql-conference-2013/