Log parsing buffer overflow

[Tomislav Plavcic]

The following error can happen during the re-start of the server after it has been killed:
[ERROR] InnoDB: Log parsing buffer overflow. Recovery may have failed!

It has been reported by PZ and I have reproduced the issue, but currently I couldn't reduce it. I tried with using smaller number of tables (instead of 100000 tried with 50000 but couldn't reproduce). So this needs more work but reporting it here since maybe somebody will bump into same issue and provide some other details.

Steps to reproduce:
I used: Percona Server 5.7.14-7 release build

mysqld options:
--no-defaults --skip-external-locking --max_allowed_packet=16M --thread_stack=192K --thread_cache_size=8 --max_connections=4096 --query_cache_limit=1M --query_cache_size=16M --expire_logs_days=1 --max_binlog_size=100M --innodb_buffer_pool_size=24G --innodb_log_file_size=1G --innodb_log_buffer_size=64M --innodb_flush_log_at_trx_commit=0 --innodb_flush_method=O_DIRECT --innodb_io_capacity=1000 --innodb_io_capacity_max=10000 --innodb_checksum_algorithm=strict_crc32 --innodb_purge_threads=8 --innodb_flush_neighbors=0 --innodb_change_buffer_max_size=5 --innodb_flush_log_at_trx_commit=2 --innodb_max_dirty_pages_pct=95 --innodb_sync_array_size=4 --innodb_monitor_enable="%" --innodb_max_purge_lag=1000000 --innodb_max_purge_lag_delay=10000 --innodb_undo_tablespaces=2 --innodb_undo_log_truncate=1

sysbench prepare (it will run few hours depending on the machine):
sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --report-interval=1 --oltp-tables-count=100000 --oltp-table-size=1000 --max-time=0 --oltp-read-only=off --max-requests=0 --num-threads=64 --rand-type=pareto --db-driver=mysql --mysql-user=sbtest --mysql-password=sbtest --mysql-db=sbtest prepare

sysbench run:
sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --report-interval=1 --oltp-tables-count=100000 --oltp-table-size=1000 --max-time=0 --oltp-read-only=off --max-requests=0 --num-threads=64 --rand-type=pareto --db-driver=mysql --mysql-user=root --mysql-db=sbtest run

After the sysbench has been running for 10min just kill the server with something like "sudo kilall -9 mysqld" and then try to start the server again.

[Tomislav Plavcic (tplavcic)]

Logged upstream bug here: https://bugs.mysql.com/bug.php?id=83245

[Tomislav Plavcic (tplavcic)]

Upstream change:
"Fixed as of the upcoming 5.7.18, 8.0.1 release, and here's the changelog entry:
During a checkpoint, all MLOG_FILE_NAME redo log records were written in
a single mini-transaction (mtr), causing a log parsing buffer overflow."

[Laurynas Biveinis (laurynas-biveinis)]

https://github.com/percona/percona-server/pull/1695

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1015