Activity log for bug #1371827

Date Who What changed Old value New value Message
2014-09-19 23:45:07 Roel Van de Paar bug added bug
2014-09-19 23:45:16 Roel Van de Paar nominated for series percona-server/5.1
2014-09-19 23:45:16 Roel Van de Paar bug task added percona-server/5.1
2014-09-19 23:45:16 Roel Van de Paar nominated for series percona-server/5.6
2014-09-19 23:45:16 Roel Van de Paar bug task added percona-server/5.6
2014-09-19 23:45:16 Roel Van de Paar nominated for series percona-server/5.5
2014-09-19 23:45:16 Roel Van de Paar bug task added percona-server/5.5
2014-09-19 23:48:22 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives; mysql> show processlist; +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ | Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined | +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ 2 rows in set (0.00 sec) mysql> show processlist; +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ | Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined | +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | +----+------+-----------+--------------------+---------+------+----------------+----------------------------------------------+-----------+---------------+ 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions. Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives; (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions.
2014-09-19 23:48:31 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives; (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions. Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions.
2014-09-19 23:48:47 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions. Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions.
2014-09-19 23:49:59 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (broken in/not continuing)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions. Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (and where broken had broken in/was not continuing to run code)). - This issue also seems unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions.
2014-09-22 07:10:09 Laurynas Biveinis percona-server/5.6: importance Critical Undecided
2014-09-22 07:10:13 Laurynas Biveinis percona-server/5.6: status Confirmed New
2014-09-23 23:40:57 Roel Van de Paar summary Serious sporadic single-thread non-threadpool single-gdb-tread partial-hangup issue in mysqld Serious sporadic single-thread non-threadpool partial-hangup issue in mysqld
2014-09-23 23:49:11 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - GDB breakin at time of issue gives strange (i.e. single mysqld thread alive only) output: (gdb) bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68. 0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5. 6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6. 20-68.0/sql/main.cc:25 (gdb) thread apply all bt Thread 1 (process 5077): +bt #0 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 #1 0x0000000000627bb4 in handle_connections_sockets () at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:6424 #2 0x0000000000626cc6 in mysqld_main (argc=15, argv=0x2b7c1a0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/mysqld.cc:5833 #3 0x000000000061baf0 in main (argc=11, argv=0x7fff24a39398) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.20-68.0/sql/main.cc:25 (gdb) info threads   Id Target Id Frame * 1 process 5077 "mysqld" 0x00007fd069df6e0d in poll () from /lib64/libc.so.6 (gdb) show scheduler-locking Mode for locking scheduler during execution is "step". (gdb) show schedule-multiple Resuming the execution of threads of all processes is off. (gdb) show non-stop Controlling the inferior in non-stop mode is off. (gdb) show target-async Controlling the inferior in asynchronous mode is off. - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (and where broken had broken in/was not continuing to run code)). - This issue also seems unrelated to the previously discussed "very high "Time" count in processlist" - the second cound looks correct and is the same accross all 3 'partial hung' sessions. Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (and where broken had broken in/was not continuing to run code)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" caused by SET TIMESTAMP (see http://bugs.mysql.com/bug.php?id=73999).
2014-10-02 08:21:52 Laurynas Biveinis percona-server/5.6: status New Incomplete
2014-10-02 08:21:58 Laurynas Biveinis percona-server/5.6: assignee Laurynas Biveinis (laurynas-biveinis) Roel Van de Paar (roel11)
2014-10-08 05:03:22 Ramesh Sivaraman attachment added core_ldd.tar.gz https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4228011/+files/core_ldd.tar.gz
2014-10-09 03:45:03 Roel Van de Paar percona-server/5.6: status Incomplete New
2014-10-09 03:45:23 Roel Van de Paar percona-server/5.6: assignee Roel Van de Paar (roel11) Laurynas Biveinis (laurynas-biveinis)
2014-10-10 04:35:31 Ramesh Sivaraman attachment added reducer105.sh.zip https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4230306/+files/reducer105.sh.zip
2014-10-16 05:27:10 Roel Van de Paar tags qa qa qablock
2014-10-17 23:10:54 Roel Van de Paar percona-server/5.6: status New Confirmed
2014-10-18 07:12:45 Laurynas Biveinis percona-server/5.6: status Confirmed Incomplete
2014-10-18 07:12:56 Laurynas Biveinis percona-server/5.6: assignee Laurynas Biveinis (laurynas-biveinis) Roel Van de Paar (roel11)
2014-10-22 22:55:15 Roel Van de Paar percona-server/5.6: assignee Roel Van de Paar (roel11) Ramesh Sivaraman (rameshvs02)
2014-10-22 22:57:58 Roel Van de Paar percona-server/5.6: status Incomplete Confirmed
2014-10-23 05:03:22 Roel Van de Paar attachment added 1st glibc, seen during a pquery run https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4242173/+files/glibc.txt
2014-10-23 05:04:00 Roel Van de Paar attachment added 2nd glibc, seen during rqg run https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4242174/+files/glibc2.txt
2014-10-23 05:50:51 Ramesh Sivaraman attachment added core.2921.tar.gz https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4242177/+files/core.2921.tar.gz
2014-10-23 13:20:35 Laurynas Biveinis percona-server/5.6: assignee Ramesh Sivaraman (rameshvs02) Laurynas Biveinis (laurynas-biveinis)
2014-10-23 13:33:03 Laurynas Biveinis percona-server/5.6: assignee Laurynas Biveinis (laurynas-biveinis) Ramesh Sivaraman (rameshvs02)
2014-10-23 17:08:32 Ramesh Sivaraman attachment added mysqld.tar.gz https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4242503/+files/mysqld.tar.gz
2014-10-24 16:07:23 Laurynas Biveinis percona-server/5.6: assignee Ramesh Sivaraman (rameshvs02) Laurynas Biveinis (laurynas-biveinis)
2014-11-01 03:29:54 Roel Van de Paar percona-server/5.6: status Confirmed Triaged
2014-11-01 03:29:59 Roel Van de Paar percona-server/5.6: status Triaged Confirmed
2014-11-21 13:30:19 Laurynas Biveinis summary Serious sporadic single-thread non-threadpool partial-hangup issue in mysqld Sporadic partial-hangup
2014-11-22 06:20:48 Roel Van de Paar description Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadhpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (and where broken had broken in/was not continuing to run code)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" caused by SET TIMESTAMP (see http://bugs.mysql.com/bug.php?id=73999). Symptoms: - Looks to be sporadic (single thread SQL replay produces issue 3 out of 10 replays) - Issue seems connected with mysql.slave_relay_log_info or SHOW CREATE TABLE mysql.slave_relay_log_info. All 3 (out of 10) SQL replays showed the "partial hang" on the same line - Issue is single-sql-thread reproducible - No special mysqld settings used (i.e. likely no connection with treadpool) - CLI client remains able to connect. SHOW PROCESSLIST gives;   (cleaned up the processlist formatting a bit) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 711 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) mysql> show processlist; -------------------------------------------------------------------------- Id | User | Host | db | Command | Time | State | Info | Rows_sent | Rows_examined -------------------------------------------------------------------------- | 6 | root | localhost | information_schema | Query | 713 | Opening tables | SHOW CREATE TABLE mysql.slave_relay_log_info | 0 | 0 | | 12 | root | localhost | NULL | Query | 0 | System lock | show processlist | 0 | 0 | -------------------------------------------------------------------------- 2 rows in set (0.00 sec) - Whilst GDB is active, CLI does not work (as expected, but we had another odd bug recently where one could still CLI connect even when GDB was active (and where broken had broken in/was not continuing to run code)). - This issue is unrelated to the previously discussed "very high "Time" count in processlist" caused by SET TIMESTAMP (see http://bugs.mysql.com/bug.php?id=73999).
2014-11-22 07:15:14 Roel Van de Paar attachment added bug_1371827_new.tar.gz https://bugs.launchpad.net/percona-server/+bug/1371827/+attachment/4265535/+files/bug_1371827_new.tar.gz
2014-12-02 11:54:51 Roel Van de Paar summary Sporadic partial-hangup Sporadic partial-hangup on various queries + related (same-testcase) crashes/asserts
2014-12-02 13:52:11 Laurynas Biveinis percona-server/5.1: status New Invalid
2014-12-02 13:52:13 Laurynas Biveinis percona-server/5.5: status New Invalid
2014-12-02 13:53:03 Laurynas Biveinis percona-server/5.6: importance Undecided Medium
2014-12-02 13:53:05 Laurynas Biveinis percona-server/5.6: status Confirmed Triaged
2014-12-02 13:58:18 Laurynas Biveinis tags qa qablock backup-locks qa qablock
2015-02-28 10:51:49 Alexey Kopytov percona-server/5.6: assignee Laurynas Biveinis (laurynas-biveinis) Alexey Kopytov (akopytov)
2015-02-28 11:07:27 Alexey Kopytov percona-server/5.6: status Triaged Fix Committed
2015-02-28 12:49:29 Alexey Kopytov percona-server/5.6: status Fix Committed Fix Released
2015-02-28 12:49:37 Alexey Kopytov percona-server/5.6: milestone 5.6.24-72.2