pt-online-schema-change die handler didn't check if inside eval

Bug #1010020 reported by Baron Schwartz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Toolkit moved to https://jira.percona.com/projects/PT
Fix Released
High
Brian Fraser

Bug Description

I've observed a case where pt-online-schema-change fails to nibble from the old to new table because the new table doesn't exist. This happens because it's trying to nibble after dropping (although I don't see the rename/swap in the query log, which may be part of the bug). The behavior happens about 10-20% of the time for me on the t/pt-online-schema-change/alter_active_table.t test as of revision 262 in this branch: lp:/~percona-toolkit-dev/percona-toolkit/stabilize-test-suite/

Here is the test output I see:

[baron@localhost stabilize-test-suite]$ perl t/pt-online-schema-change/alter_active_table.t
1..9
# Loading sample dataset...
# Binlog position before altering table: mysql-bin.000001/3686358
not ok 1 - Altered OK
# Failed test 'Altered OK'
# in t/pt-online-schema-change/alter_active_table.t at line 184.
# 'Error copying rows from `pt_osc`.`t` to `pt_osc`.`_t_new`: DBD::mysql::st execute failed: Table 'pt_osc._t_new' doesn't exist [for Statement "INSERT LOW_PRIORITY IGNORE INTO `pt_osc`.`_t_new` (`id`, `c`, `d`) SELECT `id`, `c`, `d` FROM `pt_osc`.`t` FORCE INDEX(`PRIMARY`) WHERE ((`id` >= ?)) AND ((`id` <= ?)) /*pt-online-schema-change 19490 copy nibble*/" with ParamValues: 1='515', 0='504'] at /home/baron/stabilize-test-suite//bin/pt-online-schema-change line 6734.
#
# '
# doesn't match '(?-xism:Successfully altered `pt_osc`.`t`)'

Naturally, everything fails after this, because the tool has died.

I will attach the output of the query log and the output of the binary log. You can see that in the query log, at line 180, the DROP TABLE happens. Nibbling is apparently finished by this time, but then a final nibble appears to occur at line 204. This is the one that breaks the tool.

Revision history for this message
Baron Schwartz (baron-xaprb) wrote :
Revision history for this message
Baron Schwartz (baron-xaprb) wrote :
Revision history for this message
Baron Schwartz (baron-xaprb) wrote :
Revision history for this message
Baron Schwartz (baron-xaprb) wrote :

This is running on CentOS 5.5 with 5.5.25-log MySQL Community Server (GPL)

Changed in percona-toolkit:
assignee: nobody → Brian Fraser (fraserbn)
Brian Fraser (fraserbn)
tags: added: mysql-5.5
Brian Fraser (fraserbn)
Changed in percona-toolkit:
status: Confirmed → In Progress
Changed in percona-toolkit:
status: In Progress → Fix Committed
milestone: none → 2.1.2
tags: added: test-failure
Changed in percona-toolkit:
milestone: 2.1.2 → none
summary: - pt-online-schema-change can nibble after swapping the tables
+ pt-online-schema-change die handler didn't check if inside eval
Revision history for this message
Baron Schwartz (baron-xaprb) wrote :

Is this actually released now?

Changed in percona-toolkit:
status: Fix Committed → Fix Released
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-317

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.