pt-archiver misses data when only copying data
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Percona Toolkit moved to https://jira.percona.com/projects/PT |
Won't Fix
|
High
|
Carlos Salguero |
Bug Description
when using '--no-delete' option in pt-archiver (to only copy data to destination) if the where clause column can have multiple records then pt-archiver will miss records in copying.
e.g. create table with following columns:
id, timestamp
data:
| 001-25941882 | 1280490242452 |
| 001-25941883 | 1280490242452 |
| 001-25941884 | 1280490242452 |
| 001-25941885 | 1280490242452 |
| 001-25941886 | 1280495048106 |
| 001-25941887 | 1280495048108 |
| 001-25942993 | 1280495048108 |
| 001-25942994 | 1280495048108 |
| 001-25942995 | 1280490776956 |
| 001-25942996 | 1280493558772 |
use batch size of 2 (limit) and with commit-each option, the pt-archiver will miss copying few records.
pt version- 2.1.7
mysql version - 5.1.40
command:
/usr/bin/
It's pretty easy to reproduce and happens due to optimized SQL (which add in above case timestamp > [max value in last run]) in the next sql clause - which cause the next run to miss few records.
Changed in percona-toolkit: | |
assignee: | nobody → Daniel Nichter (daniel-nichter) |
importance: | Undecided → High |
status: | Triaged → In Progress |
Changed in percona-toolkit: | |
assignee: | Daniel Nichter (daniel-nichter) → Carlos Salguero (carlos-salguero) |
milestone: | none → 2.2.20 |
status: | Incomplete → Triaged |
Changed in percona-toolkit: | |
status: | Triaged → In Progress |
Thanks for looking into this. We'll try to reproduce it on our end too. Bug 820079 is similar, though unconfirmed yet.