pt-archiver deletes data despite --dry-run
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| Percona Toolkit moved to https://jira.percona.com/projects/PT |
Fix Released
|
Critical
|
Daniel Nichter | |
| 2.1 |
Fix Released
|
Critical
|
Daniel Nichter | |
| 2.2 |
Fix Released
|
Critical
|
Daniel Nichter |
Bug Description
You can see on the last iteration of the pt-archiver command that --purge --dry-run is in the middle of the other options and it was not honored.
[revin@forge ~]$ pt-archiver --version
pt-archiver 2.2.3
[revin@forge ~]$ pt-archiver --purge --dry-run --set-vars=
SELECT /*!40001 SQL_NO_CACHE */ `h_id`,
SELECT /*!40001 SQL_NO_CACHE */ `h_id`,
DELETE FROM `test`.`ft_history` WHERE (((`h_id` >= ?))) AND (((`h_id` <= ?))) AND (h_id > 900000) LIMIT 1000
[revin@forge ~]$ pt-archiver --set-vars=
SELECT /*!40001 SQL_NO_CACHE */ `h_id`,
SELECT /*!40001 SQL_NO_CACHE */ `h_id`,
DELETE FROM `test`.`ft_history` WHERE (((`h_id` >= ?))) AND (((`h_id` <= ?))) AND (h_id > 950000) LIMIT 1000
[revin@forge ~]$ pt-archiver --set-vars=
Started at 2013-07-
Source: D=test,
SELECT 19385
INSERT 0
DELETE 19385
Action Count Time Pct
bulk_deleting 20 1.0916 70.25
commit 20 0.1457 9.38
select 21 0.0165 1.06
other 0 0.3000 19.31
Related branches
- Daniel Nichter: Approve on 2013-08-13
-
Diff: 2437 lines (+766/-367)53 files modifiedChangelog (+2/-0)
Makefile.PL (+1/-1)
bin/pt-agent (+12/-2)
bin/pt-align (+12/-2)
bin/pt-archiver (+12/-2)
bin/pt-config-diff (+12/-2)
bin/pt-deadlock-logger (+12/-2)
bin/pt-diskstats (+12/-2)
bin/pt-duplicate-key-checker (+12/-2)
bin/pt-fifo-split (+12/-2)
bin/pt-find (+12/-2)
bin/pt-fingerprint (+12/-2)
bin/pt-fk-error-logger (+12/-2)
bin/pt-heartbeat (+12/-2)
bin/pt-index-usage (+12/-2)
bin/pt-ioprofile (+28/-24)
bin/pt-kill (+12/-2)
bin/pt-mext (+1/-1)
bin/pt-mysql-summary (+28/-24)
bin/pt-online-schema-change (+12/-2)
bin/pt-pmp (+28/-24)
bin/pt-query-digest (+12/-2)
bin/pt-show-grants (+12/-2)
bin/pt-sift (+28/-24)
bin/pt-slave-delay (+12/-2)
bin/pt-slave-find (+12/-2)
bin/pt-slave-restart (+12/-2)
bin/pt-stalk (+28/-24)
bin/pt-summary (+28/-24)
bin/pt-table-checksum (+12/-2)
bin/pt-table-sync (+12/-2)
bin/pt-table-usage (+12/-2)
bin/pt-upgrade (+12/-2)
bin/pt-variable-advisor (+12/-2)
bin/pt-visual-explain (+12/-2)
lib/OptionParser.pm (+12/-1)
lib/PerconaTest.pm (+14/-0)
lib/VariableAdvisorRules.pm (+1/-0)
lib/bash/parse_options.sh (+27/-23)
t/lib/Daemon.t (+58/-52)
t/lib/OptionParser.t (+54/-0)
t/lib/Percona/Toolkit.t (+2/-1)
t/lib/Percona/WebAPI/Client.t (+2/-1)
t/lib/QueryReportFormatter.t (+2/-2)
t/lib/VariableAdvisorRules.t (+1/-0)
t/lib/VersionCheck.t (+1/-0)
t/pt-archiver/bugs.t (+56/-0)
t/pt-diskstats/expected/all_int_bug-1035311.txt (+0/-31)
t/pt-diskstats/expected/disk_int_bug-1035311.txt (+0/-3)
t/pt-diskstats/expected/sample_int_bug-1035311.txt (+0/-3)
t/pt-diskstats/pt-diskstats.t (+50/-51)
t/pt-mext/pt-mext.t (+2/-0)
t/pt-stalk/pt-stalk.t (+0/-1)
- Daniel Nichter: Approve on 2013-08-14
- Diff: 0 lines
- Daniel Nichter: Approve on 2013-08-14
-
Diff: 2615 lines (+1601/-86)31 files modifiedbin/pt-archiver (+51/-1)
bin/pt-config-diff (+51/-1)
bin/pt-deadlock-logger (+51/-1)
bin/pt-diskstats (+51/-1)
bin/pt-duplicate-key-checker (+51/-1)
bin/pt-fifo-split (+51/-1)
bin/pt-find (+51/-1)
bin/pt-fingerprint (+51/-1)
bin/pt-fk-error-logger (+51/-1)
bin/pt-heartbeat (+51/-1)
bin/pt-index-usage (+51/-1)
bin/pt-kill (+51/-1)
bin/pt-log-player (+51/-1)
bin/pt-online-schema-change (+51/-1)
bin/pt-query-advisor (+51/-1)
bin/pt-query-digest (+51/-1)
bin/pt-show-grants (+51/-1)
bin/pt-slave-delay (+51/-1)
bin/pt-slave-find (+51/-1)
bin/pt-slave-restart (+51/-1)
bin/pt-table-checksum (+51/-1)
bin/pt-table-sync (+51/-1)
bin/pt-table-usage (+57/-10)
bin/pt-tcp-model (+51/-1)
bin/pt-trend (+51/-1)
bin/pt-upgrade (+51/-1)
bin/pt-variable-advisor (+51/-1)
bin/pt-visual-explain (+57/-10)
lib/OptionParser.pm (+52/-40)
t/lib/OptionParser.t (+53/-0)
t/pt-archiver/bugs.t (+56/-0)
tags: | added: risk |
Changed in percona-toolkit: | |
status: | New → Confirmed |
Changed in percona-toolkit: | |
milestone: | none → 2.2.5 |
importance: | Undecided → Critical |
summary: |
- pt-archiver Does not honor --dry-run if its not on the first 2 or last 2 - options before --source + pt-archiver deletes data despite --dry-run |
Changed in percona-toolkit: | |
status: | Confirmed → In Progress |
assignee: | nobody → Daniel Nichter (daniel-nichter) |
Daniel Nichter (daniel-nichter) wrote : | #1 |
Daniel Nichter (daniel-nichter) wrote : | #2 |
Every string type opt in every tool is broken all the way back to Maatkit. For example, if --foo takes a string, and the command line is --foo --bar, then "--bar" becomes the value of --foo, so it's like --bar was never specified.
This probably explains many heisenbugs over the years. It's not made every tool fall apart mostly likely because options as vals to other options are usually invalid input. For example, something like "--input-file --foo" would probably fail with an error like "File --foo does not exist". But as this bug shows, not all options have good input validation, and the consequence can be bad if the wrong option (like --dry-run) is silently consumed.
This will be fixed in 2.2.5 and backported to 2.1.11 (so yes, there will be another 2.1 release).
Knowing this may help you detect such heisenbugs in the future, i.e. if the tool seems to be running oddly, not doing the right thing (when you're sure it should), then double-check that all string opts were actually given values.
tags: |
added: all-tools option-parsing removed: pt-archiver |
Shahriyar Rzayev (rzayev-sehriyar) wrote : | #3 |
Percona now uses JIRA for bug reports so this bug report is migrated to: https:/
One problem is that --optimize is not being used correctly: the option takes an argument: d, s, or ds (see --analyze). The real problem is that --optimize is consuming the next option, which is --dry-run in this case. This shouldn't happen; it means the option parser is failing to notice that --dry-run is not the string val to --optimize but rather an option; it should catch this and the tool should fail to start with an error like "--optimize requires a value".