pt-diskstats doesnt process multiple files

[Chris Pappalardo]

pt-diskstats doesn't seem to process more than one file from the command-line. I've tried using both a wildcard and individually listed filenames. When I combine the file contents, the data is processed fine. I've attacahed a tar of the input data I'm using.

[Jericho Rivera (jericho-rivera)]

Can you provide us with the full command used? Thanks.

[Chris Pappalardo (cpappala)]

No options, just the file name(s). I tried using a single file name glob and multiple explicit file names, neither processed multiple files, just the first file:

pt-diskstats <file_glob>
pt-diskstats <file1> <file2>

[Jericho Rivera (jericho-rivera)]

It works for me using your sample data. Can you check which version of the tool you are using?

root@ubuntu:/home/percona# pt-diskstats --version
pt-diskstats 2.2.7

root@ubuntu:/home/percona# pt-diskstats 5-sec-status-2014-02-27_09-diskstats 5-sec-status-2014-02-27_10-diskstats

  #ts device rd_s rd_avkb rd_mb_s rd_mrg rd_cnc rd_rt wr_s wr_avkb wr_mb_s wr_mrg wr_cnc wr_rt busy in_prg io_s qtime stime
  5.0 sda 174.8 46.0 7.9 5% 4.0 21.8 1.6 20.5 0.0 80% 0.0 1.7 97% 4 176.4 15.8 5.1
  5.0 sda1 174.8 46.0 7.9 5% 4.0 21.8 1.6 20.5 0.0 80% 0.0 1.7 97% 4 176.4 15.8 5.1

  #ts device rd_s rd_avkb rd_mb_s rd_mrg rd_cnc rd_rt wr_s wr_avkb wr_mb_s wr_mrg wr_cnc wr_rt busy in_prg io_s qtime stime
 10.0 sda 192.9 46.6 8.8 5% 4.1 20.4 8.2 110.6 0.9 45% 0.9 60.5 100% 4 201.1 18.4 4.6
 10.0 sda1 192.9 46.6 8.8 5% 4.1 20.4 8.2 110.6 0.9 45% 0.9 60.5 100% 4 201.1 18.4 4.6

  #ts device rd_s rd_avkb rd_mb_s rd_mrg rd_cnc rd_rt wr_s wr_avkb wr_mb_s wr_mrg wr_cnc wr_rt busy in_prg io_s qtime stime
 15.0 sda 158.6 47.2 7.3 5% 4.2 25.2 2.8 10.3 0.0 65% 0.0 0.2 100% 34 161.4 18.6 5.7
 15.0 sda1 158.6 47.2 7.3 5% 4.2 25.2 2.8 10.3 0.0 65% 0.0 0.2 100% 34 161.4 18.6 5.7

  #ts device rd_s rd_avkb rd_mb_s rd_mrg rd_cnc rd_rt wr_s wr_avkb wr_mb_s wr_mrg wr_cnc wr_rt busy in_prg io_s qtime stime
 20.0 sda 147.8 53.5 7.7 6% 4.1 26.3 13.2 119.3 1.5 38% 11.9 562.7 99% 4 161.0 86.7 5.6
 20.0 sda1 147.8 53.5 7.7 6% 4.1 26.3 13.2 119.3 1.5 38% 11.9 562.7 99% 4 161.0 86.7 5.6

[Chris Pappalardo (cpappala)]

I don't see how your response shows it's working. The problem isn't that pt-diskstats isn't producing something, it's that the something doesn't contain the data from the files after the first file listed on the command line.

Here's another demonstration of the bug using the number of lines that pt-diskstats is producing:

cap@vega:/tmp/foo$ ~/tmp/percona-toolkit-2.2.7/bin/pt-diskstats --version
pt-diskstats 2.2.7
cap@vega:/tmp/foo$ ~/tmp/percona-toolkit-2.2.7/bin/pt-diskstats 5-sec-status-2014-02-27_09-diskstats | wc -l
1215
cap@vega:/tmp/foo$ ~/tmp/percona-toolkit-2.2.7/bin/pt-diskstats 5-sec-status-2014-02-27_10-diskstats | wc -l
1860
cap@vega:/tmp/foo$ ~/tmp/percona-toolkit-2.2.7/bin/pt-diskstats 5-sec-status-2014-02-27_09-diskstats 5-sec-status-2014-02-27_10-diskstats | wc -l
1215
cap@vega:/tmp/foo$ ~/tmp/percona-toolkit-2.2.7/bin/pt-diskstats 5-sec-status-2014-02-27_10-diskstats 5-sec-status-2014-02-27_09-diskstats | wc -l
1860

The output from the last two commands is the same as the output from the first two commands, meaning multiple files from the command line are not all being processed.

[Jericho Rivera (jericho-rivera)]

Thank you for the clarification Chris.
Confirmed as described in #5 using bug reporter's data files on pt-diskstats versions 2.2.7, 2.1.9, and 2.0.5.

[Frank Cizmich (frank-cizmich)]

Hello all,

I think this should be better handled outside the tool, because it is important for the files to be in the right order.
In the samples provided by Chris, if you concatenate the files in reverse order the tool complains about the time differences being negative.
Regardless, I coded a patch for 2.2.7 which allows the tool to accept multiple files (provided they are in the right order)

Regards!

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-850