pt-stalk occasionally removes non-empty dump files

[Maciej Dobrzanski]

After it finishes collecting data, pt-stalk removes any dump files that contain no data. However that part of the code suffers two problems and even valid files can be removed.

1. The filter condition is too strict.

if [ -z "$(grep -vE '^TS ' --max-count 1 "$file")" ]; then

Whenever a file begins with an empty line or such line immediately follows time stamp line, grep returns empty string and the file is removed.

2. Race condition.

Dumping data is sent to background to run in parallel:

   $CMD_MYSQL $EXT_ARGV -e "$innostat" >> "$d/$p-innodbstatus2" & <---
   $CMD_MYSQL $EXT_ARGV -e "$mutex" >> "$d/$p-mutex-status2" & <---
   open_tables >> "$d/$p-opentables2" & <---

   ...

   for file in "$d/$p-"*; do
      if [ -z "$(grep -vE '^(TS |$)' --max-count 1 "$file")" ]; then
         log "Removing empty file $file";
         rm "$file"
     fi
  done

However, sometimes the removal loop executes before some of the output files actually receive any data and they are removed.

[Maciej Dobrzanski (mushu)]

The second case also contains a suggested solution to the first problem:

if [ -z "$(grep -vE '^(TS |$)' --max-count 1 "$file")" ]; then

Replace grep -v '^TS ' with grep -vE '^(TS |$)'

[Daniel Nichter (daniel-nichter)]

Thanks for the suggested solution. I'll keep it in mind when fixing bug 1047701--same problem as this bug.

[Daniel Nichter (daniel-nichter)]

What do you think about https://bugs.launchpad.net/percona-toolkit/+bug/1047701 comment #5 Maciek?