--config files don't support quoted values

Bug #1012673 reported by Maciej Dobrzanski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Toolkit moved to https://jira.percona.com/projects/PT
Confirmed
Undecided
Unassigned

Bug Description

It seems that when columns-regex is set through a config file, pt-diskstat loses the outermost parts of the regular expression. Examples:

(garfield:~) macko% cat ~/.pt-diskstats.conf
group-by=all
columns-regex='rd_s|rd_avgkb|rd_rt|rd_cnc|wr_s|wr_avgkb|wr_rt|wr_cnc|busy|qtime'
(garfield:~) macko% pt-diskstats --devices-regex='sda$' 2012_06_13_11_42_48-diskstats | head -2
  #ts device rd_cnc rd_rt wr_s wr_cnc wr_rt busy
  1.0 sda 2.2 12.6 0.0 0.0 0.0 100%
^C

(garfield:~) macko% cat ~/.pt-diskstats.conf
group-by=all
columns-regex='rd_s|rd_avgkb|rd_rt|rd_cnc|wr_s|wr_avgkb|wr_rt|wr_cnc|busy|qtime|'
(garfield:~) macko% pt-diskstats --devices-regex='sda$' 2012_06_13_11_42_48-diskstats | head -2
  #ts device rd_cnc rd_rt wr_s wr_cnc wr_rt busy qtime
  1.0 sda 2.2 12.6 0.0 0.0 0.0 100% 13.5
^C

(garfield:~) macko% cat ~/.pt-diskstats.conf
group-by=all
columns-regex='rd_s|rd_avgkb|rd_rt|rd_cnc|wr_s|wr_avgkb|wr_rt|wr_cnc|busy|qtime|stime'
(garfield:~) macko% pt-diskstats --devices-regex='sda$' 2012_06_13_11_42_48-diskstats | head -2
  #ts device rd_cnc rd_rt wr_s wr_cnc wr_rt busy qtime
  1.0 sda 2.2 12.6 0.0 0.0 0.0 100% 13.5
^C

(garfield:~) macko% cat ~/.pt-diskstats.conf
group-by=all
columns-regex='rd_s|rd_avgkb|rd_rt|rd_cnc|wr_s|wr_avgkb|wr_rt|wr_cnc|busy|qtime|stime|whatever'
(garfield:~) macko% pt-diskstats --devices-regex='sda$' 2012_06_13_11_42_48-diskstats | head -2
  #ts device rd_cnc rd_rt wr_s wr_cnc wr_rt busy qtime stime
  1.0 sda 2.2 12.6 0.0 0.0 0.0 100% 13.5 5.7

In every case 'rd_s' as well as the last column listed is missing in the output. This does not happen when columns-regex is specified in the command line.

Revision history for this message
Baron Schwartz (baron-xaprb) wrote :

The problem is that the quotes enclosing the option are considered to be part of the value itself. The real bug here is that we don't support leading or trailing whitespace, which could be considered a feature, not a bug. We'd need to understand quotes in order to do that. As it is, quotes are just characters after the = sign, nothing more.

I'll mark this as a documentation bug and we can fix it in the man page that applies to option parsing.

Changed in percona-toolkit:
status: New → Confirmed
summary: - pt-diskstats loses part of columns-regex when set in a config file
+ Configuration files don't support quoted values
Changed in percona-toolkit:
importance: Undecided → Low
tags: added: docs
Revision history for this message
Baron Schwartz (baron-xaprb) wrote : Re: Configuration files don't support quoted values

The offending code snippet in OptionParser.pm, which trims leading and trailing whitespace but doesn't understand quotes:

1139 sub _read_config_file {
1140 my ( $self, $filename ) = @_;
1141 open my $fh, "<", $filename or die "Cannot open $1142 my @args;
1143 my $prefix = '--';
1144 my $parse = 1;
1145
1146 LINE:
1147 while ( my $line = <$fh> ) {
1148 chomp $line;
1149 # Skip comments and empty lines
1150 next LINE if $line =~ m/^\s*(?:\#|\;|$)/;
1151 # Remove inline comments
1152 $line =~ s/\s+#.*$//g;
1153 # Remove whitespace
1154 $line =~ s/^\s+|\s+$//g;

Brian Fraser (fraserbn)
Changed in percona-toolkit:
assignee: nobody → Daniel Nichter (daniel-nichter)
tags: added: all-tools config-file value-quoting
removed: docs
Changed in percona-toolkit:
assignee: Daniel Nichter (daniel-nichter) → nobody
importance: Low → Undecided
summary: - Configuration files don't support quoted values
+ --config files don't support quoted values
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-991

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.