pt-archiver --file doesn't create a file

[maxim.volkov]

App: pt-archiver

The combination of flags which do rase the error despite --dry-run passes without a problem:

pt-archiver \
--source h=localhost,D=database,t=table_I,b=1,i=index \
--dest h=localhost,D=database,t=table_II,b=1,i=index \
--columns column1,column2,column3,column4,column5 \
--where "changed_when < '2011-12-01 00:00:00'" --limit 1000 --commit-each \
--nocheck-columns --replace --commit-each --bulk-insert --bulk-delete \
--statistics \
--no-check-charset \
--file '/tmp/mysql/%Y-%m-%d-%D_%H:%i:%s.%t'

Version: 1.0.1-1

Issue: Having --file flag in pt-archiver results in the following error:
• Use of uninitialized value in lc at /usr/bin/pt-archiver line 3871.

3868 # Open the file and print the header to it.
3869 if ( $archive_file ) {
3870 my $need_hdr = $o->get('header') && !-f $archive_file;

3871 my $charset = lc $o->get('charset'); <------ ERROR LINE.

3872 if ( $charset ) {
3873 $archive_fh = IO::File->new($archive_file, ">>:$charset")
3874 or die "Cannot open $charset $archive_file: $OS_ERROR\n";
3875 }

Attempted to by pass problem:
--- added "--charset latin1" flag - hence, the above error line expects its value - got further error:
• Cannot open latin1 /tmp/mysql/2011-12-12-database_12:09:25.table: No such file or directory

[Brian Fraser (fraserbn)]

Ack, IO::File. It should be dying before the "No such file or directory" error: latin1 is not a valid encoding name, and with warnings enabled it should croak with 'Unknown PerlIO layer "latin1"'; the second argument of open expects the canonical name, which is iso-8859-1. Unfortunately, IO::File doesn't have warnings on internally: the best long-term solution, if feasible, would be to change all the IO::File uses with simple opens, but a good intermediate step would be to wrap those calls like this:

{
   local $^W = 1; # $WARNING with use English
   IO::File->new(...);
}

Or maybe wrap the constructor's glob itself, But that seems like a lot of work for too little gain.

In any case, the solution to the base problem is pretty simple: $charset should get a sane default, and also go through Encode::resolve_alias() before being passed to open() or IO::File.

(There's also another possible problem here: if $charset were to somehow be an empty string, you would end up with >>: or similar as the mode, which is magical -- It'll skip fetching context layers set by 'use open qw( yadda )'; might be best to have those colons be part of $charset)

[Rob Smith (kormoc)]

With 2.0.3, I get this problem with this line:

pt-archiver --source h=stuff --where 'created <= current_date - interval 90 day' --charset utf8 --pid /var/run/pt-archiver.pid --txn-size 500 --progress 1000 --sleep 1 --sleep-coef 2 --file '/path/%D/%t/%Y-%m-%dT%H:%i:%s' --why-quit --statistics

Cannot open utf8 '/path/2012-02-09T16:25:26': No such file or directory

Removing the charset, I get

Cannot open '/path/2012-02-09T16:25:26': No such file or directory

This is preventing the tool from working at all for file archiving. Is there any work arounds to get this to run correctly?

[Rob Smith (kormoc)]

I applied the patch from r144 to the version in 2.0.3 and I get

Undefined subroutine &Encode::resolve_alias called at /usr/local/bin/pt-archiver line 3957.

Does this commit depend on a previous commit I haven't found or am I missing some library?

Thanks!

[Brian Fraser (fraserbn)]

Whoops, no, that was me forgetting the most important line! Thank you for following this, Rob. I committed a newer version, think you can try giving that one a spin?

[Rob Smith (kormoc)]

Progress!

Without setting --charset utf8 it works now. Thanks so much!

However when I do set --charset utf8 I get

TIME ELAPSED COUNT
2012-03-07T10:40:21 0 0
IO::Handle: bad open mode: >>utf8 at /usr/local/bin/pt-archiver line 3972

This is on centos 5.4.

Thanks much!

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-444