Random DBD::mysql::db selectrow_array failed: fetch() without execute() error

[Juan]

hello,

We've been using ec2-consisten-snapshot from some time now on multiple servers. Las couple of week we started to get this error:

ec2-consistent-snapshot: Fri Jul 30 00:00:42 2010: MySQL flush & lock DBD::mysql::db selectrow_hashref failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 243.
DBD::mysql::db selectrow_array failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 250.
ec2-consistent-snapshot: Fri Jul 30 00:00:42 2010: sync

I tried to look in the mysql DBD and DBI bug database but nothing reported there.

We run it this way:

ec2-consistent-snapshot --debug \
--aws-access-key-id $KEY0
--aws-secret-access-key $KEY1
--region eu-west-1 \
--description $DESCRIPTION \
--xfs-filesystem /ebsmysql \
--mysql \
--mysql-host localhost \
--mysql-username root \
--mysql-password $DBPASS \
--snapshot-timeout 10 \
--lock-timeout 0.5 \
--lock-tries 60 \
--lock-sleep 5 \
$VOLUMEID

Any recomendation workaround?

Thanks in advance!

Juan

[Eric Hammond (esh)]

Please upgrade to the latest ec2-consistent-snapshot and provide the exact error message if it continues to happen. I'd like to see the correct line numbers where it occurs.

[Juan (juan-brein)]

We have updated to the 0.32 version last week and unfortunatelly it happened again this week couple of times.

The error is:

ec2-consistent-snapshot: Trying again in 5 seconds
ec2-consistent-snapshot: Tue Aug 24 00:00:33 2010: MySQL flush & lock
DBD::mysql::db selectrow_hashref failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 246.
DBD::mysql::db selectrow_array failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 253.
ec2-consistent-snapshot: Tue Aug 24 00:00:33 2010: sync

The code is:

241
242 my ($mysql_logfile, $mysql_position,
243 $mysql_binlog_do_db, $mysql_binlog_ignore_db);
244 if ( not $Noaction ) {
245 # This might be a slave database already
246 my $slave_status = $mysql_dbh->selectrow_hashref(q{ SHOW SLAVE STATUS });
247 $mysql_logfile = $slave_status->{Master_Log_File};
248 $mysql_position = $slave_status->{Read_Master_Log_Pos};
249 $mysql_binlog_do_db = $slave_status->{Replicate_Do_DB};
250 $mysql_binlog_ignore_db = $slave_status->{Replicate_Ignore_DB};
251
252 # or this might be the master
253 ($mysql_logfile, $mysql_position,
254 $mysql_binlog_do_db, $mysql_binlog_ignore_db) =
255 $mysql_dbh->selectrow_array(q{ SHOW MASTER STATUS })
256 unless $mysql_logfile;
257 }
258

Pleae let me know on anything I can help

Juan

[MatthiasP (mpdude)]

Same here. With version 0.40, the message is

DBD::mysql::db selectrow_hashref failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 270.

I'm running it in quiet mode (nightly cron job), so I don't have any other output right now. Snapshot has been taken despite this.

[MatthiasP (mpdude)]

Oh, line 270 is:

my $slave_status = $mysql_dbh->selectrow_hashref(q{ SHOW SLAVE STATUS });

... which was reported above as line 246 in v0.32.

[Jerry Boonstra (jerby)]

i believe this bug is more severe than as rated. The failure "DBD::mysql::db selectrow_hashref failed: fetch() without execute()" happens when trying to get the binlog position from "SHOW SLAVE STATUS". If it fails, a binlog position file cannot be written, and the snapshot is basically corrupt. At minimum, the script should exit with a non-zero code at this point so a corrupt snapshot is not generated. Otherwise, there is pollution of good snapshots mixed in with invalid ones (since this problem is intermittent and appears to be related to database load).

[Dale Morin (5kle)]

This is part of the output from the snapshot process:

DBD::mysql::db selectrow_hashref failed: fetch() without execute() at /usr/bin/ec2-consistent-snapshot line 305.
ec2-consistent-snapshot: master_log_file="Waiting for master to send event", master_log_pos=mysql4.mustanginternetservices.com

This is the latest ec2-consistent -snapshot version 0.44. yes, the server is busy, there were several "MySQL timeout at 0.5 sec" messages prior to the failure.

Line 305 contains:

    my $slave_status = $mysql_dbh->selectrow_hashref(q{ SHOW SLAVE STATUS });

in version 0.44.