CHLD signals causing connection lost

[John Gravatt]

I have encountered what I believe is a bug from a Perl script on CentOS 7 across multiple Percona versions. I cannot reproduce from the mysql cmd line utility that comes with Percona nor when using the MariaDB that comes with in distro.

In all tests I am using perl-DBD-mysql rpm that comes with the distro:
perl-DBD-MySQL-4.023-5.el7.x86_64

The test script:
================
#!/usr/bin/env perl

use Modern::Perl;
use DBI;

my $dbh = DBI->connect('dbi:mysql:host=127.0.0.1');
$dbh->{RaiseError} = 1;
$dbh->{PrintError} = 0;

$SIG{CHLD} = \&catch_child;

my $sql = qq{
    select sleep(30)
};
say "my pid=$$";
$dbh->do($sql);
say "query done";

sub catch_child {
    say "in catch_child()";
}

======================
Versions and results:
======================

$ rpm -qa | grep Percona
Percona-Server-shared-56-5.6.26-rel74.0.el7.x86_64
Percona-Server-client-56-5.6.26-rel74.0.el7.x86_64
Percona-Server-server-56-5.6.26-rel74.0.el7.x86_64

<Terminal A>
-----------
$ ./junk.pl
my pid=11161
DBD::mysql::db do failed: Lost connection to MySQL server during query at ./junk.pl line 16.
in catch_child()

<Terminal B>
------------
$ kill -CHLD 11098
$ kill -CHLD 11098
$ kill -CHLD 11098
-bash: kill: (11098) - No such process

======================

$ rpm -qa | grep Percona
Percona-Server-shared-56-5.6.29-rel76.2.el7.x86_64
Percona-Server-client-56-5.6.29-rel76.2.el7.x86_64
Percona-Server-server-56-5.6.29-rel76.2.el7.x86_64

<Terminal A>
-----------
$ ./junk.pl
my pid=11693
DBD::mysql::db do failed: Lost connection to MySQL server during query at ./junk.pl line 16.
in catch_child()

<Terminal B>
------------
$ kill -CHLD 11693
$ kill -CHLD 11693
$ kill -CHLD 11693
-bash: kill: (11693) - No such process

======================

$ rpm -qa | grep Percona
Percona-Server-shared-57-5.7.11-4.1.el7.x86_64
Percona-Server-client-57-5.7.11-4.1.el7.x86_64
Percona-Server-server-57-5.7.11-4.1.el7.x86_64

<Terminal A>
-----------
$ ./junk.pl
my pid=13601
DBD::mysql::db do failed: Lost connection to MySQL server during query at ./junk.pl line 16.
in catch_child()

<Terminal B>
------------
$ kill -CHLD 13601
$ kill -CHLD 13601
$ kill -CHLD 13601
-bash: kill: (13601) - No such process

======================

$ rpm -qa | grep mariadb
mariadb-5.5.47-1.el7_2.x86_64
mariadb-libs-5.5.47-1.el7_2.x86_64
mariadb-server-5.5.47-1.el7_2.x86_64

<Terminal A>
-----------
$ ./junk.pl
my pid=15541
in catch_child()
query done

<Terminal B>
------------
$ kill -CHLD 15541
$ kill -CHLD 15541
$ kill -CHLD 15541
$ kill -CHLD 15541
$ kill -CHLD 15541
....

[John Gravatt (john-gravatt)]

Just to clarify the four scenarios above.....

====

Vendor: Percona
rpm version: 56-5.6.26-rel74.0.el7.x86_64
mysql cmd line: ok
perl script: fail

====

Vendor: Percona
rpm version: 56-5.6.29-rel76.2.el7.x86_64
mysql cmd line: ok
perl script: fail

====

Vendor: Percona
rpm version: 57-5.7.11-4.1.el7.x86_64
mysql cmd line: ok
perl script: fail

====

Vendor: MariaDB
rpm version: 5.5.47-1.el7_2.x86_64
mysql cmd line: ok
perl script: ok

[John Gravatt (john-gravatt)]

This appears to be related to this bug: https://bugs.launchpad.net/percona-server/+bug/1055641?comments=all

[John Gravatt (john-gravatt)]

Here is a workaround where I wrapped the execute() with signal blocking. This worked perfectly in my testing...

======

#!/usr/bin/env perl

use Modern::Perl;
use DBI;
use POSIX qw(signal_h);

use vars qw($OldSig);

my $dbh = DBI->connect('dbi:mysql:host=127.0.0.1;database=mysql');
$dbh->{RaiseError} = 1;
$dbh->{PrintError} = 0;

$SIG{CHLD} = \&catch_child;
say "my pid: $$";

my $sql = qq{
        select sleep(30);
};
my $sth = $dbh->prepare($sql);

block_signals();
$sth->execute;
unblock_signals();

while(my @aref = $sth->fetchrow_array) {
        sleep 1;
}

say "query done";

##############

sub block_signals {

        say "blocking signals";

        $OldSig = POSIX::SigSet->new;

        my $newsig = POSIX::SigSet->new;
        $newsig->fillset;

        sigprocmask(SIG_BLOCK, $newsig, $OldSig);
}

sub unblock_signals {

        say "unblocking signals";

        sigprocmask(SIG_SETMASK, $OldSig);
}

sub catch_child {

        say "in catch_child()";
}

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3435