Crash when using AES_ENCRYPT on empty string

[Brian Franklin]

Server crashes when trying to use AES_ENCRYPT on an empty string.

# uname -a
Linux [hostname] 2.6.32-279.2.1.el6.x86_64 #1 SMP Fri Jul 20 01:55:29 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release
CentOS release 6.3 (Final)

Using Percona-Server-5.6.12-rc60.4-393.Linux.x86_64

From mysqld.log:
=================
evp_enc.c(146): OpenSSL internal error, assertion failed: inl > 0
01:13:15 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=33554432
read_buffer_size=2097152
max_used_connections=7
max_threads=102
thread_count=7
connection_count=7
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1078693 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0xc78bdc0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 2b0e2fdef070 thread_stack 0x30000
/usr/local/mysql-dev/bin/mysqld(my_print_stacktrace+0x35)[0x90e0d5]
/usr/local/mysql-dev/bin/mysqld(handle_fatal_signal+0x3fb)[0x673b7b]
/lib64/libpthread.so.0[0x384f20eca0]
/lib64/libc.so.6(gsignal+0x35)[0x384ea302c5]
/lib64/libc.so.6(abort+0x110)[0x384ea31d70]
/lib64/libcrypto.so.6[0x38512d5f6f]
/lib64/libcrypto.so.6(EVP_EncryptUpdate+0x1f8)[0x38512815c8]
/usr/local/mysql-dev/bin/mysqld(my_aes_encrypt+0xc1)[0x91a9e1]
/usr/local/mysql-dev/bin/mysqld(_ZN21Item_func_aes_encrypt7val_strEP6String+0xbd)[0x6268cd]
/usr/local/mysql-dev/bin/mysqld(_ZN4Item4sendEP8ProtocolP6String+0x53)[0x59de03]
/usr/local/mysql-dev/bin/mysqld(_ZN8Protocol19send_result_set_rowEP4ListI4ItemE+0x92)[0x66e962]
/usr/local/mysql-dev/bin/mysqld(_ZN11select_send9send_dataER4ListI4ItemE+0x6c)[0x6b872c]
/usr/local/mysql-dev/bin/mysqld(_ZN4JOIN4execEv+0x5c8)[0x6d05e8]
/usr/local/mysql-dev/bin/mysqld[0x716539]
/usr/local/mysql-dev/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xbc)[0x71686c]
/usr/local/mysql-dev/bin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x175)[0x716a75]
/usr/local/mysql-dev/bin/mysqld[0x6f1609]
/usr/local/mysql-dev/bin/mysqld(_Z21mysql_execute_commandP3THD+0x40ac)[0x6f6fac]
/usr/local/mysql-dev/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x518)[0x6fa138]
/usr/local/mysql-dev/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x837)[0x6fab07]
/usr/local/mysql-dev/bin/mysqld(_Z10do_commandP3THD+0x117)[0x6fc007]
/usr/local/mysql-dev/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x130)[0x6c5890]
/usr/local/mysql-dev/bin/mysqld(handle_one_connection+0x45)[0x6c5995]
/usr/local/mysql-dev/bin/mysqld(pfs_spawn_thread+0x13b)[0xb3fa0b]
/lib64/libpthread.so.0[0x384f20683d]
/lib64/libc.so.6(clone+0x6d)[0x384ead503d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (c7b3f30): SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL
Connection ID (thread ID): 13
Status: NOT_KILLED
==============================

[Valerii Kravchuk (valerii-kravchuk)]

No crash with 5.6.11-rc60.3, so it seems a recent regression.

[Valerii Kravchuk (valerii-kravchuk)]

No crash on upstream 5.6.12:

openxs@debian7:~/sandboxes/rcsandbox_mysql-5_6_12/node1$ ./use test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

node1 [localhost] {msandbox} (test) > SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| ▌ZЦq÷▄ ╪JyйЗ |
+----------------------------------------------------+
1 row in set (0.04 sec)

[Valerii Kravchuk (valerii-kravchuk)]

No crash with 5.6.12-rc60.4 (32-bit) built from sources:

openxs@debian7:~/sandboxes/msb_5_6_12$ ./use test
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.12-rc60.4 Percona Server with XtraDB (GPL), Release rc60.4, Revision 404

Copyright (c) 2009-2013 Percona Ireland Ltd.
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql [localhost] {msandbox} (test) > SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| ▌ZЦq÷▄ ╪JyйЗ |
+----------------------------------------------------+
1 row in set (0.00 sec)

It seems the problem may be related to specific binaries and Linux version used.

[Brian Franklin (brqan)]

In case it helps, the problem does not exist with MySQL Community Edition 5.6.12 on the same platform/servers.

[Valerii Kravchuk (valerii-kravchuk)]

No crash with recent 32-bit 5.6.12 from repositories on Debian:

openxs@debian7:~/dbs/p5.6$ mysql -uroot -proot test
Warning: Using a password on the command line interface can be insecure.
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 45
Server version: 5.6.12-rc60.4-log Percona Server (GPL), Release 60.4

Copyright (c) 2009-2013 Percona Ireland Ltd.
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| ▌ZЦq÷▄ ╪JyйЗ |
+----------------------------------------------------+
1 row in set (0.06 sec)

mysql> show variables like '%version%';
+-------------------------+------------------------------------+
| Variable_name | Value |
+-------------------------+------------------------------------+
| innodb_version | 5.6.12-rc60.4 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.6.12-rc60.4-log |
| version_comment | Percona Server (GPL), Release 60.4 |
| version_compile_machine | i686 |
| version_compile_os | Linux |
+-------------------------+------------------------------------+
7 rows in set (0.00 sec)

Will try to check on CentOS 6.x x86_64 later tonight or next week.

As this has something to do with openssl, please, send the output of:

ldd `which mysqld`

This is what I have on system where it works:

root@debian7:/home/openxs/dbs/p5.6# ldd `which mysqld`
        linux-gate.so.1 => (0xb77d6000)
        libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xb77b2000)
        libaio.so.1 => /lib/i386-linux-gnu/libaio.so.1 (0xb77af000)
        librt.so.1 => /lib/i386-linux-gnu/i686/cmov/librt.so.1 (0xb77a5000)
        libcrypt.so.1 => /lib/i386-linux-gnu/i686/cmov/libcrypt.so.1 (0xb7773000)
        libssl.so.1.0.0 => /usr/lib/i386-linux-gnu/i686/cmov/libssl.so.1.0.0 (0xb771b000)
        libcrypto.so.1.0.0 => /usr/lib/i386-linux-gnu/i686/cmov/libcrypto.so.1.0.0 (0xb755c000)
        libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0xb7558000)
        libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xb746b000)
        libm.so.6 => /lib/i386-linux-gnu/i686/cmov/libm.so.6 (0xb7445000)
        libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xb7428000)
        libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb72c5000)
        /lib/ld-linux.so.2 (0xb77d7000)
     ...

[Brian Franklin (brqan)]

I couldn't exactly use ldd `which mysqld` because this server has multiple instances. I modified the command to use the Percona mysqld that was being invoked:

# ldd /usr/local/Percona-Server-5.6.12-rc60.4-393.Linux.x86_64/bin/mysqld
        linux-vdso.so.1 => (0x00007fff265fd000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x000000384f200000)
        libaio.so.1 => /usr/lib64/libaio.so.1 (0x0000003c8fa00000)
        librt.so.1 => /lib64/librt.so.1 (0x0000003850600000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003850a00000)
        libssl.so.6 => /lib64/libssl.so.6 (0x0000003852200000)
        libcrypto.so.6 => /lib64/libcrypto.so.6 (0x0000003851200000)
        libdl.so.2 => /lib64/libdl.so.2 (0x000000384ee00000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003851a00000)
        libm.so.6 => /lib64/libm.so.6 (0x000000384fa00000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003851600000)
        libc.so.6 => /lib64/libc.so.6 (0x000000384ea00000)
        /lib64/ld-linux-x86-64.so.2 (0x000000384e600000)
        libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003851e00000)
        libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003854600000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003853600000)
        libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003854a00000)
        libz.so.1 => /lib64/libz.so.1 (0x000000384f600000)
        libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003854e00000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003852e00000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003853200000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003850200000)
        libsepol.so.1 => /lib64/libsepol.so.1 (0x000000384fe00000)

[Brian Franklin (brqan)]

The installed openssl package version is 0.9.8e-22.el5_8.4.

I can try updating to 0.9.8e-26.el5_9.1 to see if that resolves the problem later today.

[Brian Franklin (brqan)]

Updating openssl package to 0.9.8e-26.el5_9.1 showed no improvement. Still crashed.

New ldd output:
        linux-vdso.so.1 => (0x00007fff2adfd000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003fb8600000)
        libaio.so.1 => /usr/lib64/libaio.so.1 (0x00002b23be36a000)
        librt.so.1 => /lib64/librt.so.1 (0x0000003fb9a00000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003fb9e00000)
        libssl.so.6 => /lib64/libssl.so.6 (0x00002b23be56c000)
        libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00002b23be7ba000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000003fb8200000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003fbaa00000)
        libm.so.6 => /lib64/libm.so.6 (0x0000003fb8e00000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003fbb200000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003fb7e00000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003fb7a00000)
        libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003fbae00000)
        libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003fbda00000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003fbce00000)
        libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003fbde00000)
        libz.so.1 => /lib64/libz.so.1 (0x0000003fb8a00000)
        libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0000003fbd600000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x0000003fbc200000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003fbc600000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003fb9200000)
        libsepol.so.1 => /lib64/libsepol.so.1 (0x0000003fb9600000)

[Valerii Kravchuk (valerii-kravchuk)]

If you use CentOS release 6.3, why openssl*el5*? I also would suggest to upgrade to openssl 1.0.x (should be default on CentOS 6.x)

[Valerii Kravchuk (valerii-kravchuk)]

Yet another system (CentOS 6.4 upgraded today) where official binaries from yum repository just works:

[root@centos openxs]# mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.12-56 Percona Server (GPL), Release rc60.4, Revision 393

Copyright (c) 2009-2013 Percona Ireland Ltd.
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| ▌ZЦq÷▄ ╪JyйЗ |
+----------------------------------------------------+
1 row in set (0.00 sec)

mysql> exit
Bye
[root@centos openxs]# ldd `which mysqld`
        linux-vdso.so.1 => (0x00007fff45cb0000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa8fa20e000)
        libaio.so.1 => /lib64/libaio.so.1 (0x00007fa8fa00d000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fa8f9e04000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fa8f9bcd000)
        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007fa8f9970000)
        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007fa8f95d5000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fa8f93d1000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fa8f90cb000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fa8f8e46000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa8f8c30000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fa8f889d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa8fa432000)
        libfreebl3.so => /lib64/libfreebl3.so (0x00007fa8f863a000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007fa8f83f6000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fa8f8110000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fa8f7f0b000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007fa8f7cdf000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fa8f7ac9000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007fa8f78bd000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007fa8f76ba000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fa8f74a0000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fa8f7280000)
[root@centos openxs]# uname -a
Linux centos 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@centos openxs]# rpm -q -a | grep openssl
openssl-devel-1.0.0-27.el6_4.2.x86_64
openssl-1.0.0-27.el6_4.2.x86_64

[Brian Franklin (brqan)]

That's a good question (about the el5 packages). I just checked and all of the packages and yum repositories are CentOS 5-based, despite it being CentOS 6. This is a dedicated server, so I am checking with the provider.

I am going to try to sort out that discrepancy before upgrading to openssl 1.x. As soon as I can, I will let you know whether that solves the problem.

In the meantime, does it make sense that MySQL 5.6.12 does not exhibit the problem versus Percona server?

[Valerii Kravchuk (valerii-kravchuk)]

Just apply ldd to mysqld binary of MySQL 5.6.12 and check if it points to the same openssl shared library at all.

Oracle binaries may be statically linked with yassl or some openssl version (had not checked, sorry, but it used to be the case about yassl for a long time).

[Brian Franklin (brqan)]

Here is the output for mysql 5.6.12:

# ldd /usr/local/mysql-5.6.12-linux-glibc2.5-x86_64/bin/mysqld
        linux-vdso.so.1 => (0x00007fff7fdfd000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003fb8600000)
        libaio.so.1 => /usr/lib64/libaio.so.1 (0x00002acd44779000)
        librt.so.1 => /lib64/librt.so.1 (0x0000003fb9a00000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x0000003fb9e00000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000003fb8200000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003fbaa00000)
        libm.so.6 => /lib64/libm.so.6 (0x0000003fb8e00000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003fbb200000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003fb7e00000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003fb7a00000)

[Valerii Kravchuk (valerii-kravchuk)]

As you can see, there is no link to openssl library, like in your case:

        libssl.so.6 => /lib64/libssl.so.6 (0x00002b23be56c000)
        libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00002b23be7ba000)

or in my:

        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007fa8f9970000)
        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007fa8f95d5000)

So, they statically link SSL library (whatever it is). In Percona it was decided some time ago to use OS versions of libraries as much as possible. Check https://bugs.launchpad.net/percona-server/+bug/1108016.

What you see is the result of this work in progress. Not all results so far are positive, especially when environment is somewhat weird, like in your case.

[Brian Franklin (brqan)]

I see.

After looking into this, it appears that I must have pasted in the wrong info for # cat /etc/redhat-release (probably a result of comparing to one of our other servers at the time and the end of a long day).

I really apologize for the confusion and any time wasted on checking 6.x. The original post should contain the following:

# uname -a
Linux [hostname] 2.6.18-308.1.1.el5 #1 SMP Wed Mar 7 04:16:51 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release
CentOS release 5.9 (Final)

Would this mean that Percona 5.6 requires at least CentOS 6? The packages looks like they are updated as far as they will go on 5.9.

[Valerii Kravchuk (valerii-kravchuk)]

We have to check. I do not have CentOS 5.x at hand, so no more updates from me today.

[Nilnandan Joshi (nilnandan-joshi)]

Confirmed with PS 5.6.12 on CentOS 5.9

[root@dhcppc0 ~]# mysql -umsandbox -p --socket=/tmp/mysql_sandbox5612.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.12-rc60.4 Percona Server with XtraDB (GPL), Release rc60.4, Revision 393

...
mysql>
mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 1
Current database: *** NONE ***

ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
mysql> quit

2013-07-17 14:09:11 18988 [Note] /root/5.6.12/bin/mysqld: ready for connections.
Version: '5.6.12-rc60.4' socket: '/tmp/mysql_sandbox5612.sock' port: 5612 Percona Server with XtraDB (GPL), Release rc60.4, Revision 393
evp_enc.c(146): OpenSSL internal error, assertion failed: inl > 0
08:39:13 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=153
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 69119 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x15590d10
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 2b903df60070 thread_stack 0x40000
/root/5.6.12/bin/mysqld(my_print_stacktrace+0x35)[0x90e0d5]
/root/5.6.12/bin/mysqld(handle_fatal_signal+0x3fb)[0x673b7b]
/lib64/libpthread.so.0[0x32a460eca0]
/lib64/libc.so.6(gsignal+0x35)[0x32a36302c5]
/lib64/libc.so.6(abort+0x110)[0x32a3631d70]
/lib64/libcrypto.so.6[0x32aecd5f6f]
/lib64/libcrypto.so.6(EVP_EncryptUpdate+0x1f8)[0x32aec815c8]
/root/5.6.12/bin/mysqld(my_aes_encrypt+0xc1)[0x91a9e1]
/root/5.6.12/bin/mysqld(_ZN21Item_func_aes_encrypt7val_strEP6String+0xbd)[0x6268cd]
/root/5.6.12/bin/mysqld(_ZN4Item4sendEP8ProtocolP6String+0x53)[0x59de03]
/root/5.6.12/bin/mysqld(_ZN8Protocol19send_result_set_rowEP4ListI4ItemE+0x92)[0x66e962]
/root/5.6.12/bin/mysqld(_ZN11select_send9send_dataER4ListI4ItemE+0x6c)[0x6b872c]
/root/5.6.12/bin/mysqld(_ZN4JOIN4execEv+0x5c8)[0x6d05e8]
/root/5.6.12/bin/mysqld[0x716539]
/root/5.6.12/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_unitP13st_select_lex+0xbc)[0x71686c]
/root/5.6.12/bin/mysqld(_Z13handle_selectP3THDP13select_result...

[Nilnandan Joshi (nilnandan-joshi)]

There is no crash with latest version on 5.1 (5.1.70) and 5.5 (5.5.32)

[root@dhcppc0 ~]# mysql -umsandbox -p --socket=/tmp/mysql_sandbox5170.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.1.70rel14.8 Percona Server with XtraDB (GPL), Release rel14.8, Revision 580

...
mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| Z�q�� �Jy�� |
+----------------------------------------------------+
1 row in set (0.00 sec)

mysql> quit
Bye

[root@dhcppc0 ~]# mysql -umsandbox -p --socket=/tmp/mysql_sandbox5532.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.32-rel31.0 Percona Server with XtraDB (GPL), Release rel31.0, Revision 549

...
mysql> SELECT AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') FROM DUAL;
+----------------------------------------------------+
| AES_ENCRYPT('','6b2d440b57b3bae74e3cf71e415b3965') |
+----------------------------------------------------+
| Z�q�� �Jy�� |
+----------------------------------------------------+
1 row in set (0.00 sec)

mysql>

[Roel Van de Paar (roel11)]

[56qual validated on discussion w/ Laurynas]

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-689