Exported YaSSL symbols conflict with OpenSSL, crash Apache

[Kris Karas]

Apache with mod_ssl loaded dies with a SIGSEGV if it has been dynamically linked against MariaDB 5.3.6.
If I downgrade to MariaDB 5.3.5, the problem goes away.

I used "nm -P -s -g /usr/lib64/mysql/libmysqlclient_r.so" from both 5.3.5 and 5.3.6 to get a list of exported symbols, and then did a diff to see what changed. Right at the top of the list, we have the culprit:

 --- /tmp/mariadb-5.3.5.symbols 2012-04-09 17:03:09.916073329 -0400
 +++ /tmp/mariadb-5.3.6.symbols 2012-04-09 17:02:43.412084697 -0400
 @@ -1,3 +1,7 @@
 +CRYPTO_add_lock T
 +CRYPTO_lock T
 +CRYPTO_mem_ctrl T
 +EVP_CIPHER_CTX_init T
  TERMINATE T

These symbols being exported from libmysqlclient_r.so are owned by OpenSSL (libcrypto.so). After Apache loads both libcrypto.so and libmysqlclient, it winds up using the MariaDB versions from YaSSL, which causes ssl_init_ModuleKill() to become very unhappy:

[MariaDB 5.3.5 is currently loaded, and httpd fires up.]
 ~# httpd
 ~# killall httpd
 ~# upgradepkg mariadb-5.3.6-x86_64-1.txz
 +==============================================================================
 | Upgrading mariadb-5.3.5-x86_64-1 package using ./mariadb-5.3.6-x86_64-1.txz
 +==============================================================================
 ~# httpd
 Segmentation fault (core dumped)
 ~# gdb /usr/sbin/httpd core
[snip]
 Core was generated by `httpd'.
 Program terminated with signal 11, Segmentation fault.
 #0 0x00007f1d4fa02c33 in ASN1_primitive_free () from /lib64/libcrypto.so.1
 (gdb) bt
 #0 0x00007f1d4fa02c33 in ASN1_primitive_free () from /lib64/libcrypto.so.1
 #1 0x00007f1d4fa02c57 in ASN1_primitive_free () from /lib64/libcrypto.so.1
 #2 0x00007f1d4fa02fdf in ASN1_template_free () from /lib64/libcrypto.so.1
 #3 0x00007f1d4fa02eda in asn1_item_combine_free () from /lib64/libcrypto.so.1
 #4 0x00007f1d4fa02fdf in ASN1_template_free () from /lib64/libcrypto.so.1
 #5 0x00007f1d4fa02eda in asn1_item_combine_free () from /lib64/libcrypto.so.1
 #6 0x00007f1d4fa03015 in ASN1_item_free () from /lib64/libcrypto.so.1
 #7 0x00007f1d4aee9d1a in ssl_init_ModuleKill () from /usr/libexec/apache/mod_ssl.so
 #8 0x00007f1d4f2b0b8e in apr_pool_clear () from /usr/lib64/libapr-1.so.0
 #9 0x000000000042828d in main ()
 (gdb) quit

Sorry, didn't have a chance to track it down to a particular source file.

[Kristian Nielsen (knielsen)]

Which package is this?
Google mentions Slackware when searching for "upgradepkg" - is this from a
slackware package of mariadb?

I think the root cause of this problem is that MariaDB is linked statically
against YaSSL. It should instead link the system ssl libs dynamically,
eg. with ./configure --with-ssl=/usr (this is what our .deb packages do).

Maybe you can report this upstream?

Still, this crashes with 5.3.6 but not with 5.3.5. In 5.3.6, bundled YaSSL was
upgraded to 2.1.4 - this is a merge from recent MySQL 5.1. I can try to
discuss with some other devevelopers and see if something can be done to avoid
the crash.

But in the end - I think trying to link both static yassl and dynamic openssl
into Apache will cause problems - not just this one. The better fix is to make
your packages link libopenssl.so dynamically.

[Kris Karas (the-ktk)]

This is a largely hand-built Linux distribution that starts off as Slackware and is then heavily modified with the replacement of 50 or so of the major packages with feature-enhanced and/or security-hardened versions. I typically remove vendor patches, and compile unmodified versions directly from upstream authors; Slackware has a better-than-average tendency to omit "vendor patches" anyhow.

The MariaDB "package" is constructed as a standard Slackware package, but is custom built from unmodified sources, configured thusly:

 ./configure --with-ssl --enable-dependency-tracking --enable-assembler --enable-profiling --with-charset=utf8 --with-extra-charsets=all --with-unix-socket-path=/var/mysql/socket --with-mysqld-user=mysql --with-plugins=max-no-ndb --with-readline --without-libedit --with-aria-tmp-tables --docdir=/usr/doc/mariadb-5.3.5 --sysconfdir=/etc --infodir=/usr/info --libdir=/usr/lib64 --mandir=/usr/man --prefix=/usr --localstatedir=/var/mysql

The only thing SSL related is "--with-ssl" allowing the ./configure script to best figure out how to implement that in a compatible way.

When YaSSL is compiled into MySQL/MariaDB statically, its definitions are supposed to be static as well, which prevents collisions when linking against some other SSL implementation. YaSSLs namespace has (up until now) not collided with that of OpenSSL, allowing both to coexist nicely. I've had no issues running MySQL 5.5.2x, for example.

Even if the symbols collide, proper commands to the linker during assembly of the libmysqlclient.so should allow those to be kept private with the shared object, yes? It's been awhile since I read the gory details of the argument list to GNU "ld", but I assume such is possible.

[Kristian Nielsen (knielsen)]

As I said, try --with-ssl=/usr. You could say that should be default, but it doesn't seem a safe change in a stable (5.3) release.

5.5 is built differently (mariadb or mysql both). I think it does not export SSL symbols from the libmysqlclient.so, as you suggest to do. You could try mariadb 5.5.23.

I sympatise with your statement "allowing the ./configure script to best figure out ..." - unfortunately it is not tradition in the MySQL development to focus on good defaults for people compiling themselves.