Apache DBD Auth not working with mysql

[Andreas Moehrlein]

[Impact]
The MySQL dbd driver fails to load into apache. This causes apache2 to fail to start if it's configured in this way.

Since MySQL 8.0.2[1], the my_init() function is no longer exported, and it's not expected that clients will call it.

Confirming that, the current build logs for apr-util show:
/home/ubuntu/deb/apr-util/apr-util-1.6.1/dbd/apr_dbd_mysql.c:1267:5: warning: implicit declaration of function ‘my_init’; did you mean ‘mysql_init’? [-Wimplicit-function-declaration]
 1267 | my_init();
      | ^~~~~~~
      | mysql_init

Furthermore, they also show[2] that loading the mysql driver failed, but that doesn't cause the build to fail (unknown reason: not addressed in this update):

(...)
Loaded pgsql driver OK.
Failed to open pgsql[]
Failed to load driver file apr_dbd_mysql.so <<<<<<<
Loaded sqlite3 driver OK.
(...)

The fix is to not call my_init(). This was confirmed with MySQL upstream.

[Test Case]
* install and enable the dbd authentication module
sudo apt update
sudo apt install apache2 libaprutil1-dbd-mysql -y
sudo a2enmod authn_dbd
sudo systemctl restart apache2

* confirm apache is still working:
curl localhost

* now enable the mysql driver
echo DBDriver mysql | sudo tee /etc/apache2/conf-available/dbd_mysql.conf
sudo a2enconf dbd_mysql

* confirm error when restarting apache
sudo systemctl restart apache2

With the fixed packages, the restart above works just fine.

[Regression Potential]
A deprecated initialization function isn't being called anymore. This SRU isn't testing actual authentication using MySQL, just that the module now loads. There could be specific details in apr-utils that were being handled by my_init() before, and now aren't.

[Other Info]
Patch wasn't upstreamed because it should better check when to call my_init() and when not, i.e., check the mysql version properly and decide.

[Original Description]
Using Apache DBD auth for MySQL server 8 causes an error in Focal 20.04
Same setup is working fine in 16.04 and 18.04

The following line in Apache2/conf-enabled/ causes the error:
DBDriver mysql

apache start failed:
apachectl[1188]: Can't load driver file apr_dbd_mysql.so

The file exists in
/usr/lib/x86_64-linux-gnu/apr-util-1/apr_dbd_mysql.so

linking the file to
/lib/apache2/modules/
doesn't change the error message

1. https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-2.html
2. https://launchpadlibrarian.net/436958095/buildlog_ubuntu-eoan-amd64.apr-util_1.6.1-4build1_BUILDING.txt.gz

[Paride Legovini (paride)]

Thanks Andreas for this report. Apparently the same problem happens in Eoan too. The minimal steps to reproduce are:

1. start a fresh LXD container
2. apt update
3. apt install apache2 libaprutil1-dbd-mysql
4. a2enmod authn_dbd (also enables the dbd module)
5. systemctl restart apache2
6. curl localhost => It works!
7. echo DBDriver mysql > /etc/apache2/conf-available/dbd_mysql.conf
8. a2enconf dbd_mysql
9. systemctl reload apache2

This last reload fails on Eoan and Focal with error:

apachectl[2695]: AH00526: Syntax error on line 1 of /etc/apache2/conf-enabled/dbd_mysql.conf:
apachectl[2695]: Can't load driver file apr_dbd_mysql.so
apachectl[2695]: Action 'graceful' failed.

but succeeds on Bionic.

[Dirk Hohndel (dirk-3)]

This is still present six weeks later in the 20.04 daily build.
Running this with strace seems to indicate that indeed the library is successfully loaded:

openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/apr-util-1/apr_dbd_mysql-1.so", O_RDONLY|O_CLOEXEC) = 7
read(7, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20(\0\0\0\0\0\0"..., 832) = 832
fstat(7, {st_mode=S_IFREG|0644, st_size=27288, ...}) = 0
mmap(NULL, 29208, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f90a4122000
mmap(0x7f90a4124000, 12288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x2000) = 0x7f90a4124000
mmap(0x7f90a4127000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x5000) = 0x7f90a4127000
mmap(0x7f90a4128000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x5000) = 0x7f90a4128000
close(7) = 0
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 7
fstat(7, {st_mode=S_IFREG|0644, st_size=78933, ...}) = 0
mmap(NULL, 78933, PROT_READ, MAP_PRIVATE, 7, 0) = 0x7f90a410e000
close(7) = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libmysqlclient.so.21", O_RDONLY|O_CLOEXEC) = 7
read(7, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \222\2\0\0\0\0\0"..., 832) = 832
fstat(7, {st_mode=S_IFREG|0644, st_size=7256008, ...}) = 0
mmap(NULL, 7279960, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f90a3a1c000
mprotect(0x7f90a3a43000, 3997696, PROT_NONE) = 0
mmap(0x7f90a3a43000, 942080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x27000) = 0x7f90a3a43000
mmap(0x7f90a3b29000, 3051520, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x10d000) = 0x7f90a3b29000
mmap(0x7f90a3e13000, 3100672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x3f6000) = 0x7f90a3e13000
mmap(0x7f90a4108000, 21848, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f90a4108000
close(7) = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 7
read(7, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\341\t\0\0\0\0\0"..., 832) = 832
fstat(7, {st_mode=S_IFREG|0644, st_size=1952928, ...}) = 0
mmap(NULL, 1968128, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f90a383b000
mprotect(0x7f90a38d1000, 1286144, PROT_NONE) = 0
mmap(0x7f90a38d1000, 983040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x96000) = 0x7f90a38d1000
mmap(0x7f90a39c1000, 299008, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x186000) = 0x7f90a39c1000
mmap(0x7f90a3a0b000, 57344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x1cf000) = 0x7f90a3a0b000
mmap(0x7f90a3a19000, 10240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f90a3a19000
close(7) = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 7
read(7, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3405\0\0\0\0\0\0"..., 832) = 832
fstat(7, {st_mode=S_IFREG|0644, st_size=104984, ...}) = 0
mmap(NULL, 107600, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f90a3820000
mmap(0x7f90a3823000, 73728, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x...

[Andreas Hasenack (ahasenack)]

The build logs show this:
/home/ubuntu/deb/apr-util/apr-util-1.6.1/dbd/apr_dbd_mysql.c:1267:5: warning: implicit declaration of function ‘my_init’; did you mean ‘mysql_init’? [-Wimplicit-function-declaration]
 1267 | my_init();
      | ^~~~~~~
      | mysql_init

Checking the mysql 8 release notes at https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-2.html we see:
"""
The my_init() function is no longer included in the list of symbols exported from libmysqlclient. It need not be called explicitly by client programs because it is called implicitly by other C API initialization functions.
"""

In the dbd/apr_dbd_mysql.c code, it is called like this:
static void dbd_mysql_init(apr_pool_t *pool)
{
#if MYSQL_VERSION_ID < 100000
    my_init();
#endif
    mysql_thread_init();

[Andreas Hasenack (ahasenack)]

Dropping that my_init() call fixed the loading of the module for me. I'll publish a ppa for people to test.

[Andreas Hasenack (ahasenack)]

PPA with builds for eoan and focal: https://launchpad.net/~ahasenack/+archive/ubuntu/mysql8-my-init/

Note: focal build does not have proposed enabled.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apr-util - 1.6.1-4ubuntu2

---------------
apr-util (1.6.1-4ubuntu2) focal; urgency=medium

  * d/p/mysql8-my_init.patch: don't call my_init() since it's not
    exported anymore since MySQL 8.0.2 (LP: #1859773)

 -- Andreas Hasenack <email address hidden> Wed, 04 Mar 2020 16:51:27 -0300

[Brian Murray (brian-murray)]

Hello Andreas, or anyone else affected,

Accepted apr-util into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apr-util/1.6.1-4ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Haven Hash (havenster)]

Tested the 1.6.1-4ubuntu0.1 version and it worked for me. After upgrading to 19.10 and newer MySQL, the apr_dbd_mysql.so would not load. All configuration, permissions, apache and apr-dbd packages seemed to be the same. Found this bug and did a one off install of the apr and apr-mysql .debs from proposed and everything works again.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apr-util - 1.6.1-4ubuntu0.1

---------------
apr-util (1.6.1-4ubuntu0.1) eoan; urgency=medium

  * d/p/mysql8-my_init.patch: don't call my_init() since it's not
    exported anymore since MySQL 8.0.2 (LP: #1859773)

 -- Andreas Hasenack <email address hidden> Wed, 04 Mar 2020 17:00:31 -0300

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for apr-util has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.