[SRU] wrong path to libxml2.so.2 in mod_security - broken by multiarch enabled libraries

[Christoph_vW]

[Impact]

The libapache2-modsecurity and libapache2-mod-proxy-html packages does not work out of the box (but workarounds are available).

[Test Case]

apt-get -y install apache2 <libapache2-modsecurity or libapache2-mod-proxy-html>

This fails with the following error, although the postinst does exit 0:
 Setting up libapache2-modsecurity (2.6.3-1) ...
 Action 'configtest' failed.
 The Apache error log may have more information.
 Your apache2 configuration is broken, so we're not restarting it for you.

$ sudo apachectl configtest
apache2: Syntax error on line 210 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/mod-security.load: Cannot load /usr/lib/libxml2.so.2 into server: /usr/lib/libxml2.so.2: cannot open shared object file: No such file or directory
Action 'configtest' failed.
The Apache error log may have more information.

Expected results:

1. The installations should succeed.
2. "sudo apachectl configtest" should return "Syntax OK" with a zero exit status.
3. "sudo grep libxml2.so.2 /proc/$(cat /run/apache2.pid)/maps" should display load locations of libxml2.so.2.<version>, in order to verify that the runtime linker has successfully linked the required module.

[Fix]

Debian has fixed this by updating apache2 to use dlopen's search path and changing mod-security.load to not use any absolute path. We have merged apache2. modsecurity-apache and mod-proxy-html have synced and I have verified that Quantal is fixed.

For the Precise SRU, it was concluded that the change to apache2 in Debian is too invasive. Instead, we have removed the LoadFile directives entirely, after ensuring that the modules do depend correctly on libxml2.so.2.

[Regression Potential]

With the new approach, apache2 does not need an update.

Previously, libapache2-modsecurity and libapache2-mod-proxy-html did not load at all in the default configuration, so I don't see how there could be a regression here.

We have changed a config file, but since it is a config file, an administrator who has manually worked around the problem by changing the config file differently will be prompted and so should not get an unexpected regression.

/usr/lib/apache2/modules/mod_proxy_html.so now explicitly imports symbols from libxml2.so.2, but this was done by the LoadFile directive anyway, so I don't see that there would be a problem here.

So the area to look for regressions is in the existence of XML functionality in these two modules, but I think this change is so minimal it is very unlikely.

Original bug description:

service apache2 restart
apache2: Syntax error on line 210 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/mod-security.load: Cannot load /usr/lib/libxml2.so.2 into server: /usr/lib/libxml2.so.2: cannot open shared object file: No such file or directory
Action 'configtest' failed.
The Apache error log may have more information.
   ...fail!

in file /etc/apache2/mods-enabled/mod-security.load:
LoadFile /usr/lib/libxml2.so.2

correct path on x86 would be /usr/lib/i386-linux-gnu/libxml2.so.2

maybe a symlink could fix this issue?

[Christoph_vW (christoph-apiviewer)]

Ubuntu 12.04 LTS

[Christoph_vW (christoph-apiviewer)]

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=670247

[James Page (james-page)]

sudo apt-get install apache2 libapache2-modsecurity

reproduced on 12.04:

sudo apachectl configtest
apache2: Syntax error on line 210 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/mod-security.load: Cannot load /usr/lib/libxml2.so.2 into server: /usr/lib/libxml2.so.2: cannot open shared object file: No such file or directory
Action 'configtest' failed.
The Apache error log may have more information.

[James Page (james-page)]

Issue is that this package has not been updated to support the multiarch version of libxml2.

Obviously this can be fixed by updating /etc/apache2/mods-enabled/mod-security.load but it should probably be SRU'ed as well as its a pretty low risk change.

[Hugo.Batel (hugo-batel)]

The same happens on 64 bit ubuntu 12.04.

Creating a symbolic link like the one bellow is a workaround for this problem.

ln -s /usr/lib/x86_64-linux-gnu/libxml2.so.2 /usr/lib/libxml2.so.2

[Robie Basak (racb)]

Debian has fixed this by updating apache2 to use dlopen's search path and changing mod-security.load to not use any absolute path. This is a nice clean fix.

For an SRU, I think our options are:

1) Also SRU apache2 to use the search path as Debian has done, but this seems a bit too major for an SRU.
2a) Change mod-security.load dynamically on postinst, but this will prompt on future upgrade (as it's a conffile).
2b) Use ucf to update mod-security.load dynamically on postinst, but this requires introducing ucf which seems excessive for an SRU.
3) Symlink /usr/lib/libxml2.so.2 on postinst, but this would break m-a for libxml2.

I don't like any of these options. What other fix is possible? Or are one of these acceptable?

[Robie Basak (racb)]

Debian have fixed this by changing both modsecurity-apache and apache2. modsecurity-apache has synced. I have prepared a merge for apache2 which is ready for sponsorship.

For the SRU, please see my questions above.

Thanks!

[Sebastien Bacher (seb128)]

(unsubscribing sponsors, jamespage said it's being tracked)

[Robie Basak (racb)]

I've redone the merge with feedback integrated. I used merges.u.c this time.

Tested build and that it runs on Quantal with this bug fixed.

[James Page (james-page)]

Uploaded to quantal - this resolves the apache2 task for development - marking 'Fix Released'

[Robie Basak (racb)]

SRU debdiffs for apache2 and modsecurity-apache attached.

[Robie Basak (racb)]

I have test built and tested these on Precise to verify that the problem goes away and that Apache still works.

[Clint Byrum (clint-fewbar)]

I'm a little concerned about how deep inside the core of Apache this change is. I think its probably, as you say, a corner case if it causes somebody issues, but we should make sure it actually works as we think it does by testing some of those corner cases.

[Clint Byrum (clint-fewbar)]

Hello Christoph_vW, or anyone else affected,

Accepted apache2 into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Clint Byrum (clint-fewbar)]

Hello Christoph_vW, or anyone else affected,

Accepted modsecurity-apache into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[James Page (james-page)]

sudo apt-get install apache2 libapache2-modsecurity
[....]
Setting up apache2.2-common (2.2.22-1ubuntu1.1) ...
Enabling site default.
Enabling module alias.
Enabling module autoindex.
Enabling module dir.
Enabling module env.
Enabling module mime.
Enabling module negotiation.
Enabling module setenvif.
Enabling module status.
Enabling module auth_basic.
Enabling module deflate.
Enabling module authz_default.
Enabling module authz_user.
Enabling module authz_groupfile.
Enabling module authn_file.
Enabling module authz_host.
Enabling module reqtimeout.
Setting up apache2-mpm-worker (2.2.22-1ubuntu1.1) ...
 * Starting web server apache2 [ OK ]
Setting up apache2 (2.2.22-1ubuntu1.1) ...
Setting up libapache2-modsecurity (2.6.3-1ubuntu0.1) ...
 * Restarting web server apache2 ... waiting . [ OK ]
Setting up ssl-cert (1.0.28ubuntu0.1) ...
Setting up modsecurity-crs (2.2.0-1) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

/var/log/apache2# cat error.log
[Mon Jun 18 12:15:42 2012] [notice] Apache/2.2.22 (Ubuntu) configured -- resuming normal operations
[Mon Jun 18 12:15:43 2012] [notice] caught SIGTERM, shutting down
[Mon Jun 18 12:15:45 2012] [notice] ModSecurity for Apache/2.6.3 (http://www.modsecurity.org/) configured.
[Mon Jun 18 12:15:45 2012] [notice] ModSecurity: APR compiled version="1.4.6"; loaded version="1.4.6"
[Mon Jun 18 12:15:45 2012] [notice] ModSecurity: PCRE compiled version="8.12"; loaded version="8.12 2011-01-15"
[Mon Jun 18 12:15:45 2012] [notice] ModSecurity: LUA compiled version="Lua 5.1"
[Mon Jun 18 12:15:45 2012] [notice] ModSecurity: LIBXML compiled version="2.7.8"
[Mon Jun 18 12:15:46 2012] [notice] Apache/2.2.22 (Ubuntu) configured -- resuming normal operations

[James Page (james-page)]

The specific issue this fix resolves tested OK - but as Clint states this probably needs more thorough testing before acceptance into -updates.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Steve Langasek (vorlon)]

I have serious misgivings about the particular approach taken here, in SRU or otherwise. Why does mod-security.load need to hard-code libxml2.so.2 *at all*? Isn't that why DSOs are dynamically linked to their dependent libraries (which mod_security2.so is)? And why is libxml2.so.2 being special-cased here, when mod_security2.so is also linked against liblua5.1.so.0? Did anyone try to fix this by just removing the LoadFile line completely? That would be a much safer fix than SRUing the core of apache2, and would probably fix the problem that this SRU seems to be stalled waiting for indeterminate regression testing of "corner cases".

A naive test here of commenting out this 'LoadFile' line shows that the apache config validates, and apache2-mpm-prefork starts without any problems.

[Stefan Fritsch (sf-sfritsch)]

There are two schools of thought here. Some people prefer linking to the libraries directly (the saner approach for a Linux distribution) and some people prefer to load depending libraries with LoadFile (to make it easier to avoid loading to different versions of the same library in the same process). Mod_security used to take the second approach. If that has changed, or the packaging modifies it to link to libxml, then simply removing the LoadFile line is the logical thing to do.

[Robie Basak (racb)]

The SRU I proposed just backports the fix that Debian have applied. If we want to do something else in Quantal, then we'd be deviating from Debian on this. Is this worth doing?

Of course, the SRU can be done differently. If commenting out LoadFile works, then perhaps that's all we need to do for the SRU?

[Clint Byrum (clint-fewbar)]

Since the fix in -proposed is in doubt, I've marked the bug verification-failed. We can remove the package from precise-proposed if need be. Otherwise, upload whatever fix is appropriate to proposed.

[Robie Basak (racb)]

Removing the "LoadFile /usr/lib/libxml2.so.2" line from /etc/apache2/mods-available/mod_security2.so works fixes the immediate problem and apache does load OK and serves the root page after this. So this fix is a candidate for a simple SRU to Precise.

However, I am concerned to make sure that mod-security is actually functional after this change, in case it can't resolve symbols further down the line.

Although I note from /proc/../maps that /usr/lib/x86_64-linux-gnu/libxml2.so.2.7.8 has been loaded, I'd like to make sure before proposing a replacement SRU.

I am unfamiliar with mod-security itself. Could someone affected by this bug please post a test case with expected results that exercises the XML functionality in mod-security that would use libxml2.so.2?

[Robie Basak (racb)]

<jamespage> rbasak, I think that the test case can be technical
<jamespage> so for example I think its sufficient to say - yes we can restart apache OK with it installed and we can see that this library has been loaded by using lsof or suchlike

Based on this, my test case for SRU verification will be based on loading apache and checking /proc/.../maps for presence of the modules expected.

[James Page (james-page)]

Marked bug task for apache2 as 'Invalid' as this is being fix in just the modules.

[James Page (james-page)]

Robie - both MP's looked good - uploaded.

[Adam Conrad (adconrad)]

Hello Christoph_vW, or anyone else affected,

Accepted modsecurity-apache into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/modsecurity-apache/2.6.3-1ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Adam Conrad (adconrad)]

Hello Christoph_vW, or anyone else affected,

Accepted mod-proxy-html into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/mod-proxy-html/3.0.1-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Stéphane Graber (stgraber)]

libapache2-modsecurity now installs fine.

[Stéphane Graber (stgraber)]

libapache2-mod-proxy-html also installs fine.

[Stéphane Graber (stgraber)]

The tests above were done without the updated apache2 from -proposed as this one was the previous fix and was now pulled out of -proposed.

Marking verification-done.

[Scott Kitterman (kitterman)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package modsecurity-apache - 2.6.3-1ubuntu0.2

---------------
modsecurity-apache (2.6.3-1ubuntu0.2) precise-proposed; urgency=low

  * debian/mod-security.load: revert previous change, since an alternate
    solution has been agreed in the bug.
  * debian/mod-security.load: drop LoadFile directive entirely, and rely on
    the loader to resolve all required dependencies. This avoids the
    hardcoded path which fails on multiarch-enabled systems (LP: #988819).

modsecurity-apache (2.6.3-1ubuntu0.1) precise-proposed; urgency=low

  * debian/mod-security.load: remove libxml2.so.2 path, since with
    multiarch it is always incorrect on i386 and amd64. Use no path and
    a corresponding apache2 change to use the standard dlopen search
    path in this case to allow the library to be found (LP: #988819).
 -- Robie Basak <email address hidden> Thu, 19 Jul 2012 11:33:40 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package mod-proxy-html - 3.0.1-1ubuntu0.1

---------------
mod-proxy-html (3.0.1-1ubuntu0.1) precise-proposed; urgency=low

  * debian/rules: explicitly link against libxml2.so, so that it gets
    declared as a dependency. This will allow apache to load the module
    without explicitly specifying the correct multiarch path.
  * debian/conf/mod-proxy.load: drop LoadFile directive entirely, and rely
    on the loader to resolve all required dependencies. This avoids the
    hardcoded path which fails on multiarch-enabled systems (LP: #988819).
 -- Robie Basak <email address hidden> Thu, 19 Jul 2012 12:09:11 +0000