use of mod_deflate can break XMLENT

[James Fournie]

I have backported the mod_deflate patch to 1.6.1 by copying the dojo folders and relevant apache config lines to a 1.6.1 install. It works well for the most part, except for certain files such as /openils/var/web/xul/server/cat/marc_view.html which report errors like this:

XMLENT XML Parse Error: not well-formed (invalid token) at line 1: parsing /openils/var/web/xul/rel_1_6_1_2/server/cat/marc_view.html: data \x1f\x8b\b, referer: http://catalogue.bclibraries.ca/xul/rel_1_6_1_2/server/cat/z3950.xul

All of them have the \x1f\x8b which I believe is a gzip code.

I have also seen errors on /openils/var/web/conify/global/permission/grp_tree.html and the slimpac

~J

[Dan Scott (denials)]

I haven't been able to reproduce this bug in the Evergreen 2.0 alpha 3 release (virtual image built on Debian Lenny). I tried several different entry points to "View MARC" - from the "Import via Z39.50" interface, as well as the two different entry points from the MARC Editor.

I'm not sure what the dojo folders have to do with r17593 - it looks like a pure eg_vhost.conf patch?

Closing the bug as I can't reproduce it in the release for which it is intended.

If you want to try and backport this to 1.6.1, please post a complete eg.conf / eg_vhost.conf configuration file to the dev list, as it's not clear what other elements of your local configuration might be interacting with these changes in particular. I do know that when I attempted to enable mod_deflate in an earlier 1.6 era, I saw similar problems that you described and quickly ended the experiment.

[Dan Scott (denials)]

s/Debian Lenny/Debian Squeeze/ for the virtual image in the prior comment - sorry.

[Jason Etheridge (phasefx)]

Incidentally, Mike just backported this: http://svn.open-ils.org/trac/ILS/changeset/18120

I asked him about these errors (which I've seen in Debian Lenny) and he said that he saw them with older apache's when the deflated content was served over SSL.

[James Fournie (jfournie)]

I have modified our test box and replaced our eg.conf and eg_vhost.conf with the stock configs from rel_1_6_1 (only difference is I have commented out Listen 443 to support Ubuntu and changed the SSL cert path). The problem persists.

Box is running Hardy, apache is the latest from aptitude, apache2-prefork-dev 2.2.8-1ubuntu0.18

[James Fournie (jfournie)]

I did some research starting from Mike/Jason's premise that it is an apache bug. Here is a changelog for the Ubuntu package:

http://changelogs.ubuntu.com/changelogs/pool/main/a/apache2/apache2_2.2.14-5ubuntu8.2/changelog

There are a bunch of mod_deflate related patches in there, so I wouldn't be surprised if one of them fixes it.

Hardy uses apache 2.2.8, so assuming that 2.2.8 is broken, it looks like the oldest possible patch mentioning mod_deflate is 2.2.11 (jaunty), or it could be one of the patches in 2.2.12 (Karmic), so basically this means that either Jaunty or Karmic is the cutoff for using mod_deflate, unless you're willing to install a backported apache.

Debian's changelogs are completely different, but it looks to me like there are a lot more mod_deflate patches going on there, so probably having the latest version on lenny should be adequate.

Does this sound reasonable? Can anyone confirm this further based on their version of apache, or have an idea of which patch it would be in the changelog?

[James Fournie (jfournie)]

Alright, further update, I have also managed to test this on a Debian install with apache package 2.2.9-10+lenny8, can anyone else confirm if they have it working on this version?

[James Fournie (jfournie)]

Should add that it doesn't seem to work on 2.2.9-10+lenny8

[Jeff Davis (jdavis-sitka)]

I can replicate this problem with a stock Evergreen 2.0alpha4 install on Ubuntu Karmic with apache 2.2.12-1ubuntu2.3. After enabling mod_deflate, I get this error in the apache logs:

XMLENT XML Parse Error: not well-formed (invalid token) at line 1: parsing /openils/var/web/opac/extras/slimpac/start.html: data \x1f\x8b\b

[Dan Scott (denials)]

As Ubuntu Hardy is based on Debian Lenny, and Ubuntu Lucid is based on a snapshot of Debian Squeeze, I believe that's why Hardy/Lenny show this problem, while Lucid/Squeeze do not.

As we now have the mod_deflate directives in the 1.6 branch, I suggest we investigate the use of the IfVersion directive (http://httpd.apache.org/docs/current/mod/mod_version.html#ifversion) to prevent mod_deflate from being used if the version of Apache in use is less than 2.2.14 (version shipped with Lucid).

Basically, wrap:

<IfVersion >= 2.2.14>
</IfVersion>

around any of the:

<IfModule mod_deflate.c>
...
</IfModule>

sections in eg_vhost.conf.

[Dan Scott (denials)]

My mistake - I have now reproduced this problem on Debian Squeeze with Apache 2.2.16. The problem was hidden from me during previous testing because Makefile.install currently disables mod_deflate (duh). Once I enabled mod_deflate, I got the same error just trying to use the SlimPAC as was reported by James in the initial bug.

On the other hand, on Fedora 13 with Apache 2.2.15, the configuration works.

Until we're sure that this works on every system where mod_deflate is enabled, I think we should revert this patch or comment out the pertinent lines.

Aside: to test whether mod_deflate is enabled:

/usr/sbin/apache2ctl -t -D DUMP_MODULES | grep deflate

[Thomas Berezansky (tsbere)]

This is my attempt at fixing the problem by disabling gzip output for all xmlent filtered files:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/tsbere/xmlent_hates_deflate

[Mike Rylander (mrylander)]

Merged to master and 2.1. Let mod_deflate work where it can.