Apache2 magic mime file is broken (2.4.52-1ubuntu4.2)

Bug #2003709 reported by Eli Barzilay
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apache2 (Ubuntu)

Bug Description

The default `/etc/apache2/magic` file that comes with apache2 seems to be broken.

I don't know what exactly is the problem, but here are a few details that demonstrate that it is indeed broken:

* echo -e "<!DOCTYPE HTML>\nblah blah" > /tmp/foo

* file -m /etc/apache2/magic /tmp/foo
  => Shows a bunch of "string modifier `B' invalid" warnings
     and finally: "could not find any valid magic files!"

* Trying to fetch this file via apache for a configuration that has "a2enmod mime_magic" results in no Content-Type header

* Now, get the original magic file from https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
  drop it in the same place:
  => the `file` command now works as expected,
  => fetching the file via apache has the expected Content-Type header

* Alternatively, modify the original magic file by removing all "B" modifiers
  (on lines 187, 402, 403, 404, 405, 622, 627, 631, 632)
  => both the `file` command and the header from apache work


version information:

$ lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04

$ apt-cache policy apache2
  Installed: 2.4.52-1ubuntu4.2
  Candidate: 2.4.52-1ubuntu4.2
  Version table:
 *** 2.4.52-1ubuntu4.2 500
        500 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.4.52-1ubuntu4.1 500
        500 http://us.archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     2.4.52-1ubuntu4 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

Thanks for reporting this bug and trying to make Ubuntu better.

I tried to reproduce what you described in a Ubuntu Jammy container:

root@apache2-jammy:~# dpkg -l apache2
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
ii apache2 2.4.52-1ubuntu4.2 amd64 Apache HTTP Server
root@apache2-jammy:~# cat /tmp/foo
blah blah
root@apache2-jammy:~# file -m /etc/apache2/magic /tmp/foo
/etc/apache2/magic, 187: Warning: string modifier `B' invalid
/etc/apache2/magic, 402: Warning: string modifier `B' invalid
/etc/apache2/magic, 619: Warning: string modifier `B' invalid
/etc/apache2/magic, 624: Warning: string modifier `B' invalid
/etc/apache2/magic, 628: Warning: string modifier `B' invalid
/etc/apache2/magic, 629: Warning: string modifier `B' invalid
file: could not find any valid magic files!
root@apache2-jammy:~# wget https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
--2023-01-24 20:29:34-- https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
Resolving raw.githubusercontent.com (raw.githubusercontent.com)...,,, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)||:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13064 (13K) [text/plain]
Saving to: ‘magic’

magic 100%[==================>] 12.76K --.-KB/s in 0.001s

2023-01-24 20:29:35 (9.45 MB/s) - ‘magic’ saved [13064/13064]

root@apache2-jammy:~# file -m ./magic /tmp/foo
/tmp/foo: text/html

The file shipped in the Ubuntu package diverges from the upstream one. This requires extra investigation to understand why this was done. Adding this bug to our backlog.

Changed in apache2 (Ubuntu Jammy):
status: New → Triaged
Revision history for this message
Eli Barzilay (elibarzilay) wrote :

Yeah, the file is very different, and also much bigger. I couldn't find which variant of `file` supports those B modifiers...

Revision history for this message
Bryce Harrington (bryce) wrote :

Debian appears to provide its own magic file (debian/config-dir/magic), which has been shipped since 2.2.12-1, and gets installed to /etc/apache2/magic. That release includes this changelog entry about it:

      * Ship our own version of the magic config file (taken from file 4.17-5etch3)
        which is still compatible with mod_mime_magic (closes: #483111).

apache2.README.Debian mentions it:

        Patterns for mod_mime_magic. This is not compatible with the format
        used by current versions of the file/libmagic packages.

LP: #601501 is a wishlist bug to switch to using shared-mime-info. mime-info is different from mime-magic, however the bug includes this comment relating to magic:

    Apache's mod_mime_magic does not support all features that are used in /usr/share/mime/magic. Using that file would result in error messages. And from a quick glance at libmagic, it doesn't seem possible to use it in mod_mime_magic either because it is not thread safe. Therefore apache brings its own magic file.

So, sounds like it's kind of expected that file would be broken, but presumably the broken content-type is a legit bug. Although since the file has apparently been there since Apache 2.2, it seems weird this would crop up as a problem now. Did this use to work for you pre-jammy? Testing on bionic, I don't see a content-type listed there either.

    $ curl -i | less
      % Total % Received % Xferd Average Speed Time Time Time Current
                                     Dload Upload Total Spent Left Speed
    100 33 100 33 0 0 28896 0 --:--:-- --:--:-- --:--:-- 33000
    HTTP/1.1 200 OK
    Date: Wed, 08 Feb 2023 03:32:54 GMT
    Server: Apache/2.4.29 (Ubuntu)
    Last-Modified: Wed, 08 Feb 2023 03:31:42 GMT
    ETag: "21-5f427e7279e93"
    Accept-Ranges: bytes
    Content-Length: 33

    blah blah bionic

It seems the same regardless of whether module mime_magic is enabled

    # a2enmod mime_magic
    # systemctl restart apache2
    # apache2ctl -M | grep magic
     mime_magic_module (shared)
    # curl -i | less

Can you explain in more detail how you got apache2 to include the content-type header from the magic config info, for extensionless files?

I can get content-type to detect based on extension, e.g. index.html and such, but getting it to work via magic I'm not reproducing.

Bryce Harrington (bryce)
Changed in apache2 (Ubuntu Jammy):
status: Triaged → Incomplete
Revision history for this message
Athos Ribeiro (athos-ribeiro) wrote :

Since the issue also applies for the current devel release (mm), I am applying the same status for it here.

Changed in apache2 (Ubuntu):
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for apache2 (Ubuntu) because there has been no activity for 60 days.]

Changed in apache2 (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for apache2 (Ubuntu Jammy) because there has been no activity for 60 days.]

Changed in apache2 (Ubuntu Jammy):
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.