Ubuntu
apache2 package

Apache2 magic mime file is broken (2.4.52-1ubuntu4.2)

Bug #2003709 reported by Eli Barzilay on 2023-01-23

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	apache2 (Ubuntu)	New	Undecided	Unassigned
	Jammy	Incomplete	Undecided	Unassigned

Bug Description

The default `/etc/apache2/magic` file that comes with apache2 seems to be broken.

I don't know what exactly is the problem, but here are a few details that demonstrate that it is indeed broken:

* echo -e "<!DOCTYPE HTML>\nblah blah" > /tmp/foo

* file -m /etc/apache2/magic /tmp/foo
=> Shows a bunch of "string modifier `B' invalid" warnings
and finally: "could not find any valid magic files!"

* Trying to fetch this file via apache for a configuration that has "a2enmod mime_magic" results in no Content-Type header

* Now, get the original magic file from https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
  drop it in the same place:
  => the `file` command now works as expected,
  => fetching the file via apache has the expected Content-Type header

* Alternatively, modify the original magic file by removing all "B" modifiers
(on lines 187, 402, 403, 404, 405, 622, 627, 631, 632)
=> both the `file` command and the header from apache work

---

version information:

$ lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04

$ apt-cache policy apache2
apache2:
  Installed: 2.4.52-1ubuntu4.2
  Candidate: 2.4.52-1ubuntu4.2
  Version table:
*** 2.4.52-1ubuntu4.2 500
        500 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.4.52-1ubuntu4.1 500
        500 http://us.archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     2.4.52-1ubuntu4 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages

Revision history for this message

Lucas Kanashiro (lucaskanashiro) wrote on 2023-01-24:

Thanks for reporting this bug and trying to make Ubuntu better.

I tried to reproduce what you described in a Ubuntu Jammy container:

root@apache2-jammy:~# dpkg -l apache2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-=================-============-============================>
ii apache2 2.4.52-1ubuntu4.2 amd64 Apache HTTP Server
root@apache2-jammy:~# cat /tmp/foo
<!DOCTYPE HTML>
blah blah
root@apache2-jammy:~# file -m /etc/apache2/magic /tmp/foo
/etc/apache2/magic, 187: Warning: string modifier `B' invalid
/etc/apache2/magic, 402: Warning: string modifier `B' invalid
/etc/apache2/magic, 619: Warning: string modifier `B' invalid
/etc/apache2/magic, 624: Warning: string modifier `B' invalid
/etc/apache2/magic, 628: Warning: string modifier `B' invalid
/etc/apache2/magic, 629: Warning: string modifier `B' invalid
file: could not find any valid magic files!
root@apache2-jammy:~# wget https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
--2023-01-24 20:29:34-- https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13064 (13K) [text/plain]
Saving to: ‘magic’

magic 100%[==================>] 12.76K --.-KB/s in 0.001s

2023-01-24 20:29:35 (9.45 MB/s) - ‘magic’ saved [13064/13064]

root@apache2-jammy:~# file -m ./magic /tmp/foo
/tmp/foo: text/html

The file shipped in the Ubuntu package diverges from the upstream one. This requires extra investigation to understand why this was done. Adding this bug to our backlog.

Thanks for reporting this bug and trying to make Ubuntu better.

I tried to reproduce what you described in a Ubuntu Jammy container:

root@apache2-jammy:~# dpkg -l apache2
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version           Architecture Description
+++-==============-=================-============-============================>
ii  apache2        2.4.52-1ubuntu4.2 amd64        Apache HTTP Server
root@apache2-jammy:~# cat /tmp/foo 
<!DOCTYPE HTML>
blah blah
root@apache2-jammy:~# file -m /etc/apache2/magic /tmp/foo 
/etc/apache2/magic, 187: Warning: string modifier `B' invalid
/etc/apache2/magic, 402: Warning: string modifier `B' invalid
/etc/apache2/magic, 619: Warning: string modifier `B' invalid
/etc/apache2/magic, 624: Warning: string modifier `B' invalid
/etc/apache2/magic, 628: Warning: string modifier `B' invalid
/etc/apache2/magic, 629: Warning: string modifier `B' invalid
file: could not find any valid magic files!
root@apache2-jammy:~# wget https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
--2023-01-24 20:29:34--  https://raw.githubusercontent.com/apache/httpd/trunk/docs/conf/magic
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13064 (13K) [text/plain]
Saving to: ‘magic’

magic               100%[==================>]  12.76K  --.-KB/s    in 0.001s

2023-01-24 20:29:35 (9.45 MB/s) - ‘magic’ saved [13064/13064]

root@apache2-jammy:~# file -m ./magic /tmp/foo
/tmp/foo: text/html

The file shipped in the Ubuntu package diverges from the upstream one. This requires extra investigation to understand why this was done. Adding this bug to our backlog.

Changed in apache2 (Ubuntu Jammy):
status:	New → Triaged

Revision history for this message

Eli Barzilay (elibarzilay) wrote on 2023-01-25:

Yeah, the file is very different, and also much bigger. I couldn't find which variant of `file` supports those B modifiers...

Revision history for this message

Bryce Harrington (bryce) wrote on 2023-02-08:

Debian appears to provide its own magic file (debian/config-dir/magic), which has been shipped since 2.2.12-1, and gets installed to /etc/apache2/magic. That release includes this changelog entry about it:

* Ship our own version of the magic config file (taken from file 4.17-5etch3)
which is still compatible with mod_mime_magic (closes: #483111).

apache2.README.Debian mentions it:

   magic
        Patterns for mod_mime_magic. This is not compatible with the format
        used by current versions of the file/libmagic packages.

LP: #601501 is a wishlist bug to switch to using shared-mime-info. mime-info is different from mime-magic, however the bug includes this comment relating to magic:

Apache's mod_mime_magic does not support all features that are used in /usr/share/mime/magic. Using that file would result in error messages. And from a quick glance at libmagic, it doesn't seem possible to use it in mod_mime_magic either because it is not thread safe. Therefore apache brings its own magic file.

So, sounds like it's kind of expected that file would be broken, but presumably the broken content-type is a legit bug. Although since the file has apparently been there since Apache 2.2, it seems weird this would crop up as a problem now. Did this use to work for you pre-jammy? Testing on bionic, I don't see a content-type listed there either.

    $ curl -i http://10.69.244.198/foo | less
      % Total % Received % Xferd Average Speed Time Time Time Current
                                     Dload Upload Total Spent Left Speed
    100 33 100 33 0 0 28896 0 --:--:-- --:--:-- --:--:-- 33000
    HTTP/1.1 200 OK
    Date: Wed, 08 Feb 2023 03:32:54 GMT
    Server: Apache/2.4.29 (Ubuntu)
    Last-Modified: Wed, 08 Feb 2023 03:31:42 GMT
    ETag: "21-5f427e7279e93"
    Accept-Ranges: bytes
    Content-Length: 33

<!DOCTYPE HTML>
blah blah bionic

It seems the same regardless of whether module mime_magic is enabled

    # a2enmod mime_magic
    # systemctl restart apache2
    # apache2ctl -M | grep magic
     mime_magic_module (shared)
    # curl -i http://10.69.244.198/foo | less

Can you explain in more detail how you got apache2 to include the content-type header from the magic config info, for extensionless files?

I can get content-type to detect based on extension, e.g. index.html and such, but getting it to work via magic I'm not reproducing.

Debian appears to provide its own magic file (debian/config-dir/magic), which has been shipped since 2.2.12-1, and gets installed to /etc/apache2/magic.  That release includes this changelog entry about it:

* Ship our own version of the magic config file (taken from file 4.17-5etch3)
        which is still compatible with mod_mime_magic (closes: #483111).

apache2.README.Debian mentions it:

magic
        Patterns for mod_mime_magic. This is not compatible with the format
        used by current versions of the file/libmagic packages.

LP: #601501 is a wishlist bug to switch to using shared-mime-info.  mime-info is different from mime-magic, however the bug includes this comment relating to magic:

So, sounds like it's kind of expected that file would be broken, but presumably the broken content-type is a legit bug.  Although since the file has apparently been there since Apache 2.2, it seems weird this would crop up as a problem now.  Did this use to work for you pre-jammy?  Testing on bionic, I don't see a content-type listed there either.

$ curl -i http://10.69.244.198/foo | less
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100    33  100    33    0     0  28896      0 --:--:-- --:--:-- --:--:-- 33000
    HTTP/1.1 200 OK
    Date: Wed, 08 Feb 2023 03:32:54 GMT
    Server: Apache/2.4.29 (Ubuntu)
    Last-Modified: Wed, 08 Feb 2023 03:31:42 GMT
    ETag: "21-5f427e7279e93"
    Accept-Ranges: bytes
    Content-Length: 33

<!DOCTYPE HTML>
    blah blah bionic

It seems the same regardless of whether module mime_magic is enabled

# a2enmod mime_magic
    # systemctl restart apache2
    # apache2ctl -M | grep magic
     mime_magic_module (shared)
    # curl -i http://10.69.244.198/foo | less

Can you explain in more detail how you got apache2 to include the content-type header from the magic config info, for extensionless files?

I can get content-type to detect based on extension, e.g. index.html and such, but getting it to work via magic I'm not reproducing.

Bryce Harrington (bryce) on 2023-04-17

Changed in apache2 (Ubuntu Jammy):
status:	Triaged → Incomplete

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntuapache2 package

Apache2 magic mime file is broken (2.4.52-1ubuntu4.2)

Bug Description

Other bug subscribers

Remote bug watches

Ubuntu
apache2 package