file incorrectly labeled as Erlang JAM file (OOo does not print on Tuesdays)

[jgallo]

Binary package hint: file

anon@x-X:~$ echo "1/2 Tue" >> file && file file
file: Jan 22 14:32:44 MET 1991\011Erlang JAM file - version 4.2

anon@x-X:~$ file --version
file-4.21
magic file from /etc/magic:/usr/share/file/magic

If you look at the magic file:
4 string Tue Jan 22 14:32:44 MET 1991 Erlang JAM file - version 4.2

So at the fourth byte of the file, it's looking for Tue Jan 22 14:32:44 MET 1991
Since the spaces aren't escaped, it sees that the file contains "Tue" at the fourth byte
and determines it to be an Erlang JAM file.

I've tried it with file-4.21, file-4.23, file-4.24.

In 4.21 and 4.24, the bug exists, but 4.23 correctly determines the file to be ASCII text.

This was on Ubuntu 8.04.1.

This bug was introduced in file 4.20-4 (after dapper), and fixed in Debian file 5.01-1 by escaping the spaces in the relevant magic entries. Patches applicable to stable releases are here:

  http://launchpadlibrarian.net/27497360/hardy.diff
  http://launchpadlibrarian.net/27497364/intrepid.diff
  http://launchpadlibrarian.net/27497372/jaunty.diff

TEST CASE: Download http://launchpadlibrarian.net/26055361/WillNotPrintOnTuesday.ps and run 'file' on it. Incorrect output looks like "WillNotPrintOnTuesday.ps: Jan 22 14:32:44 MET 1991\011Erlang JAM file - version 4.2"; correct output looks like "WillNotPrintOnTuesday.ps: PostScript document text conforming DSC level 3.0, Level 3".

REGRESSION POTENTIAL: It's worth trying out printing in general, since this is how this bug came to prominence, and if anyone actually has some real Erlang JAM files to try out then that wouldn't hurt, as well as perhaps running 'file' on some random things. Otherwise I don't anticipate much in the way of regression potential here.

[Chris Cheney (ccheney)]

This also apparently can cause PostScript files to be misidentified as Erlang JAM files which in turn causes them not to be able to be printed as noted in bug 255161.

[Steve (paddy-stevepaine)]

This bug is causing printing using the brother cups drivers to fail on Tuesdays. I am attaching a sample PostScript file which file 4.21 misidentifies.

4.21-3 seems to be the current version in the Ubuntu repository.

[Colin Watson (cjwatson)]

This was fixed in version 5.01-1 in Debian; I've confirmed that Karmic no longer suffers from this bug.

file (5.01-1) unstable; urgency=low

  * Adding patch from Adam Buchbinder <email address hidden> to
    fix false matches against Z-machine pattern (Closes: #499748).
  * Adding patch from Adam Buchbinder <email address hidden> to
    improve XWD magic in order to not give false results on mp3 files
    (Closes: #511764).
  * Adding patch from Adam Buchbinder <email address hidden> to fix
    unescaped spaces in erlang magic (Closes: #514056).
  * Updating UUID patches to cope with leading zeroes, thanks to Bjorn
    Mork <email address hidden> (Closes: #515019).
  * Updating section for python-magic-dbg.
  * Adding patch from Adam Buchbinder <email address hidden> to fix
    a spacing error in the manpage (Closes: #515761).
  * Adding patch from Adam Buchbinder <email address hidden> to
    updated utf-8 big-endian magic (Closes: #513526).
  * Adding patch from Adam Buchbinder <email address hidden> to add
    new magic for git packs and indexes (Closes: #509942).
  * Adding patch from Adam Buchbinder <email address hidden> to
    update magic for spectrum tap files (Closes: #501589).
  * Merging upstream version 5.01.
  * Removing magic-update-erlang.patch, went upstream.
  * Rediffing magic-add-qdbm.patch.
  * Rediffing magic-add-tokyocabinet.patch.
  * Manually renaming magic directory in rules to correct name.
  * Updating file-mgc.patch for file 5, produces raw magics again now
    (Closes: #522433).

 -- Daniel Baumann <email address hidden> Sun, 03 May 2009 11:02:00 +0200

[Colin Watson (cjwatson)]

I've uploaded a fix for this to all of hardy-proposed, intrepid-proposed, and jaunty-proposed.

[Martin Pitt (pitti)]

Accepted file into jaunty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Accepted file into hardy-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Accepted file into intrepid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Steve Beattie (sbeattie)]

I have reproduced the file misidentification of the Postscript file as an Erlang JAM file with the version of file in hardy-updates, 4.21-3ubuntu1, and have verified that the version in hardy-proposed correctly identifies it as a Postscript file. I was unable to reproduce the printing issue, probably because I'm using a different Brother printer (HL-5250DN); however, printing continued to function post-upgrade.

I installed most of the erlang packages along with other software that was implemented in erlang (there's not much in the archive), and did a wholesale comparision of file output on the majority of the filesystem (a nominally stock hardy desktop install). Out of 102507 files examined, I had one false-positive, /usr/lib/erlang/lib/typer-0.1.0.1/ebin/typer.appup from the erlang-nox package, which gets mis-detected as ASCII text (it's not clearto me that file's detection of typer.appup as a JAM file is correct, either). The only other detected differences were 3 changelog files that had been mis-identified originally as JAM files that were correctly identified post-upgrade. Fortunately, true JAM files are likely to be uncommon, as the community has (mostly?) moved on the BEAM virtual machine (which is what our erlang packages support).

[Colin Watson (cjwatson)]

This sounds pretty acceptable to me. Thanks for the testing! Is that enough to consider this verification-done for 8.04.3 purposes? I realise you couldn't strictly reproduce the original problem ...

[Steve Beattie (sbeattie)]

On Wed, Jun 17, 2009 at 03:37:48PM -0000, Colin Watson wrote:
> This sounds pretty acceptable to me. Thanks for the testing! Is that
> enough to consider this verification-done for 8.04.3 purposes? I realise
> you couldn't strictly reproduce the original problem ...

Yes, I consider it verification-done for hardy; I didn't tag it as such
because I haven't gotten to testing it on intrepid and jaunty yet.
Thanks.

--
Steve Beattie
<email address hidden>
http://NxNW.org/~steve/

[Steve Langasek (vorlon)]

I'm content to treat this as verified.

[Launchpad Janitor (janitor)]

This bug was fixed in the package file - 4.21-3ubuntu2

---------------
file (4.21-3ubuntu2) hardy-proposed; urgency=low

  * 307-magic-add-erlang.dpatch: Escape spaces in Erlang magic to avoid
    misidentifications, particularly PostScript files produced on Tuesdays
    (LP: #248619).

 -- Colin Watson <email address hidden> Thu, 04 Jun 2009 13:12:19 +0100

[Martin Pitt (pitti)]

Setting back to v-needed for intrepid/jaunty.

[Steve Beattie (sbeattie)]

I have reproduced the file misidentification of the Postscript file as
an Erlang JAM file with the versions in intrepid, 4.24-4, and jaunty,
4.26-2ubuntu3, and have confirmed that the versions in
intrepid-proposed, 4.24-4ubuntu1, and jaunty-proposed, 4.26-2ubuntu4,
correctly identifies it as a Postscript file. Again, I am unable to
reproduce the printing issue as before.

I also again attempted to look for regressions by comparing the output
of wholesale file output. In the intrepid version, the only change was
again some changelogs that had been mis-identified as before, and
curiously /var/log/syslog, which went from 'ASCII text' to 'ASCII Pascal
program text'. However, there was a DHCP lease renewal that occurred
between the two runs reported in /var/log/syslog which may have thrown
things off. Running both versions against the current contents result in
both reporting it to be 'ASCII English text'. So I don't believe there
any regressions here.

I also performed a similar check against the jaunty versions, with 1.5 million files on a live system. This had several files that the original version incorrectly identified as Erlang JAM files that were correctly identified by the version in jaunty-proposed. No new mis-identitifications were introduced by the package upgrade.

Based on this, I believe these are okay to release, and am marking verification-done. Thanks.

[Launchpad Janitor (janitor)]

This bug was fixed in the package file - 4.24-4ubuntu1

---------------
file (4.24-4ubuntu1) intrepid-proposed; urgency=low

  * 213-magic-update-erlang.dpatch: Escape spaces in Erlang magic to avoid
    misidentifications, particularly PostScript files produced on Tuesdays
    (LP: #248619).

 -- Colin Watson <email address hidden> Thu, 04 Jun 2009 13:16:12 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package file - 4.26-2ubuntu4

---------------
file (4.26-2ubuntu4) jaunty-proposed; urgency=low

  * 220-magic-update-erlang.dpatch: Escape spaces in Erlang magic to avoid
    misidentifications, particularly PostScript files produced on Tuesdays
    (LP: #248619).

 -- Colin Watson <email address hidden> Thu, 04 Jun 2009 12:29:05 +0100

[kelups (engelup)]

I had the same problem called the "tuesday bug" under 255161 with Suse 11.1, and solved it as suggested in that report. However, I was able to correctly print as root. So it seems to me that there ist in effect a problem with the users rights.
Kurt