eol content filters are never loaded and thus never applied

[Brian de Alwis]

[Revised the description and summary; added eol keyword.]

With the integration of the former bzr-eol plugin into bzr's core, the eol content filters are no longer actually loaded. I've verified this on MacOS X and Linux by putting a pdb breakpoint and a print at the top of bzrlib/filters/eol.py, and following the steps at the bottom. Perhaps most telling is that there is no eol.pyc generated.

The eol content filters were previously registered in the former bzr-eol plugin by virtue of the registration being done at the end of its __init__.py.

With the integration of bzr-eol the bzr codebase, the eol content filters are registered when bzrlib/filters/eol.py is loaded. But bzr.filters.eol is never explicitly imported, and bzrlib/filters/__init__.py doesn't dynamically import any of the contents of bzrlib/filters, and so eol.py is never loaded and the eol content filters are never registered.

Actually, bzrlib.filters.eol is explicitly imported in one place, *in the tests*, which is why the tests pass.

Steps to repeat:

0. Verify this is the current bzr-1.14; only diff to this code is a print and pdb.set_trace() in eol.py:

$ uname -a
Linux virtuoso 2.6.24.7-server-2mnb #1 SMP Thu Oct 30 14:50:37 EDT 2008 x86_64 AMD Opteron(tm) Processor 850 GNU/Linux
$ alias bzr='/staff/bsd178/bzr-1.14/bzr --no-plugins -Dfilters'
$ bzr version
Bazaar (bzr) 1.14rc1
  from bzr checkout /tmp_mnt/staff/bsd178/bzr-1.14
    revision: 4249
    revid: <email address hidden>
    branch nick: bzr-1.14
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib64/python2.5
  bzrlib: /tmp_mnt/staff/bsd178/bzr-1.14/bzrlib
  Bazaar configuration: /staff/bsd178/.bazaar
  Bazaar log file: /staff/bsd178/.bzr.log

Copyright 2005, 2006, 2007, 2008, 2009 Canonical Ltd.
http://bazaar-vcs.org/

bzr comes with ABSOLUTELY NO WARRANTY. bzr is free software, and
you may use, modify and redistribute it under the terms of the GNU
General Public License version 2 or later.

$ (cd /staff/bsd178/bzr-1.14/; bzr diff)
=== modified file 'bzrlib/filters/eol.py'
--- bzrlib/filters/eol.py 2009-04-06 22:09:05 +0000
+++ bzrlib/filters/eol.py 2009-04-08 16:35:02 +0000
@@ -19,6 +19,8 @@
 See bzr help eol for details.
 """

+print "==> loaded eol.py"
+import pdb; pdb.set_trace()

 import re, sys

1. Create .bazaar/rules with a rule that should apply to all files and that definitely should cause some content transformation:

$ cat ~/.bazaar/rules
[name *]
eol=lf-with-crlf-in-repo

2. Create a 1.14 format branch

$ bzr init --1.14 /tmp/test-branch
Created a standalone tree (format: 1.14)
$ cd /tmp/test-branch

3. Populate with some LF-terminated content.

$ cat >test1.txt
this
is
a
test
$ cat >test2.txt
son
of
a
gun
$ hexdump -C test*
00000000 74 68 69 73 0a 69 73 0a 61 0a 74 65 73 74 0a 73 |this.is.a.test.s|
00000010 6f 6e 0a 6f 66 0a 61 0a 67 75 6e 0a |on.of.a.gun.|
0000001c
$ bzr add
adding test1.txt
adding test2.txt
$

4. Commit. Note that we previously aliased bzr with -Dfilters, so we should see some output here.

$ bzr commit -m 'First commit'
Committing to: /tmp/test-branch/
added test1.txt
added test2.txt
Committed revision 1.
$ tail ~/.bzr.log
0.051 bzr arguments: [u'--no-plugins', u'-Dfilters', u'commit', u'-m', u'First commit']
0.123 encoding stdout as sys.stdout encoding 'UTF-8'
0.358 opening working tree '/tmp/test-branch'
0.418 preparing to commit
[ 5995] 2009-04-08 10:35:53.674 INFO: Committing to: /tmp/test-branch/
0.437 Selecting files for commit with filter []
[ 5995] 2009-04-08 10:35:53.677 INFO: added test1.txt
[ 5995] 2009-04-08 10:35:53.677 INFO: added test2.txt
[ 5995] 2009-04-08 10:35:53.828 INFO: Committed revision 1.
0.593 return code 0

5. Verify that the files do not have CR-LF in the repo:

$ bzr cat test1.txt | hexdump -C
00000000 74 68 69 73 0a 69 73 0a 61 0a 74 65 73 74 0a |this.is.a.test.|
0000000f
$ bzr cat test2.txt | hexdump -C
00000000 73 6f 6e 0a 6f 66 0a 61 0a 67 75 6e 0a |son.of.a.gun.|
0000000d

6. Try reverting the files, and see there is no change:

$ bzr revert test*
$ hexdump -C test*
00000000 74 68 69 73 0a 69 73 0a 61 0a 74 65 73 74 0a 73 |this.is.a.test.s|
00000010 6f 6e 0a 6f 66 0a 61 0a 67 75 6e 0a |on.of.a.gun.|
0000001c

7. Verify eol was never loaded (file size for eol.py are slightly larger as I added a print and pdb.set_trace() to the top)

$ ls -l /staff/bsd178/bzr-1.14/bzrlib/filters/
total 22
-rw-r--r-- 1 bsd178 wheel 2375 2009-04-08 10:35 eol.py
-rw-r--r-- 1 bsd178 wheel 9170 2009-04-08 09:49 __init__.py
-rw-r--r-- 1 bsd178 wheel 9635 2009-04-08 09:50 __init__.pyc

[Ian Clatworthy (ian-clatworthy)]

A patch has been sent to the mailing list for this. Please apply and retest.

[Brian de Alwis (slyguy)]

That does the trick. Many thanks!

[Bob Tanner (tanner)]

Could you tell me where I can get the bug fix?

I'm attempting to merge this into 1.14rc2.

Normally someone say rXXXX in bzr.dev or makes me a cherrypick branch. I don't see where the fix is located.

Thanks.

[Ian Clatworthy (ian-clatworthy)]

bzr.dev r4278.

[Bob Tanner (tanner)]

Merged into 1.14rc2. Thanks.

[Martin Pool (mbp)]

So it should be fixed in 1.14rc2