TypeError: argument of type 'dbm.dbm' is not iterable

[Benjamin Rister]

While attempting to migrate a 2GB svn dump file into a bzr repository, I received the following:

(bdr@Fortinbras:Downloads)$ ./svn2bzr/svn2bzr.py --log=debug synk-dump bzr/
Revision 0 read
debug: Tree has 0 entries
Revision 1 read
debug: Tree has 615 entries
...
Revision 161 read
debug: Tree has 2691 entries
Revision 162 read
debug: Tree has 2691 entries
Traceback (most recent call last):
  File "./svn2bzr/svn2bzr.py", line 1083, in <module>
    main()
  File "./svn2bzr/svn2bzr.py", line 1076, in main
    opts.prefix, opts.filter)
  File "./svn2bzr/svn2bzr.py", line 996, in svn2bzr
    dump = Dump(dump_file)
  File "./svn2bzr/svn2bzr.py", line 662, in __init__
    self._read()
  File "./svn2bzr/svn2bzr.py", line 947, in _read
    building=True)
  File "./svn2bzr/svn2bzr.py", line 800, in _change_tree
    copy_tree = self._build_tree(copy_revno)
  File "./svn2bzr/svn2bzr.py", line 683, in _build_tree
    if str(revno) in self._tree_cache:
TypeError: argument of type 'dbm.dbm' is not iterable
Exception exceptions.OSError: (2, 'No such file or directory', '/var/folders/xt/xttFFi+eGkmb+EbV0V4r6E+++TI/-Tmp-/tmpN2cqiF-saved-trees') in <bound method Dump.__del__ of <__main__.Dump object at 0x61d0d0>> ignored

I've tried this with the following branches of svn2bzr, all with the same results:
http://bzr.labix.org/svn2bzr/trunk
http://bazaar.launchpad.net/%7Eniemeyer/svn2bzr/trunk
http://bazaar.launchpad.net/%7Ejamesh/svn2bzr/trunk

(bdr@Fortinbras:Downloads)$ bzr --version
Bazaar (bzr) 1.5
  Python interpreter: /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python 2.5.1
  Python standard library: /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
  bzrlib: /Library/Python/2.5/site-packages/bzr-1.5-py2.5-macosx-10.5-i386.egg/bzrlib
  Bazaar configuration: /Users/bdr/.bazaar
  Bazaar log file: /Users/bdr/.bzr.log

Much like bug #187891, I'm afraid I can't really post the dump publicly, but also am more than happy to do work here to assist with diagnosis, and could possibly privately give access to a partial dump file if necessary.

[Benjamin Rister (bdrister)]

I'm a total Python novice, but some poking and Googling makes me think that this might be helpful information:

>>> import dbhash
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/dbhash.py", line 5, in <module>
    import bsddb
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/bsddb/__init__.py", line 51, in <module>
    import _bsddb
ImportError: No module named _bsddb
>>> import gdbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named gdbm
>>> import dbm
>>>

Are one of the more advanced dbm modules required for svn2bzr to function properly?

[Benjamin Rister (bdrister)]

There's a possible fix in my branch; it may not be applicable to other installs if more advanced dbm modules are indeed required for svn2bzr, but it's stopped the crash for me at least.

[Walter Mundt (walter-okcupid)]

I ran into this, and tried installing py-gdbm to see if that would help. This simply changes the error to the following:

Traceback (most recent call last):
  File "svn2bzr/svn2bzr.py", line 1083, in <module>
    main()
  File "svn2bzr/svn2bzr.py", line 1076, in main
    opts.prefix, opts.filter)
  File "svn2bzr/svn2bzr.py", line 996, in svn2bzr
    dump = Dump(dump_file)
  File "svn2bzr/svn2bzr.py", line 662, in __init__
    self._read()
  File "svn2bzr/svn2bzr.py", line 947, in _read
    building=True)
  File "svn2bzr/svn2bzr.py", line 800, in _change_tree
    copy_tree = self._build_tree(copy_revno)
  File "svn2bzr/svn2bzr.py", line 683, in _build_tree
    if str(revno) in self._tree_cache:
TypeError: argument of type 'gdbm.gdbm' is not iterable

Going to try the version with the proposed fix now.

[Ian Brandt (ian-ianbrandt)]

I got this as well followed by a "No such file or directory" error...

$ ~/Development/svn2bzr/svn2bzr.py --scheme=trunk ~/Desktop/svn-dump-20081230/project.dump project
Revision 0 read
Revision 1 read
Revision 2 read
Revision 3 read
Revision 4 read
Revision 5 read
Revision 6 read
Revision 7 read
Revision 8 read
Revision 9 read
Revision 10 read
Revision 11 read
Traceback (most recent call last):
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 1083, in <module>
    main()
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 1076, in main
    opts.prefix, opts.filter)
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 996, in svn2bzr
    dump = Dump(dump_file)
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 662, in __init__
    self._read()
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 947, in _read
    building=True)
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 800, in _change_tree
    copy_tree = self._build_tree(copy_revno)
  File "/Users/ibrandt/Development/svn2bzr/svn2bzr.py", line 683, in _build_tree
    if str(revno) in self._tree_cache:
TypeError: argument of type 'dbm.dbm' is not iterable
Exception exceptions.OSError: (2, 'No such file or directory', '/var/folders/6i/6ifayoUiFc0MlyhgrKZggE+++TI/-Tmp-/tmprP185X-saved-trees') in <bound method Dump.__del__ of <__main__.Dump object at 0x5d9670>> ignored

The bdrister branch got me past the TypeError, but not the No such file error:

$ ~/Development/svn2bzr-bdrister/svn2bzr.py --scheme=trunk ~/Desktop/svn-dump-20081230/project.dump project
[...]
Committing revision 1218
Exception exceptions.OSError: (2, 'No such file or directory', '/var/folders/6i/6ifayoUiFc0MlyhgrKZggE+++TI/-Tmp-/tmpKCYM9Y-saved-trees') in <bound method Dump.__del__ of <__main__.Dump object at 0x5da670>> ignored

Oddly enough 1217 was the last revision read...

[...]
Revision 1217 read
Nothing changed in revision 1
Committing revision 2
[...]

Is this a separate bug?

[Ian Brandt (ian-ianbrandt)]

As an update I've filed my 'No such file or directory' error as a separate bug: https://bugs.launchpad.net/svn2bzr/+bug/312800.

[Martin Spacek (mspacek)]

I've merged Benjamin's fix into trunk. Yes, it looks like dbm.dbm isn't directly iterable, although other dbm implementations are. Just as a note, on all of my Python installs (all 2.5) on win32, cygwin, and Ubuntu, dbhash is importable. This is apparently the module anydbm prefers to choose.