NoneType has no attribute st_mode in _readdir_pyx.UTF8DirReader.read_dir

[Stewart Smith]

Committing to: /home/stewart/drizzle/static-functions/
aborting commit write group: AttributeError("'NoneType' object has no attribute 'st_mode'",)
bzr: ERROR: exceptions.AttributeError: 'NoneType' object has no attribute 'st_mode'

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    attaching the crash file
        /home/stewart/.cache/crash/bzr-20091008054128-29728.crash
    and including a description of the problem.

    The crash file is plain text and you can inspect or edit it to remove
    private information.

[Stewart Smith (stewart)]

(the commit worked the second time)

[Martin Pool (mbp)]

Not surprisingly it looks like we're getting None when we should get a stat value. This might be because of a race where a file (maybe an editor lock or backup) appears and then disappears in the tree while it's being read.

 Traceback (most recent call last):
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 842, in exception_to_return_code
     return the_callable(*args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1037, in run_bzr
     ret = run(*run_argv)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 654, in run_argv_aliases
     return self.run(**all_cmd_args)
   File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 3051, in run
     exclude=safe_relpath_files(tree, exclude))
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree_4.py", line 197, in commit
     result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
     result = unbound(self, *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/mutabletree.py", line 228, in commit
     *args, **kwargs)
   File "/usr/lib/python2.6/dist-packages/bzrlib/commit.py", line 360, in commit
     self._update_builder_with_changes()
   File "/usr/lib/python2.6/dist-packages/bzrlib/commit.py", line 633, in _update_builder_with_changes
     self.work_tree, self.basis_revid, iter_changes):
   File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 636, in record_iter_changes
     for change in iter_changes:
   File "/usr/lib/python2.6/dist-packages/bzrlib/commit.py", line 654, in _filter_iter_changes
     for change in iter_changes:
   File "_dirstate_helpers_pyx.pyx", line 1405, in _dirstate_helpers_pyx.ProcessEntryC.__next__
   File "_dirstate_helpers_pyx.pyx", line 1678, in _dirstate_helpers_pyx.ProcessEntryC._iter_next
   File "_dirstate_helpers_pyx.pyx", line 1847, in _dirstate_helpers_pyx.ProcessEntryC._loop_one_block
   File "/usr/lib/python2.6/dist-packages/bzrlib/osutils.py", line 1485, in _walkdirs_utf8
     dirblock = sorted(read_dir(relroot, top))
   File "_readdir_pyx.pyx", line 257, in _readdir_pyx.UTF8DirReader.read_dir

[Martin Pool (mbp)]

It looks pretty easy: _readdir_pyx in _read_dir uses a statvalue of None for files that disappear(?) between the readdir and the stat, but UTF8DirReader.read_dir doesn't handle it.

We could handle None but I'd actually question whether we're doing anything useful by returning data about these files- why not just skip them and treat them as totally gone?

[Martin Pool (mbp)]

https://code.edge.launchpad.net/~mbp/bzr/446033-readdir/+merge/13049

[Eddie Parker (o-launchpad-eddieparker-safersignup-com)]

I also get this on a bzr st. I'm using cygwin.

If this helps, here's my information:

bzr: ERROR: exceptions.AttributeError: 'NoneType' object has no attribute 'st_mode'

Traceback (most recent call last):
  File "commands.py", line 842, in exception_to_return_code
  File "commands.py", line 1037, in run_bzr
  File "commands.py", line 654, in run_argv_aliases
  File "commands.py", line 1052, in ignore_pipe
  File "builtins.py", line 319, in run
  File "status.py", line 118, in show_tree_status
  File "tree.py", line 95, in changes_from
  File "decorators.py", line 138, in read_locked
  File "tree.py", line 977, in compare
  File "delta.py", line 230, in _compare_trees
  File "_dirstate_helpers_pyx.pyx", line 1405, in _dirstate_helpers_pyx.ProcessEntryC.__next__
  File "_dirstate_helpers_pyx.pyx", line 1551, in _dirstate_helpers_pyx.ProcessEntryC._iter_next
  File "osutils.py", line 1485, in _walkdirs_utf8
  File "_readdir_pyx.pyx", line 257, in _readdir_pyx.UTF8DirReader.read_dir
AttributeError: 'NoneType' object has no attribute 'st_mode'

bzr 2.0.0 on python 2.5.2 (CYGWIN_NT-6.0-WOW64-1.5.25-0.156-4-2-i686-32bit)
arguments: ['/usr/bin/bzr', 'st']
encoding: 'US-ASCII', fsenc: 'US-ASCII', lang: 'en_CA.UTF-8'
plugins:
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [2.0.0]
  netrc_credential_store /usr/lib/python2.5/site-packages/bzrlib/plugins/netrc_credential_store [2.0.0]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

[Eddie Parker (o-launchpad-eddieparker-safersignup-com)]

Ah. I checked out the link the Martin posted to the merge request, and subsequently started using the dev version on my box. The problem no longer exists with the development version.

[Eddie Parker (o-launchpad-eddieparker-safersignup-com)]

Ermm.. Hi again. Still me, Margaret.

Anyhow, looks like I had forgotten to compile the modules, which was giving me all sorts of erroneous data, but no actual 'errors' (bzr st reported that everything had been removed, but it didn't cause an exception or anything, just a warning about modules not being compiled).

Compiling modules caused this to still occur. Applying the patch mentioned in Martin's link to the .pyx file in question, recompiling the module, and everything appears to be working now.