after I move dir with newly added dir/files, bzr unable to operate with WT: unsubscriptable object error

[Alexander Belchenko]

I have a tree in which I add new dir, new files to this dir. Then I move parent dir (that already exists) to one level up:

G:\project\data\game\simple>bzr mv alert ../
data/game/simple/alert => data/game/alert

G:\project>bzr st
bzr: ERROR: exceptions.TypeError: unsubscriptable object

Traceback (most recent call last):
  File "E:\work\erik.tags\bzrlib\commands.py", line 638, in run_bzr_catch_errors
    return run_bzr(argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 600, in run_bzr
    ret = run(*run_argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 296, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "E:\work\erik.tags\bzrlib\commands.py", line 610, in ignore_pipe
    result = func(*args, **kwargs)
  File "E:\work\erik.tags\bzrlib\builtins.py", line 202, in run
    to_file=self.outf, short=short, versioned=versioned)
  File "E:\work\erik.tags\bzrlib\status.py", line 157, in show_tree_status
    want_unversioned=want_unversioned)
  File "E:\work\erik.tags\bzrlib\tree.py", line 91, in changes_from
    want_unversioned=want_unversioned,
  File "E:\work\erik.tags\bzrlib\decorators.py", line 127, in read_locked
    return unbound(self, *args, **kwargs)
  File "E:\work\erik.tags\bzrlib\tree.py", line 596, in compare
    want_unversioned=want_unversioned)
  File "E:\work\erik.tags\bzrlib\delta.py", line 222, in _compare_trees
    want_unversioned=want_unversioned):
  File "E:\work\erik.tags\bzrlib\workingtree_4.py", line 2137, in _iter_changes
    for result in _process_entry(current_entry, None):
  File "E:\work\erik.tags\bzrlib\workingtree_4.py", line 1918, in _process_entry
    target_parent_id = target_parent_entry[0][2]
TypeError: unsubscriptable object

bzr 0.16.0dev0 on python 2.4.4.final.0 (win32)
arguments: ['E:\\work\\erik.tags\\bzr', 'st']

G:\project>bzr di
bzr: ERROR: exceptions.TypeError: unsubscriptable object

Traceback (most recent call last):
  File "E:\work\erik.tags\bzrlib\commands.py", line 638, in run_bzr_catch_errors
    return run_bzr(argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 600, in run_bzr
    ret = run(*run_argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 296, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "E:\work\erik.tags\bzrlib\commands.py", line 610, in ignore_pipe
    result = func(*args, **kwargs)
  File "E:\work\erik.tags\bzrlib\builtins.py", line 1427, in run
    old_label=old_label, new_label=new_label)
  File "E:\work\erik.tags\bzrlib\diff.py", line 375, in diff_cmd_helper
    extra_trees=extra_trees)
  File "E:\work\erik.tags\bzrlib\diff.py", line 403, in show_diff_trees
    extra_trees=extra_trees)
  File "E:\work\erik.tags\bzrlib\diff.py", line 434, in _show_diff_trees
    extra_trees=extra_trees, require_versioned=True)
  File "E:\work\erik.tags\bzrlib\tree.py", line 91, in changes_from
    want_unversioned=want_unversioned,
  File "E:\work\erik.tags\bzrlib\decorators.py", line 127, in read_locked
    return unbound(self, *args, **kwargs)
  File "E:\work\erik.tags\bzrlib\tree.py", line 596, in compare
    want_unversioned=want_unversioned)
  File "E:\work\erik.tags\bzrlib\delta.py", line 222, in _compare_trees
    want_unversioned=want_unversioned):
  File "E:\work\erik.tags\bzrlib\workingtree_4.py", line 2137, in _iter_changes
    for result in _process_entry(current_entry, None):
  File "E:\work\erik.tags\bzrlib\workingtree_4.py", line 1918, in _process_entry
    target_parent_id = target_parent_entry[0][2]
TypeError: unsubscriptable object

bzr 0.16.0dev0 on python 2.4.4.final.0 (win32)
arguments: ['E:\\work\\erik.tags\\bzr', 'di']

Before I move dir all works OK. I think it's related to bug #103870

[Alexander Belchenko (bialix)]

More info:

when I move those dir back I found that new file added to new dir now marked as unknown:

G:\project\data\game>bzr mv alert simple/
data/game/alert => data/game/simple/alert

G:\project>bzr st
removed:
  data/inter_game/alert/camera/return-to-betmode.test
added:
  data/game/simple/alert/camera/
modified:
  packets.h
unknown:
  data/game/simple/alert/camera/1-player-win.test

Last file was added before I start to move dirs forth and back.

[Alexander Belchenko (bialix)]

Story continue:

G:\project>bzr add
added data/game/simple/alert/camera/1-player-win.test
ignored 54 file(s).
If you wish to add some of these files, please add them by name.

G:\project>bzr st
removed:
  data/inter_game/alert/camera/return-to-betmode.test
added:
  data/game/simple/alert/camera/
modified:
  game_house.h
  game_states.h
  libgame.py
  log_states.c
  packet_player.py
  packets.h
  player.c
  sync_devices/player_data.c
unknown:
  data/game/simple/alert/camera/1-player-win.test

G:\project>bzr added
data/game/simple/alert/camera/1-player-win.test
data/game/simple/alert/camera

[Alexander Belchenko (bialix)]

More:

`bzr ci` is also don't want to commit that file (that simultaneously
added and unknown).

When I try to do `bzr revert` I have another error:

G:\project>bzr revert
bzr: ERROR: exceptions.AssertionError: file 1player.test-20070411105435-d0xfcaghj1lf944x-2
relocation in row (('data/game/simple
/alert/camera', '1-player-win.test', '1player.test-20070411105435-d0xfcaghj1lf944x-2'), [('r',
'data/game/alert/camera/1-player.
test', 0, False, ''), ('a', '', 0, False, '')]) but also at
'data/game/simple/alert/camera/1-player-win.test'

Traceback (most recent call last):
  File "E:\work\erik.tags\bzrlib\commands.py", line 638, in run_bzr_catch_errors
    return run_bzr(argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 600, in run_bzr
    ret = run(*run_argv)
  File "E:\work\erik.tags\bzrlib\commands.py", line 296, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "E:\work\erik.tags\bzrlib\builtins.py", line 2754, in run
    not no_backup, pb, report_changes=True)
  File "E:\work\erik.tags\bzrlib\mutabletree.py", line 38, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "E:\work\erik.tags\bzrlib\workingtree.py", line 1815, in revert
    self.set_parent_ids(self.get_parent_ids()[:1])
  File "E:\work\erik.tags\bzrlib\mutabletree.py", line 38, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "E:\work\erik.tags\bzrlib\workingtree_4.py", line 1051, in set_parent_ids
    self.current_dirstate()._validate()
  File "E:\work\erik.tags\bzrlib\dirstate.py", line 2255, in _validate
    raise AssertionError(
AssertionError: file 1player.test-20070411105435-d0xfcaghj1lf944x-2 relocation in row
(('data/game/simple/alert/camera', '1-play
er-win.test', '1player.test-20070411105435-d0xfcaghj1lf944x-2'), [('r',
'data/game/alert/camera/1-player.test', 0, False, ''), (
'a', '', 0, False, '')]) but also at 'data/game/simple/alert/camera/1-player-win.test'

bzr 0.16.0dev0 on python 2.4.4.final.0 (win32)
arguments: ['E:\\work\\erik.tags\\bzr', 'revert']

And now this tree stays in this weird state.

[John A Meinel (jameinel)]

I would guess this has been fixed by the recent updates to handle unknowns in empty directories, etc.

I would certainly like to make sure everything is working. Can you reproduce this error.

Even better if you could reproduce this with a small test script.

[Alexander Belchenko (bialix)]

> bzr version
Bazaar (bzr) 0.16.0dev0
  from bzr checkout E:/work/bzr.dev
    revision: 2432
    revid: <email address hidden>
    branch nick: bzr.dev
Using python interpreter: E:\Python24\python.exe
Using python standard library: E:\Python24\lib
Using bzrlib: E:\work\bzr.dev\bzrlib
...

The bug is still here. As well as https://bugs.launchpad.net/bzr/+bug/104257
See:

E:\temp\1>bzr st
E:\temp\1\data\game\simple\alert>bzr mkdir camera
added camera
E:\temp\1\data\game\simple\alert>cd camera
E:\temp\1\data\game\simple\alert>echo spam > spam
E:\temp\1\data\game\simple\alert\camera>bzr st
added:
  data/game/simple/alert/camera/

^^^^ You see??? No unknown file listed

E:\temp\1\data\game\simple\alert\camera>bzr unknowns
data/game/simple/alert/camera/spam

E:\temp\1\data\game\simple\alert\camera>cd ../..

E:\temp\1\data\game\simple>bzr mv alert ../
data/game/simple/alert => data/game/alert

E:\temp\1\data\game\simple>bzr st
bzr: ERROR: exceptions.TypeError: unsubscriptable object

Traceback (most recent call last):
  File "E:\work\bzr.dev\bzrlib\commands.py", line 647, in run_bzr_catch_errors
    return run_bzr(argv)
  File "E:\work\bzr.dev\bzrlib\commands.py", line 609, in run_bzr
    ret = run(*run_argv)
  File "E:\work\bzr.dev\bzrlib\commands.py", line 305, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "E:\work\bzr.dev\bzrlib\commands.py", line 619, in ignore_pipe
    result = func(*args, **kwargs)
  File "E:\work\bzr.dev\bzrlib\builtins.py", line 203, in run
    to_file=self.outf, short=short, versioned=versioned)
  File "E:\work\bzr.dev\bzrlib\status.py", line 157, in show_tree_status
    want_unversioned=want_unversioned)
  File "E:\work\bzr.dev\bzrlib\tree.py", line 91, in changes_from
    want_unversioned=want_unversioned,
  File "E:\work\bzr.dev\bzrlib\decorators.py", line 127, in read_locked
    return unbound(self, *args, **kwargs)
  File "E:\work\bzr.dev\bzrlib\tree.py", line 596, in compare
    want_unversioned=want_unversioned)
  File "E:\work\bzr.dev\bzrlib\delta.py", line 222, in _compare_trees
    want_unversioned=want_unversioned):
  File "E:\work\bzr.dev\bzrlib\workingtree_4.py", line 2140, in _iter_changes
    for result in _process_entry(current_entry, None):
  File "E:\work\bzr.dev\bzrlib\workingtree_4.py", line 1919, in _process_entry
    target_parent_id = target_parent_entry[0][2]
TypeError: unsubscriptable object

bzr 0.16.0dev0 on python 2.4.4.final.0 (win32)
arguments: ['E:\\work\\bzr.dev\\bzr', 'st']

** please send this report to <email address hidden>

[Alexander Belchenko (bialix)]

The same behavior with Python 2.5

[Alexander Belchenko (bialix)]

Alexander Belchenko пишет:
>> bzr version
> Bazaar (bzr) 0.16.0dev0
> from bzr checkout E:/work/bzr.dev
> revision: 2432
> revid: <email address hidden>
> branch nick: bzr.dev
> Using python interpreter: E:\Python24\python.exe
> Using python standard library: E:\Python24\lib
> Using bzrlib: E:\work\bzr.dev\bzrlib
> ...
>
> The bug is still here. As well as https://bugs.launchpad.net/bzr/+bug/104257
> See:
>
> E:\temp\1>bzr st
> E:\temp\1\data\game\simple\alert>bzr mkdir camera
> added camera
> E:\temp\1\data\game\simple\alert>cd camera
> E:\temp\1\data\game\simple\alert>echo spam > spam
> E:\temp\1\data\game\simple\alert\camera>bzr st
> added:
> data/game/simple/alert/camera/
>
> ^^^^ You see??? No unknown file listed

Your patch for bug #104257 fix this problem with unknown file,
but main error with unsubscriptable object error is still here.

[µ]

[Alexander Belchenko (bialix)]

John A Meinel пишет:
> I would guess this has been fixed by the recent updates to handle
> unknowns in empty directories, etc.
>
> I would certainly like to make sure everything is working. Can you
> reproduce this error.
>
> Even better if you could reproduce this with a small test script.

OK. I can reproduce it in new tree with these steps:

bzr init
bzr mkdir data
cd data
bzr mkdir game
cd game
bzr mkdir simple
cd simple
bzr mkdir alert
cd alert
bzr mkdir power
bzr ci -m 1

bzr mkdir camera
cd ..
bzr mv alert ../
bzr st

And here I get traceback.

It's strange, but this sequence sensitive to used directory names.
When I switch to abstract names as 'one', 'two', 'three' etc, I get no error.
Probably it depends on the length of these names?

[µ]

[John A Meinel (jameinel)]

I can confirm this with the bzr.dev 2428.

I'm guessing it isn't the length of the filenames, but sort order. (like if X sorts before Y versus Z, or sort-order in a set() which is much harder to get correct).

[John A Meinel (jameinel)]

Looking at the .bzr/checkout/dirstate file after the move, I believe this is a bug in the "move" code, and not in the "bzr status" code.

Specifically, I see the entries:
dir name: new tree, old tree)

'' '': directory, directory
'' data: directory, directory
data game: directory, directory
data/game alert: directory, rename from data/game/simple/alert
data/game simple: directory, directory
data/game/alert camera: directory, absent
data/game/simple alert: renamed to data/game/alert, directory
data/game/simple/alert power: directory, directory

Everything is correct except for the last line. We have a proper "rename from/to" pair for "data/game/simple/alert=>data/game/alert".

But we are incorrectly reporting "data/game/simple/alert/power" as unmoved, when really it has been moved to "data/game/alert/power".

So what is happening is that while we are iterating through the records, we find "data/game/simple/alert/power" and try to look up its parent, but there is no "data/game/simple/alert" in the working tree.

So I'll try to fix up the renaming code with more tests and hopefully can track down where this is happening.

I've fixed some similar bugs recently, but I think this one is different.

[John A Meinel (jameinel)]

Oh, it is also important that "camera" is added before the rename. Without it, "power" is properly renamed into the correct location. This is a simplified test case.

bzr init
bzr mkdir c
bzr mkdir c/b
bzr mkdir c/b/d
bzr ci -m 1

bzr mkdir c/b/a
bzr mv c/b .
bzr st

The important thing seems to be that we get to the new directory before we get to the existing subdirectory.
Specifically, if I use "bzr mkdir c/b/e" which comes after the existing 'd', it passes.

In the associated branch I have DirState._validate() checking for this case (a entry existing without being in a valid directory), and a test case which triggers it

[John A Meinel (jameinel)]

Fix available for this, and several other directory move related bugs, in the associated branch.

[Alexander Belchenko (bialix)]

already merged