Switching between pages doesn't properly update the position in the index

[Raphaël Hertzog]

Everytime that I switch between wiki pages via links (or via the calendar plugin), the position in the index doesn't get correctly updated (although some unrelated trees do get expanded). I get an error like this:

INFO: Open page: <FileStorePage: Journal:2013:05:19> (<Path: Journal:2013:05:19>)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 760, in on_open_page
    self.treeview.select_page(path, vivificate=True)
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 672, in select_page
    rowreference = gtk.TreeRowReference(model, treepath)
RuntimeError: could not create GtkTreeRowReference object

When I click directly on the index, I also get the same message. I'm running Zim 0.60 in Debian Unstable.

Versions of packages zim depends on:
ii python 2.7.3-5
ii python-gobject 3.8.1-3
ii python-gtk2 2.24.0-3+b1
ii python-xdg 0.19-5

[Raphaël Hertzog (hertzog)]

After a forced index update, it gets worse:

DEBUG: Action: reload_index
INFO: Updating index
DEBUG: Opening ProgressBarDialog
INFO: Index update done
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 745, in <lambda>
    ui.connect('end-index-update', lambda o: self.reload_model())
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 790, in reload_model
    self.treeview.set_model(model)
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 533, in set_model
    self.select_page(self.ui.page, vivificate=True)
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 672, in select_page
    rowreference = gtk.TreeRowReference(model, treepath)
RuntimeError: could not create GtkTreeRowReference object
ERROR: BUG: Invalid treepath: row-has-child-toggled <IndexPath: Gestion de ma vie:INBOX> (8, 3)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 167, in on_changed
    treeiter = self.get_iter(treepath)
ValueError: invalid tree path
ERROR: BUG: Invalid treepath: row-has-child-toggled <IndexPath: Gestion de ma vie:INBOX> (8, 3)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 167, in on_changed
    treeiter = self.get_iter(treepath)
ValueError: invalid tree path
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 807, in <lambda>
    ('end-index-update', lambda o: self.reload_model()),
  File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 876, in reload_model
    self._reload_model(type, filter)
  File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 894, in _reload_model
    self.treeview.set_model(model, filter)
  File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 598, in set_model
    PageTreeView.set_model(self, filtermodel)
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 533, in set_model
    self.select_page(self.ui.page, vivificate=True)
  File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 666, in select_page
    assert treepath, 'BUG: failed to touch placeholder'
AssertionError: BUG: failed to touch placeholder

** (zim:5219): CRITICAL **: pygtk_generic_tree_model_ref_node: assertion `VALID_ITER(iter, tree_model)' failed

** (zim:5219): CRITICAL **: pygtk_generic_tree_model_iter_has_child: assertion `VALID_ITER(iter, tree_model)' failed

** (zim:5219): CRITICAL **: pygtk_generic_tree_model_iter_next: assertion `VALID_ITER(iter, tree_model)' failed
/usr/lib/python2.7/dist-packages/zim/gui/__init__.py:604: GtkWarning: /tmp/buildd/gtk+2.0-2.24.17/gtk/gtktreeview.c:6057 (validate_visible_area): assertion `has_child' failed.
There is a disparity between the internal view of the GtkTreeView,
and the GtkTreeModel. This generally means that the model has changed
without letting the view know. Any display from now on is likely to
be incorrect.

  gtk.main()

(A supplementary difference is that this time I have run it with --standalone, in case it matters)

[Jaap Karssenberg (jaap.karssenberg)]

Very likely your index file is corrupted. Try restoring it by running "zim
--index -V /path/to/notebook" from the terminal.

Hope that helps,

Jaap

On Mon, May 20, 2013 at 11:35 AM, Raphaël Hertzog <email address hidden>wrote:

> After a forced index update, it gets worse:
>
> DEBUG: Action: reload_index
> INFO: Updating index
> DEBUG: Opening ProgressBarDialog
> INFO: Index update done
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 745,
> in <lambda>
> ui.connect('end-index-update', lambda o: self.reload_model())
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 790,
> in reload_model
> self.treeview.set_model(model)
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 533,
> in set_model
> self.select_page(self.ui.page, vivificate=True)
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 672,
> in select_page
> rowreference = gtk.TreeRowReference(model, treepath)
> RuntimeError: could not create GtkTreeRowReference object
> ERROR: BUG: Invalid treepath: row-has-child-toggled <IndexPath: Gestion de
> ma vie:INBOX> (8, 3)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 167,
> in on_changed
> treeiter = self.get_iter(treepath)
> ValueError: invalid tree path
> ERROR: BUG: Invalid treepath: row-has-child-toggled <IndexPath: Gestion de
> ma vie:INBOX> (8, 3)
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 167,
> in on_changed
> treeiter = self.get_iter(treepath)
> ValueError: invalid tree path
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 807,
> in <lambda>
> ('end-index-update', lambda o: self.reload_model()),
> File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 876,
> in reload_model
> self._reload_model(type, filter)
> File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 894,
> in _reload_model
> self.treeview.set_model(model, filter)
> File "/usr/lib/python2.7/dist-packages/zim/plugins/tags.py", line 598,
> in set_model
> PageTreeView.set_model(self, filtermodel)
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 533,
> in set_model
> self.select_page(self.ui.page, vivificate=True)
> File "/usr/lib/python2.7/dist-packages/zim/gui/pageindex.py", line 666,
> in select_page
> assert treepath, 'BUG: failed to touch placeholder'
> AssertionError: BUG: failed to touch placeholder
>
> ** (zim:5219): CRITICAL **: pygtk_generic_tree_model_ref_node: assertion
> `VALID_ITER(iter, tree_model)' failed
>
> ** (zim:5219): CRITICAL **: pygtk_generic_tree_model_iter_has_child:
> assertion `VALID_ITER(iter, tree_model)' failed
>
> ** (zim:5219): CRITICAL **: pygtk_generic_tree_model_iter_next: assertion
> `VALID_ITER(iter, tree_model)' failed
> /usr/lib/python2.7/dist-packages/zim/gui/__init__.py:604: GtkWarning:
> /tmp/buildd/gtk+2.0-2.24.17/gtk/gtktreeview.c:6057 (validate_visible_area):
> assertion `has_child' failed.
> There is a disparity between the intern...

[Raphaël Hertzog (hertzog)]

On Thu, 23 May 2013, Jaap Karssenberg wrote:
> Very likely your index file is corrupted. Try restoring it by running "zim
> --index -V /path/to/notebook" from the terminal.

Indeed, this fixed it.

But how can it be corrupted? I have no other sign that my hardware is
faulty.

Could zim detect this corruption for example by handling the exception
seen?

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Do you like what I do? Support my free software work on Debian and Ubuntu:
→ http://raphaelhertzog.com/support-my-work/

[Jaap Karssenberg (jaap.karssenberg)]

The corruption is probably due to my software not being robust enough yet
:( I think this sometimes happens when a full update is triggered (e.g. by
version upgrade) and then zim is closed again before the update is
finished. In theory it should just fix itself next time around, but
sometimes it doesn't.

Have index code as next thing on my list for a refactoring, hope to improve
robustness of the caching there.

Yes catching the exception could be done. But I'm worried that triggering a
full update for each exception has a high potential for causing infinite
loops of updates.

On Thu, May 23, 2013 at 4:24 PM, Raphaël Hertzog <email address hidden> wrote:

> On Thu, 23 May 2013, Jaap Karssenberg wrote:
> > Very likely your index file is corrupted. Try restoring it by running
> "zim
> > --index -V /path/to/notebook" from the terminal.
>
> Indeed, this fixed it.
>
> But how can it be corrupted? I have no other sign that my hardware is
> faulty.
>
> Could zim detect this corruption for example by handling the exception
> seen?
>
> Cheers,
> --
> Raphaël Hertzog ◈ Debian Developer
>
> Do you like what I do? Support my free software work on Debian and Ubuntu:
> → http://raphaelhertzog.com/support-my-work/
>
> --
> You received this bug notification because you are subscribed to Zim.
> https://bugs.launchpad.net/bugs/1181980
>
> Title:
> Switching between pages doesn't properly update the position in the
> index
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/zim/+bug/1181980/+subscriptions
>

[monkeyshoulder (laurenth-i)]

Hi,

Same here but with Zim 0.62 on ArchLinux. If I update the index with "zim --index" or using the menu I can get the index displayed correctly but it still has strange behaviour like clicking on an entry opens unrelated pages.

[monkeyshoulder (laurenth-i)]

Using this branch https://code.launchpad.net/~jaap.karssenberg/zim/pyzim fixes the problem for me (revision 750 and subsequent revisions).

[Andreas Wehler (andreas-wehler)]

Hi.

I have just generated a SQL dump of index.db with sqlite3
  a) BAD case (directory 03) and
  b) GOOD case (directory 04)

b) Is generated this way:
 1) exit zim
 2) remove ./zim/*
 3) start zim
 4) Icon: Task List # will regenerate the index in an other way than "Tools - Update index", takes 00:02:30 time.
 5) cd .../.zim ; sqlite3 index.db .dump >dump.sql

Note, that in the BAD case this dump is no more ASCII, but contains a lot of binary data (!!).
See attachment DotZim.tgz. Hope it helps...

Best regards, Andreas

[Andreas Wehler (andreas-wehler)]

Concrete hint:

index.db depends on how zim has been invoked, on one and the same host.
Attached is a proof with an example zim book "~/info/Info_Zim/ScratchInfo".

Bad Case a)
===========
1) rm ~/info/Info_Zim/ScratchInfo/.zim/* # start with defined conditions
2) Launch zim via an own XFCE launcher or via the Desktop Application Menu.
   Keep in mind that the whole environment is minimum, shell startup
   scripts are not sourced and you just have no shell etc.
3) Within zim get access to he book ScratchInfo via
   File - Open Another Notebook # add it if needed, then open it
   I don't know if the location of zim books does matter, it should not.
4) # Pick Task List from Tool bar to enforce the build of a new index
   # leave zim now
5) create an sql dump of index.db
   cd ~/info/Info_Zim/ScratchInfo/.zim
   mkdir -p KeepMe/01
   sqlite3 index.db .dump >KeepMe/01/dump.sql # this file is broken
   less -NS KeepMe/01/dump.sql
  21 INSERT INTO "pages" VALUES(1,'','',0,0,1,NULL,NULL,NULL,NULL,1.42757454824159431454e+09);
  22 INSERT INTO "pages" VALUES(2,'Home','^S^Z^X^P^A ^A ',1,1,0,NULL,NULL,NULL,1.42635246455724644656e+09,NULL);
  23 INSERT INTO "pages" VALUES(3,'tmp','^_^XESC^A ^A ',1,1,0,NULL,NULL,NULL,1.42757454820559430124e+09,NULL);

Good Case b)
============
1) rm ~/info/Info_Zim/ScratchInfo/.zim/* # again, start with defined conditions
2) Launch zim via xterm or ssh -X
   zim --standalone ~/info/Info_Zim/ScratchInfo/
3) # we already are within the book, skip this step
4) # Tool bar: Task List
   # this will enforce a new index to be created
   # leave zim now
5) create an sql dump of index.db
   cd ~/info/Info_Zim/ScratchInfo/.zim
   mkdir -p KeepMe/02
   sqlite3 index.db .dump >KeepMe/02/dump.sql # this file looks good
   less -NS KeepMe/02/dump.sql
  22 INSERT INTO "pages" VALUES(1,'','',0,0,1,NULL,NULL,NULL,NULL,1.42757454824159431454e+09);
  23 INSERT INTO "pages" VALUES(2,'tmp','tmp',1,1,0,NULL,NULL,NULL,1.42757454820559430124e+09,NULL);
  24 INSERT INTO "pages" VALUES(3,'Home','home',1,1,0,NULL,NULL,NULL,1.42635246455724644656e+09,NULL);
  25 INSERT INTO "pages" VALUES(4,'Inkscape','inkscape',1,1,1,NULL,NULL,NULL,1.42670902619396615029e+09,1.4282495210700762272e+09);

You can work with the zim book with Bad Case a) as well as with Good Case b),
But if you decide to switch between the 2 different launch methods then the index is
totally broken and useless, the complete index pane is a whole mess.
I do need access to the zim book from several hosts and now I know a work around:
Just start local work also via xterm and enjoy. Local XFCE launcher will give
Bad Case a).
You can also build a launcher with a wrapper like:
   ssh -X localhost '$HOME/info/bin/StartZim.sh'
where the wrapper just holds:

#!/bin/bash
zim --standalone 2>&1 >/tmp/Zim_AW.log

This way you also can save the error messages.

Best regards, Andreas

[Jaap Karssenberg (jaap.karssenberg)]

Thanks for the additional info. Just confirms the problem is due to localization -- apparenty there is still a difference in initialization between the two ways to start zim - will make sure this is fixed.

[Jaap Karssenberg (jaap.karssenberg)]

Assume this to be fixes in zim 0.66 and up