zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle'

[Roy Gilby]

Binary package hint: zeitgeist

lsb_release -rd
Description: Ubuntu Natty (development branch)
Release: 11.04

ProblemType: Crash
DistroRelease: Ubuntu 11.04
Package: zeitgeist-core 0.7-1
ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38
Uname: Linux 2.6.38-7-generic x86_64
Architecture: amd64
Date: Sun Mar 20 07:40:48 2011
EcryptfsInUse: Yes
ExecutablePath: /usr/bin/zeitgeist-daemon
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20110318)
InterpreterPath: /usr/bin/python2.7
ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon
ProcEnviron:
 SHELL=/bin/bash
 LANGUAGE=en_AU:en
 LANG=en_AU.UTF-8
PythonArgs: ['/usr/bin/zeitgeist-daemon']
SourcePackage: zeitgeist
Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2] No such file or directory: '/home/royg/.local/share/zeitgeist/datasources.pickle'
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare

[Markus Korn (thekorn)]

This looks *very* weird to me, I'll investigate later this week.

[Markus Korn (thekorn)]

Hi @all,
we discussed this issue in the #zeitgeist IRC channel, our theory is: it's related to encrypted homedirs, and that this dirs get unmounted before the zeitgeist daemon stops. So the daemon is unable to write to $XDG_DATA_HOME.

We'll investigate further, and keep you updated.
Thanks for reporting this bug,

Markus

[Markus Korn (thekorn)]

In order to find a way to fix this bug I would like to know from someone who is affected by this bug
  - which files/dirs get encrypted?
  - esp.: are files in ~/.locale or ~/.local/zeitgeist encrypted (for example by symlinking them into ~/.Private)

Thanks in advance,
Markus

[Mikkel Kamstrup Erlandsen (kamstrup)]

Markus this doesn't seem to happen when ZG stops, but when it starts up or something. At least that's what I gather from the dupes

[Ted Gould (ted)]

On Thu, 2011-03-31 at 13:44 +0000, Markus Korn wrote:
> In order to find a way to fix this bug I would like to know from someone who is affected by this bug
> - which files/dirs get encrypted?
> - esp.: are files in ~/.locale or ~/.local/zeitgeist encrypted (for example by symlinking them into ~/.Private)

That's the older method of using encrypted home directories. Today most
modern usages have everything encrypted. So for me ~/* is encrypted.

[Paul Hoell (hoellp)]

It's the same as for Ted here, i have an encrypted /home which makes zeitgeist crash on boot, maybe because the /home is not yet ready...
Is there a way to delay zeitgeist-daemon to check this theory?

[Markus Korn (thekorn)]

I was investigating on this bug a bit further. All Tracebacks are looking like

Traceback (most recent call last):
  File "/usr/bin/zeitgeist-daemon", line 180, in <module>
    handle_exit()
  File "/usr/bin/zeitgeist-daemon", line 126, in handle_exit
    interface.Quit()
  File "/usr/share/zeitgeist/_zeitgeist/engine/remote.py", line 349, in Quit
    self._engine.close()
  File "/usr/share/zeitgeist/_zeitgeist/engine/main.py", line 133, in close
    self.extensions.unload()
  File "/usr/share/zeitgeist/_zeitgeist/engine/extension.py", line 287, in unload
    self.unload(ext)
  File "/usr/share/zeitgeist/_zeitgeist/engine/extension.py", line 292, in unload
    obj.unload()
  File "/usr/share/zeitgeist/_zeitgeist/engine/extensions/datasource_registry.py", line 115, in unload
    self._write_to_disk()
  File "/usr/share/zeitgeist/_zeitgeist/engine/extensions/datasource_registry.py", line 99, in _write_to_disk
    with open(DATA_FILE, "w") as data_file:
IOError: [Errno 2] No such file or directory: '$XDG_HOME_DATA/zeitgeist/datasources.pickle'

This is telling us a few things:

 1.) the crash happens when handle_exit() is called, this function is called when the daemon recieves a SIGHUB or SIGTERM signal, means if the systems wants zg to shut down. (apport will try to report this crash on the next session start, which makes people think this crash happen *while booting*)

 2.) If you try to write to a certain location in python, and the parent directory of this location does not exist at this point, an IOError with "[Errno 2] No such file or directory: [...]" is raised. This indicates that $XDG_HOME_DATA or $XDG_HOME_DATA/zeitgeist does not exist for the bugreporters when the systems wants zg to shut-down.

 3.) All bugreporters are having their $HOME (or parts of it) encrypted ("EcryptfsInUse: Yes"). This *could* be the reason for this bug, *maybe* the encrypted dir (and thus $XDG_HOME_DATA) get un-mounted before the system wants zg to stop.

So this leads to a simple question: How do we want zg to behave if it is impossible to write the data to its DATA_PATH (which is in most cases $XDG_DATA_HOME/zeitgeist)? We are talking about these files atm:

 1.) activity.sqlite
 2.) datasources.pickle
 3.) blacklist.(pickle|json)
 [4.) extension specific files, like the fts index]

IMHO three of them are easy to handle. We should not care about 4.) at the first place, it's up to the (3rd party) extensions to do the right thing, but we have to tell extensions authors what we think is the best they could do. If the datasource registry fails to write its config file on shutdown it does not really matter, as its data is written on every change to this file anyway. We should just ignore this error on shutdown and log a warning. If config can not be written when AddTemplate or RemoveTemplate dbus method gets called we should log the error, and return the error over dbus, but not let the daemon crash. The requested config changes should not have any effect. Some goes for blacklists.
I'm not sure how to handle the activity log here. Data gets written to this file (sqlite3.connection.commit()) as a result of dbus calls, for which we should log this error, rever...

[Ted Gould (ted)]

On Fri, 2011-04-08 at 06:37 +0000, Markus Korn wrote:
> So this leads to a simple question: How do we want zg to behave

I'd like zg to connect to the gnome-session signal for shutdown and save
before the disk gets unmounted.

[XanderCDN (xandercdn)]

Just to confirm what was said earlier in the timeline, I got this crash message and I also use encrypted homedirs on natty updated daily.

[Seif Lotfy (seif)]

Hey guys,
Can someone tell us how to reproduce this. It will help us fix the issue.
Thanks
Seif

[amay82 (andimayer82-deactivatedaccount)]

Have an encrypted $HOME too... but I don't know how to reproduce, it just occurs sometimes when logging in

[Seif Lotfy (seif)]

I managed to replicate it and tring to fix the issue

On Thu, Apr 21, 2011 at 2:15 PM, amay82 <email address hidden> wrote:

> Have an encrypted $HOME too... but I don't know how to reproduce, it
> just occurs sometimes when logging in
>
> --
> You received this bug notification because you are subscribed to The
> Zeitgeist Project.
> https://bugs.launchpad.net/bugs/738555
>
> Title:
> zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2]
> No such file or directory:
> '/home/royg/.local/share/zeitgeist/datasources.pickle'
>
> Status in Zeitgeist Framework:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Incomplete
>
> Bug description:
> Binary package hint: zeitgeist
>
> lsb_release -rd
> Description: Ubuntu Natty (development branch)
> Release: 11.04
>
> ProblemType: Crash
> DistroRelease: Ubuntu 11.04
> Package: zeitgeist-core 0.7-1
> ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38
> Uname: Linux 2.6.38-7-generic x86_64
> Architecture: amd64
> Date: Sun Mar 20 07:40:48 2011
> EcryptfsInUse: Yes
> ExecutablePath: /usr/bin/zeitgeist-daemon
> InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20110318)
> InterpreterPath: /usr/bin/python2.7
> ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon
> ProcEnviron:
> SHELL=/bin/bash
> LANGUAGE=en_AU:en
> LANG=en_AU.UTF-8
> PythonArgs: ['/usr/bin/zeitgeist-daemon']
> SourcePackage: zeitgeist
> Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno
> 2] No such file or directory:
> '/home/royg/.local/share/zeitgeist/datasources.pickle'
> UpgradeStatus: No upgrade log present (probably fresh install)
> UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare
>

[Seif Lotfy (seif)]

So as Markus said, the crash report comes from the session before.
This implies that the home directory is unmounted or encrypted again before zeitgeist can write something to it when it receives a SIGHUB or SIGTERM signal.
The solution to this either to not allow the datasource register y to write to disk upon the occurrence of these signals. This means there could be a dataloss in the datasource. I will need to further investigate this.

[Seif Lotfy (seif)]

So this looks to me like a problem not only Zeitgeist is facing but anything
that is running and wants to write to disk upon session end.

On Thu, Apr 21, 2011 at 5:07 PM, Seif Lotfy <email address hidden>wrote:

> So as Markus said, the crash report comes from the session before.
> This implies that the home directory is unmounted or encrypted again before
> zeitgeist can write something to it when it receives a SIGHUB or SIGTERM
> signal.
> The solution to this either to not allow the datasource register y to write
> to disk upon the occurrence of these signals. This means there could be a
> dataloss in the datasource. I will need to further investigate this.
>
> --
> You received this bug notification because you are subscribed to The
> Zeitgeist Project.
> https://bugs.launchpad.net/bugs/738555
>
> Title:
> zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2]
> No such file or directory:
> '/home/royg/.local/share/zeitgeist/datasources.pickle'
>
> Status in Zeitgeist Framework:
> Triaged
> Status in “zeitgeist” package in Ubuntu:
> Incomplete
>
> Bug description:
> Binary package hint: zeitgeist
>
> lsb_release -rd
> Description: Ubuntu Natty (development branch)
> Release: 11.04
>
> ProblemType: Crash
> DistroRelease: Ubuntu 11.04
> Package: zeitgeist-core 0.7-1
> ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38
> Uname: Linux 2.6.38-7-generic x86_64
> Architecture: amd64
> Date: Sun Mar 20 07:40:48 2011
> EcryptfsInUse: Yes
> ExecutablePath: /usr/bin/zeitgeist-daemon
> InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20110318)
> InterpreterPath: /usr/bin/python2.7
> ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon
> ProcEnviron:
> SHELL=/bin/bash
> LANGUAGE=en_AU:en
> LANG=en_AU.UTF-8
> PythonArgs: ['/usr/bin/zeitgeist-daemon']
> SourcePackage: zeitgeist
> Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno
> 2] No such file or directory:
> '/home/royg/.local/share/zeitgeist/datasources.pickle'
> UpgradeStatus: No upgrade log present (probably fresh install)
> UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare
>

[Seif Lotfy (seif)]

I found an easier way to replicate the error.
1) Open Terminal (terminal1) and run "zeitgeist-daemon --replace"
2) Open another terminal (terminal2) and do "cd ~/.local/share/" then "mv
zeitgeist zeitgeist-backup"
3) In terminal1 then do "Ctrl+c" to quit Zeitgeist
(remember to move the zeitgeist-backup again to zeitgeist else you will lose
your data)
Cheers
Seif

On Thu, Apr 21, 2011 at 5:24 PM, Seif Lotfy <email address hidden> wrote:

> So this looks to me like a problem not only Zeitgeist is facing but
> anything that is running and wants to write to disk upon session end.
>
>
> On Thu, Apr 21, 2011 at 5:07 PM, Seif Lotfy <email address hidden>wrote:
>
>> So as Markus said, the crash report comes from the session before.
>> This implies that the home directory is unmounted or encrypted again
>> before zeitgeist can write something to it when it receives a SIGHUB or
>> SIGTERM signal.
>> The solution to this either to not allow the datasource register y to
>> write to disk upon the occurrence of these signals. This means there could
>> be a dataloss in the datasource. I will need to further investigate this

[Seif Lotfy (seif)]

OK so a primary solution for this was putting the content of _write_to_disk
in datasource_registery.py in a normal " try - catch " block
My primary fear was that the next extension or zeitgeist would crash trying
to write to the disk. This was not the case and the error was caught.
To explain why the other stuff did not crash is:
1) This was the only extension writing to disk upon unloading
2) The engine does not write to disk upon exit since all events are written
to disk upon entrance.
My primary solution would be to put the _write_to_disk in a normal "try -
catch" block for now...
Cheers
Seif

On Thu, Apr 21, 2011 at 5:31 PM, Seif Lotfy <email address hidden> wrote:

> I found an easier way to replicate the error.
> 1) Open Terminal (terminal1) and run "zeitgeist-daemon --replace"
> 2) Open another terminal (terminal2) and do "cd ~/.local/share/" then "mv
> zeitgeist zeitgeist-backup"
> 3) In terminal1 then do "Ctrl+c" to quit Zeitgeist
> (remember to move the zeitgeist-backup again to zeitgeist else you will
> lose your data)
> Cheers
> Seif
>
>
> On Thu, Apr 21, 2011 at 5:24 PM, Seif Lotfy <email address hidden> wrote:
>
>> So this looks to me like a problem not only Zeitgeist is facing but
>> anything that is running and wants to write to disk upon session end.
>>
>>
>> On Thu, Apr 21, 2011 at 5:07 PM, Seif Lotfy <email address hidden>wrote:
>>
>>> So as Markus said, the crash report comes from the session before.
>>> This implies that the home directory is unmounted or encrypted again
>>> before zeitgeist can write something to it when it receives a SIGHUB or
>>> SIGTERM signal.
>>> The solution to this either to not allow the datasource register y to
>>> write to disk upon the occurrence of these signals. This means there could
>>> be a dataloss in the datasource. I will need to further investigate this.
>>>
>>> --
>>> You received this bug notification because you are subscribed to The
>>> Zeitgeist Project.
>>> https://bugs.launchpad.net/bugs/738555
>>>
>>> Title:
>>> zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno 2]
>>> No such file or directory:
>>> '/home/royg/.local/share/zeitgeist/datasources.pickle'
>>>
>>> Status in Zeitgeist Framework:
>>> Triaged
>>> Status in “zeitgeist” package in Ubuntu:
>>> Incomplete
>>>
>>> Bug description:
>>> Binary package hint: zeitgeist
>>>
>>> lsb_release -rd
>>> Description: Ubuntu Natty (development branch)
>>> Release: 11.04
>>>
>>> ProblemType: Crash
>>> DistroRelease: Ubuntu 11.04
>>> Package: zeitgeist-core 0.7-1
>>> ProcVersionSignature: Ubuntu 2.6.38-7.36-generic 2.6.38
>>> Uname: Linux 2.6.38-7-generic x86_64
>>> Architecture: amd64
>>> Date: Sun Mar 20 07:40:48 2011
>>> EcryptfsInUse: Yes
>>> ExecutablePath: /usr/bin/zeitgeist-daemon
>>> InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20110318)
>>> InterpreterPath: /usr/bin/python2.7
>>> ProcCmdline: /usr/bin/python /usr/bin/zeitgeist-daemon
>>> ProcEnviron:
>>> SHELL=/bin/bash
>>> LANGUAGE=en_AU:en
>>> LANG=en_AU.UTF-8
>>> PythonArgs: ['/usr/bin/zeitgeist-daemon']
>>> SourcePackage: zeitgeist
>>> Title: zeitgeist-daemon crashed with IOError in _write_to_disk(): [Errno
>>>...

[Brian Murray (brian-murray)]

If I understand things correctly there is no data lost because the write to disk happens when logging in is that correct? If so then this doesn't seem necessary to fix in natty.

[Siegfried Gevatter (rainct)]

>> If I understand things correctly there is no data lost because
>> the write to disk happens when logging in is that correct? If
>> so then this doesn't seem necessary to fix in natty.

Right, the only data that is lost is information on which data-sources logged events recently. This information isn't used for anything important (nor user-visible) currently.

I'm marking the Natty task as invalid.