Sync Daemon UnpicklingError

[jan-teichmann]

Ubuntuone client 4.0.0 on Fedora 18 x86_64

2012-12-22 14:51:50,816 - ubuntuone.SyncDaemon.sync - ERROR - unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 322, in on_event
    **kwargs)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/fsm/fsm.py", line 161, in on_event
    (action_func_name, e))
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 240, in level_log
    self.log(lvl, message, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 270, in log
    "relpath of: %r", self.key['path'])
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 110, in __getitem__
    return self.get(key)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 91, in get
    mdobj = self.fs.get_by_mdid(mdid)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py", line 789, in get_by_mdid
    mdobj = self.fs[mdid]
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 953, in __getitem__
    return self._deserialize(self._db.get(self.row_type, key))
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 978, in _deserialize
    return cPickle.loads(raw_value)
UnpicklingError: invalid load key, '|'.
2012-12-22 14:51:51,409 - ubuntuone.SyncDaemon.EQ - ERROR - Error encountered while handling: HQ_HASH_ERROR in <ubuntuone.syncdaemon.sync.Sync object at 0x50e0b50>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/event_queue.py", line 324, in _dispatch
    method(**kwargs)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 1053, in handle_HQ_HASH_ERROR
    ssmr.on_event('HQ_HASH_ERROR', {})
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 317, in on_event
    isdir=self.key.is_directory(),
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 135, in is_directory
    return str(self.fs.is_dir(**self.keys))[0]
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py", line 1215, in is_dir
    mdobj = self.fs[mdid]
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 953, in __getitem__
    return self._deserialize(self._db.get(self.row_type, key))
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 978, in _deserialize
    return cPickle.loads(raw_value)
UnpicklingError: invalid load key, '|'.
2012-12-22 14:52:49,004 - ubuntuone.SyncDaemon.EQ - ERROR - Error encountered while handling: FS_FILE_DELETE in <ubuntuone.syncdaemon.sync.Sync object at 0x50e0b50>
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/event_queue.py", line 324, in _dispatch
    method(**kwargs)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 960, in handle_FS_FILE_DELETE
    if not key.is_subscribed():
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 220, in is_subscribed
    volume_id = self.get('share_id')
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/sync.py", line 106, in get
    return getattr(self.fs.get_by_mdid(mdid), key, None)
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py", line 789, in get_by_mdid
    mdobj = self.fs[mdid]
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 953, in __getitem__
    return self._deserialize(self._db.get(self.row_type, key))
  File "/usr/lib/python2.7/site-packages/ubuntuone-client/ubuntuone/syncdaemon/tritcask.py", line 978, in _deserialize
    return cPickle.loads(raw_value)
UnpicklingError: invalid load key, '|'.

[Roberto Alsina (ralsina)]

This looks like corrupted metadata. Perhaps stopping syncdaemon, deleting ~/.cache/ubuntuone and starting it again may make it work.

[Balló György (ballogy)]

Note that this bug affects many users on Arch Linux, and makes Ubuntu One unusable. It happens after the glibc package updated from version 2.16.0 to 2.17. Downstream bug report:
https://bugs.archlinux.org/task/33240?project=5&order=id&sort=desc

Removing the cache directory does not solve the problem.

[dobey (dobey)]

Metadata is actually stored under $XDG_DATA_HOME/ubuntuone/syncdaemon and not $XDG_CACHE_HOME/ubuntuone.

[Roman Yepishev (rye)]

I can reproduce this extremely easy with eglibc 2.17 build.
Based on the logs two syncdaemons are started:

http://paste.ubuntu.com/1545811/ - first syndaemon (starting it via command line with --debug)
http://paste.ubuntu.com/1545812/ - second syncdaemon (which i did not even ask to start)

[Roman Yepishev (rye)]

Correction, two syncdaemons are started one after another because it is programmed this way - it performs a restart upon UNKNOWN_ERROR. So the actual failure is still in pickling/unpickling.

[Roman Yepishev (rye)]

Guillermo (verterok) has found the reason - a flush is required after writing now.
His patch is attached, this has a Low priority for now since Raring does not yet have libc 2.17.

[Balló György (ballogy)]

The patch solves the problem for me on Arch Linux. Thanks!

[jan-teichmann (teichmann-jan)]

The patch also solved it for me on Fedora 18

[Maximiliano Bertacchini (maxiberta)]

This patch works great on F18. Thanks!

[Maciej Czekaj (natanielcz)]

On Fedora 18 sync works now but i haven't a window with settings which folders should be synced.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "lp1093111.patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntuone-client - 4.1.90-0ubuntu1

---------------
ubuntuone-client (4.1.90-0ubuntu1) raring; urgency=low

  * New upstream release.
    - Fix support for searching local files in cloud folders. (LP: #1101271)
    - Various fixes to the test suite.
    - Fix tritcask DataFile.write to flush with new libc6. (LP: #1093111)
    - Remove most of the static glib bindings usage. (LP: #1108036)
    - Avoid g_type_init with new versions of glib. (LP: #1100926)
    - Replace simplejson usage with json. (LP: #1029094)
    - Use a standard LC_CTYPE locale when the LANG is turkish. (LP: #997326)
  * debian/control:
    - Require newer version of ubuntu-sso-client for LANG fix.
    - Update dependencies to drop fallbacks for static bindings.
 -- Rodney Dawes <email address hidden> Tue, 29 Jan 2013 15:22:08 -0500

[Charles Profitt (cprofitt)]

I can confirm that the update fixed my issue.

[Maciej Czekaj (natanielcz)]

Ok. Ubuntu One client now appears but i haven't icon in system tray on Fedora 18. Can you fix it?

[dobey (dobey)]

@Maciej, please open new bugs if you have other problems. Thanks.