[precise] Client is stuck due to Upload executing before MakeFile

Bug #978903 reported by Roman Yepishev
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ubuntu One Client
Status tracked in Trunk
Stable-3-0
Fix Released
Undecided
Unassigned
Trunk
Fix Released
Critical
Facundo Batista
ubuntuone-client (Ubuntu)
Fix Released
High
dobey
Precise
Fix Released
High
dobey

Bug Description

When more than 200 files (that's the threshold for the queue offloading) are added to the Ubuntu One folder, at some point during the syncing the Upload() task may start executing prior to other tasks that set up the upload location (such as MakeFile). Since this should never happen and signals about out-of-order queue operation processing, syncdaemon stops synchronizing after raising a KeyError.

During my tests this was 100% reproducible.

[Test Case]
1. Add/remove directory with 200 files or more

2. Ensure that files synced and an error like the one below is not output in ~/.cache/ubuntuone/log/syncdaemon.log

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/action_queue.py", line 1375, in go
    yield self.run()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1037, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/action_queue.py", line 1387, in run
    yield self._start()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1039, in _inlineCallbacks
    result = g.send(result)
  File "/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/action_queue.py", line 2504, in _start
    mdobj = fsm.get_by_node_id(self.share_id, self.node_id)
  File "/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py", line 786, in get_by_node_id
    mdid = self._idx_node_id[(share_id, node_id)]
  File "/usr/lib/python2.7/dist-packages/ubuntuone-client/ubuntuone/syncdaemon/filesystem_manager.py", line 191, in __getitem__
    return dict.__getitem__(self, key)
KeyError: ('', marker:07dac62a-ae8b-4c81-915b-7a1f5648867a)

[Regression Potential]
Minimal. Fix includes unit tests.

Related branches

Revision history for this message
Roman Yepishev (rye) wrote :
visibility: public → private
Roman Yepishev (rye)
Changed in ubuntuone-client:
importance: Undecided → Critical
status: New → Triaged
assignee: nobody → Facundo Batista (facundo)
Roman Yepishev (rye)
summary: - Client is stuck due to Upload executing before MakeFile
+ [precise] Client is stuck due to Upload executing before MakeFile
Roman Yepishev (rye)
visibility: private → public
Changed in ubuntuone-client:
status: Triaged → Fix Committed
tags: added: u1-sru
Changed in ubuntuone-client (Ubuntu):
milestone: none → precise-updates
importance: Critical → High
Roman Yepishev (rye)
description: updated
dobey (dobey)
Changed in ubuntuone-client (Ubuntu Precise):
assignee: Facundo Batista (facundo) → Rodney Dawes (dobey)
description: updated
dobey (dobey)
Changed in ubuntuone-client (Ubuntu):
milestone: precise-updates → none
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntuone-client - 3.0.1-0ubuntu1.1

---------------
ubuntuone-client (3.0.1-0ubuntu1.1) quantal; urgency=low

  * New upstream release.
    - Don't ignore test failures in from main tests suite. (LP: #993574)
    - Avoid multiple parallel GetDelta commands on same volume. (LP: #883252)
    - Fix small memory leak in folder enabled check. (LP: #981255)
    - Don't start syncdaemon with u1sdtool -q if not running. (LP: #907479)
    - Avoid having Upload execute before Makefile command. (LP: #978903)
  * debian/watch:
    - Update watch file to use stable-3-0 series for downloads now
 -- Rodney Dawes <email address hidden> Tue, 22 May 2012 13:33:55 -0400

Changed in ubuntuone-client (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Hello! This bug is attached to an upload in the queue for precise-proposed. However, it does not contain the requisite fields for the SRU team to process the upload. Please add them per the process outlined here. To be more succinct, make sure the bug description lists these fields: "Impact, Dev Fix, Stable Fix, Regression Potential, Test case".

https://wiki.ubuntu.com/StableReleaseUpdates#Procedure

The upload will likely not be looked at by the SRU team until these fields are present.

Thank you!

dobey (dobey)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Roman, or anyone else affected,

Accepted ubuntuone-client into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in ubuntuone-client (Ubuntu Precise):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

NACK.

ubuntuone-client was updated in precise-updates for the security issue in bug 882062.

This SRU needs to be updated to include the security fix.

Revision history for this message
Rick McBride (rmcbride) wrote :

Verified using :
 1. Add/remove directory with 200 files or more (generated 300 random files for this)

 2. Ensure that files synced and an error like the one below is not output in ~/.cache/ubuntuone/log/syncdaemon.log

However, due to mdeslaur's NACK, I'm holding off on setting verification-done pending the security fix.

Revision history for this message
Rick McBride (rmcbride) wrote :

Verified using :
 1. Add/remove directory with 200 files or more (generated 300 random files for this)

 2. Ensure that files synced and an error like the one below is not output in ~/.cache/ubuntuone/log/syncdaemon.log

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntuone-client - 3.0.1-0ubuntu1.0.1

---------------
ubuntuone-client (3.0.1-0ubuntu1.0.1) precise-proposed; urgency=low

  * New upstream release.
    - Don't ignore test failures in from main tests suite. (LP: #993574)
    - Avoid multiple parallel GetDelta commands on same volume. (LP: #883252)
    - Fix small memory leak in folder enabled check. (LP: #981255)
    - Don't start syncdaemon with u1sdtool -q if not running. (LP: #907479)
    - Avoid having Upload execute before Makefile command. (LP: #978903)
  * debian/watch:
    - Update watch file to use stable-3-0 series for downloads now
 -- Rodney Dawes <email address hidden> Wed, 06 Jun 2012 14:08:37 -0400

Changed in ubuntuone-client (Ubuntu Precise):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.