Ubuntu One sync daemon continuously hashing

Bug #580855 reported by Matthew on 2010-05-15
138
This bug affects 27 people
Affects Status Importance Assigned to Milestone
Ubuntu One Client
High
Ubuntu One Foundations+ team
ubuntuone-client (Ubuntu)
Medium
Ubuntu One Foundations+ team
Nominated for Lucid by Roman Yepishev
Nominated for Maverick by Roman Yepishev

Bug Description

Binary package hint: ubuntuone-client

Description: Ubuntu 10.04 LTS
Release: 10.04
ubuntuone-client:
  Installed: 1.2.1-0ubuntu2
  Candidate: 1.2.1-0ubuntu2
  Version table:
 *** 1.2.1-0ubuntu2 0
        500 http://au.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages
        100 /var/lib/dpkg/status
     1.2.1-0ubuntu1 0
        500 http://au.archive.ubuntu.com/ubuntu/ lucid/main Packages

Numerous times I've had the Ubuntu one sync daemon hit 100% CPU usage, trying to hash a file continuously. It commonly occurs when I'm working on a document in LyX, and the document is saved in a directory that is synchronised by Ubuntu One.

My syncdaemon.log files are filled with entries like so:

2010-05-15 16:13:59,199 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,225 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,242 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,262 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,288 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,313 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)
2010-05-15 16:13:59,338 - ubuntuone.SyncDaemon.sync - INFO - T:NONE:F 0ae49c73-1070-4ef7-8451-50178559a7f9 ['root'::'6524353a-167d-4166-8c1d-8f1d54ece41b'] ''Ubuntu One/University/ELEC4302/Assignment 1/qt_temp.XM1523'' | Called calculate_hash (In: T:NONE:F)

Restarting the daemon temporarily stops the hashing, but it starts up again next time I start editing a file.

Related branches

Same problem with me; in my case the directory is holding Kile/LaTeX files.

Duane Hinnen (duanedesign) wrote :

I am setting this confirmed based on the bug comments and reports from other users on irc about this issue.

thank you,
duanedesign

Changed in ubuntuone-client (Ubuntu):
assignee: nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)
importance: Undecided → Medium
tags: added: foundations+
Changed in ubuntuone-client (Ubuntu):
status: New → Confirmed

I am having the same problem. It happens when I modify a file stored in a folder that is synchronized with Ubuntu One, with either Lyx or QtiPlot.

This is really a annoying and showstopping bug for me.

Jose Joaquin (jalfarom) wrote :

Same problem for me, using Lyx.

Thanks

Dave Peticolas (dave-krondo) wrote :

This seems to be caused by a loop:

  1. File gets created.
  2. File gets written.
  3. Metadata gets created.
  4. File gets renamed.
  5. Hash queue fails (file no longer there)

Step 5 causes the file to be re-sent to the hash queue, which
keeps failing as the file is gone. This create a file and then rename
pattern is pretty typical of apps that save documents. I got it to happen
using rsync, which also uses temporary files which are then renamed.

One way to fix it is to check that the file is still accessible before re-attmempting
the hash. I have small patch here, sans tests, which at least fixes it for me.

https://code.launchpad.net/~dave-krondo/ubuntuone-client/lucid-updates-580855

Dave Peticolas (dave-krondo) wrote :

I have some more information about this problem. It's easy to trigger this bug with the attached code.
They key is to rename a non-ignored file to an ignored file (like one that ends with ~). The inotify processor
currently ignores this possibility so the rest of the state machine never learns the original file has gone
away.

I updated my branch to make the inotify processor synthesize a delete event for this case. I left the
check I put in as it just seems safer. I was, for example, able to trigger the bug by setting the file to
be totally unreadable after touching it.

Dave Peticolas (dave-krondo) wrote :

By the way, lyx saves files by writing to a temp file and then renaming it to the backup file, which ends with ~,
which explains why this happens so often with lyx.

Roman Yepishev (rye) on 2011-02-24
Changed in ubuntuone-client:
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)
Roman Yepishev (rye) on 2011-02-24
Changed in ubuntuone-client:
importance: Medium → High
tags: added: chicharra chicharra-natty
Changed in ubuntuone-client:
status: Confirmed → In Progress
Facundo Batista (facundo) wrote :

Dave, the HashQueue behaviour is correct... suppose you create a file A and rename the file to B.

HQ gets queued to hash A, but it's not there anymore! it should quit? (B will never be hashed) What it does is to send HQ_HASH_ERROR, so Sync will get that, will get the node_id, get the path (that will be B), and hash again the node (path B, now).

But, what happens if the HQ_HASH_ERROR is processed before the FS_FILE_MOVE? A is sent to process again! Normally, just a couple of times, until the FS_FILE_MOVE is processed.

In the case presented in this bug, we *do* have a bug, but it's not in HQ, it's in how the filesystem notifications are processed.

See, we're failing to issue a FS_FILE_DELETED when you rename something valid to something ignored.

So, as Sync never deletes the file, the HQ stucks in a loop.

Dave Peticolas (dave-krondo) wrote :

Facundo, I agree, I'm pretty sure I also made the change to generate FS_FILE_DELETED
when you rename in my patch, but it's been so long I can't remember exactly. I guess that's
the only change you need to fix it then?

Changed in ubuntuone-client:
status: In Progress → Fix Committed
Miłosz Kosobucki (mikom) wrote :

Sooo, when it's going to be released? This bug is killing my box. The fix is on the separate branch, it should be relased as soon as possible.

Matthew Gregg (mcg) wrote :

This seemed to be fixed and now it is back.

tags: added: u1-natty-beta
dobey (dobey) on 2011-03-10
Changed in ubuntuone-client:
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntuone-client - 1.5.6-0ubuntu1

---------------
ubuntuone-client (1.5.6-0ubuntu1) natty; urgency=low

  * New upstream release.
    - Remove already upstreamed patch
    - Release pathlock when command cancelled (LP: #729158)
    - syncdaemon continuously hashing (LP: #580855)
    - Self conflict when saving the same file a lot (LP: #718924)
    - SYS_QUOTA_EXCEEDED not being sent (LP: #720797)
    - The check_condition call may be too heavy (LP: #720844)
    - AQ_DOWNLOAD_CANCELLED is no more (LP: #721479)
    - SD should update a single notification (LP: #723227)
    - Two entries in Messaging menu (LP: #721525)
    - Singular form incorrect for notification (LP: #723327)
    - File names should be shown on notifications (LP: #729055)
  * Moved icon files to ubuntuone-client package (LP: #727195)
 -- Rodney Dawes <email address hidden> Wed, 09 Mar 2011 21:11:44 -0500

Changed in ubuntuone-client (Ubuntu):
status: Confirmed → Fix Released
Miłosz Kosobucki (mikom) wrote :

Any chance that this bugfix hits maverick version? Or do I have to wait for natty till April and hope my laptop doesn't melt by then?

marianitn (marianitn) wrote :

Is also Lucid version going to be fixed as a LTS release?

Same problem with Kile. Went to Settings > Configure Kile > Editor: Open/Save > Advanced and changed the suffix of "Backup on save" from ~ to "BACKUPONSAVE".

That cured the problem for me. I mean it's a workaround, not a clean solution. I'm also on Maverick and hoping that the bug will be released on that distrib.

Still seems to be a problem with 11.04 Natty - at least I'm getting 100% CPU from the syncdaemon making the machine unusable.

I fixed it by uninstalling ubuntuone-client and dependants using Synaptic Package Manager. Ubuntu One was just bloatware that I never asked for anyway. It should be optional to download, especially when it is buggy and causes hard-to-track problems like this.

Jim Raredon (decoy-umd) wrote :

This bug still exists on Raring and Saucy for me.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers