Songs not downloading, even after a local and server rescan, but available via the web

Reported by Guillermo Gonzalez on 2010-04-29
136
This bug affects 24 people
Affects Status Importance Assigned to Milestone
Ubuntu One Client
Status tracked in Trunk
Stable-1-2
Critical
Guillermo Gonzalez
Trunk
Critical
Guillermo Gonzalez
Ubuntu One Servers
Critical
Guillermo Gonzalez
ubuntuone-client (Ubuntu)
Critical
Rodney Dawes
Lucid
Undecided
Unassigned
Maverick
Critical
Rodney Dawes

Bug Description

jono and bac reported a very similar issue, songs (or entire albums) not downloading but available via the web.
So, the files are on the server but the client isn't downloading them.

-----------------------------------------------

WORK AROUND:
Open Applicaitons->Accessories->Terminal and run: touch ~/.ubuntuone/Purchased\ from\ Ubuntu\ One/placeholder; u1sdtool -q; u1sdtool -c

This will put a "placeholder" file in your ~/.ubuntuone/Purchased from Ubuntu One/ folder and then force the syncdaemon to reconnect and try to sync that folder.

-----------------------------------------------

TEST CASE:
Need to clients to test. Client A & B which are referenced below.

1) Sync Client A
# then start a single client (a.k.a: A) and create a UDF, e.g:
u1sdtool -c
u1sdtool -s (wait to reach idle)
u1sdtool --create-folder=$HOME/MyUDF
u1sdtool -s (wait to reach idle)
# create some files and dirs inside $HOME/MyUDF
mkdir $HOME/MyUDF/subdir
mkdir $HOME/MyUDF/subdir2
touch $HOME/MyUDF/subdir/bar
touch $HOME/MyUDF/subdir2/foo
# create a file in the root
touch $HOME/MyUDF/foo
u1sdtool -s (wait to reach idle)

2) Sync Client B
# start a second client (probably a different machine), a.k.a: B
u1sdtool -c
u1sdtool -s (wait to reach idle)
# it should download all the files/dirs created in client A
# quit the client B
u1sdtool -q

3) Sync new file on Client A
# on client A, create a new file/dir in the root
touch $HOME/MyUDF/bar
u1sdtool -s (wait to reach idle)
# quit and restart the client A to make sure everything it's ok
u1sdtool -q
u1sdtool -c
u1sdtool -s (wait to reach idle)
# now get the info of the UDF root metadata
u1sdtool --info=$HOME/MyUDF
# keep the output around, we are going to need the 'server_hash' value for the next step

4) Run md_server.py script (http://launchpadlibrarian.net/48350344/md_server.py)
# on client B, we need to fake a interrupted download
# use the server_hash from the previous step (from client A)
python md_server.py $HOME/MyUDF <server_hash>

5) Sync Client B (again)
# still on client B, let's start syncdaemon again
u1sdtool -c
u1sdtool -s (wait to reach idle)
# with the current version of ubuntuone-client we should see that the new file ($HOME/MyUDF/bar) wasn't downloaded on client B

6) Sync Client B with fix
# using the fix on #571548, we should get the new file downloaded to client B
# to check this, run:
u1sdtool -q
# get a fresh copy of trunk or the branch linked to #571548 and run:
ln -s /usr/lib/python2.6/dist-packages/ubuntuone/storageprotocol ubuntuone/storageprotocol
PYTHONPATH=. bin/ubuntuone-syncdaemon &
u1sdtool -c
u1sdtool -s (wait to reach idle)

Expected result: $HOME/MyUDF/bar is on Client B

-----------------------------------------------

Related branches

lp:~verterok/ubuntuone-client/fix-571548
Merged into lp:ubuntuone-client/stable-1-2 at revision 511
Rodrigo Moya (community): Approve on 2010-05-06
Tim Cole (community): Approve on 2010-05-04
Natalia Bidart: Approve on 2010-04-30
lp:~verterok/ubuntuone-client/fix-571548-trunk
Merged into lp:ubuntuone-client at revision 515
Tim Cole (community): Approve on 2010-05-04
Natalia Bidart: Approve on 2010-04-30
tags: added: foundations+
removed: founda
Changed in ubuntuone-servers:
assignee: nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)
Guillermo Gonzalez (verterok) wrote :

To work around this, the artist folder was created manually, and a file was created inside it.
IMO, only the first step was needed to trigger the download.

Guillermo Gonzalez (verterok) wrote :

Dump of syncdaemon metadata before creating the artist directory.

Guillermo Gonzalez (verterok) wrote :

syncdaemon metadata dump after downloading the missing files

Changed in ubuntuone-servers:
status: New → Invalid
Changed in ubuntuone-client:
importance: Undecided → Critical
assignee: nobody → Ubuntu One Foundations+ team (ubuntuone-foundations+)
status: New → Triaged
assignee: Ubuntu One Foundations+ team (ubuntuone-foundations+) → Guillermo Gonzalez (verterok)
Facundo Batista (facundo) wrote :

We need to handle Roots (for U1, Shares and UDFs) in a special way when they're in SERVER state after an interrumpted syncdaemon when it was downloading new data.

That should be done in LR, and tests for this case for the three roots should be added.

This will be done by Guillermo, we talked extensively about this this morning.

Changed in ubuntuone-servers:
assignee: Ubuntu One Foundations+ team (ubuntuone-foundations+) → Guillermo Gonzalez (verterok)
Changed in ubuntuone-client:
status: Triaged → In Progress
tags: added: u1-lucid-sru
Changed in ubuntuone-client:
status: In Progress → Fix Committed
Changed in ubuntuone-client (Ubuntu):
importance: Undecided → Critical
status: New → Confirmed
milestone: none → lucid-updates
assignee: nobody → Rodney Dawes (dobey)
description: updated
description: updated
Joshua Hoover (joshuahoover) wrote :

md_server.py script used in test case.

description: updated
description: updated
description: updated
Joshua Hoover (joshuahoover) wrote :

Ran test case and got expected results with the branch. I'll test the proposed update once it's there.

description: updated
visibility: private → public
Otto Kekäläinen (otto) wrote :

I bought the album Crazy Love by Michael Bublé around the first of May 2010. This was my first and possible last purchase from U1 music store.

The payment went ok and the files appeared in the cloud (visible from the U1 web interface) but never synced to any of my local computers. The folder "~/.ubuntuone/Purchased from Ubuntu One/" stayed empty no matter what I tried (e.g. reinstalling U1 and my account). Other files in the normal Ubuntu One folder synced fine. My computer is on several hours every day so it wasn't a matter of waiting long enough. Command u1sdtool -s showed idle all the time.

Finally today I got help from verterok and alecu on the #ubuntuone IRC channel. Running mkdir "Michael Bublé" in the folder "~/.ubuntuone/Purchased from Ubuntu One/" first destroyed the files in the cloud. Luckily there was an option to download the files again from the store, so clicking "download it again" in the U1 music store interface in Rhythmbox fixed it (except for one song, see below). Now the songs are in the cloud and they synced to one of my local computers.

I went to check the situation on another computer where I have the same U1 account: the folder "~/.ubuntuone/Purchased from Ubuntu One/" on that computer was still empty but running "touch test" in that folder somehow poked the U1 daemon and the sync started.

There is just one problem left: when I click to download the song "Baby [You've Got What It Takes] [with Sharon Jones & Dap-Kings]" (by Michael Bublé on album Crazy Love) in the Rhythmbox U1 music store interface, the page just reloads but no transfer is started.. I have to complain about that in another bug report.

But I guess the workaround to a local folder not syncing is to run "touch newfile" in it to wake up the sync calculation.

description: updated
Roman Yepishev (rye) on 2010-05-18
tags: added: current-trouble
Rodney Dawes (dobey) wrote :
John Dong (jdong) wrote :

ACK from ubuntu-sru

Accepted ubuntuone-client into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in ubuntuone-client (Ubuntu Lucid):
status: New → Fix Committed
tags: added: verification-needed
Martin Pitt (pitti) wrote :

Incidentally I purchased an album this morning (mp3), and as usual it got stuck in the "Queueing" state for a bit (some 30 minutes?) and then switched to "Transferring to Ubuntu One..." without further action (now 3 hours ago). I updated to this proposed version and restarted session (and also restarted ubuntuone in the prefs dialog just for testing), but it still doesn't start downloading. I see the files on one.u.c., but not in ~/.ubuntuone/Purchased\ from\ Ubuntu\ One/. Wasn't that what this update was supposed to fix?

Geoff123 (gsking1) wrote :

I just purchase an album today and have run into this problem. I've tried the proposed update and it is still not working. I also tried the workaround at the top of this page, rebooting, restarting the ubuntuone client, etc. but those didn't help. About the same behavior as Martin Pitt above. I can download the songs from the ubuntu files webpage, but there appears to be no other syncing going on.

Matt Griffin (mattgriffin) wrote :

Geoff123: File sync between the Ubuntu One cloud and your desktop is currently very slow. This is noted on our Status page: https://wiki.ubuntu.com/UbuntuOne/Status#Files

In addition to purchased music, this affects all files that customers are trying to sync. The issue lies on our servers and we're close to a solution. Workarounds on your desktop for this problem will probably not work.

In the meantime, purchased songs should be available for download from your personal cloud at https://one.ubuntu.com/files immediately after purchase. It's not ideal, but it will allow you to get your purchased music until we fix the slow file sync issue.

Just purchased an album. Luckily for me the "placeholder workaround" did work fine.

Daniel Holbach (dholbach) wrote :

With the new ubuntuone-client packages, the workaround seems to work for me.

Paul D Smyth (pauldsmyth) wrote :

Applied the new client, seemed to catch everything up. Purchased an album, downloaded immediately. Seems to be working properly now

papukaija (papukaija) on 2010-06-07
tags: added: patch
Martin Pitt (pitti) on 2010-06-08
tags: added: verification-done
removed: verification-needed
Martin Pitt (pitti) wrote :

Please fix this in maverick ASAP (or close the maverick task if it already is).

Changed in ubuntuone-client (Ubuntu):
milestone: lucid-updates → maverick-alpha-2
milestone: maverick-alpha-2 → none
status: Confirmed → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntuone-client - 1.2.1-0ubuntu3

---------------
ubuntuone-client (1.2.1-0ubuntu3) lucid-proposed; urgency=low

  * Add fix-571548.patch
    - Fixes songs not downloading but available on web issues (LP: #571548)
 -- Rodney Dawes <email address hidden> Tue, 01 Jun 2010 15:31:58 -0400

Changed in ubuntuone-client (Ubuntu Lucid):
status: Fix Committed → Fix Released
Martin Pitt (pitti) on 2010-06-11
Changed in ubuntuone-client (Ubuntu Maverick):
milestone: none → maverick-alpha-2
Rodney Dawes (dobey) wrote :

The fix is built in maverick already, but the packages seem to still be pending publication to the archive. Not sure why.

Changed in ubuntuone-client (Ubuntu Maverick):
status: In Progress → Fix Released
yajo (yajodj) wrote :

Same bug here.
Running Ubuntu 10.04.
Workaround worked for me too. Thanks.

E King (eakst7) wrote :

Sorry for the "me too", but this is still broken for me on 10.04. This was my first attempt at using the music store. The "placeholder" workaround gave it the kick it needed.

Changed in ubuntuone-client (Ubuntu Lucid):
status: Fix Released → Confirmed
Justin (ja-space) wrote :

Just got same error with 10.04 Lucid. First go at buying music so really disappointing, but work around did work...

Rodney Dawes (dobey) wrote :

If you think you are seeing this same issue with lucid, with ALL updates applied please file a new bug. This bug has been fixed and verified, so if you're seeing a similar issue, we should track it in a new bug, which might be a regression. Please file a new bug and ensure appropriate logs are attached to it. Please don't reopen bugs that were fixed months ago. Thanks.

Changed in ubuntuone-client (Ubuntu Lucid):
status: Confirmed → Fix Released

As I am affected by this bug still (and the workaround fixes it) I have reported bug 648396. Please go over there to confirm it and "me, too" it.

tags: added: testcase
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Bug attachments