Uploads to Rackspace fail silently

Bug #832149 reported by Scott Severance
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
Fix Released
Medium
Unassigned
Déjà Dup
Invalid
Undecided
Unassigned
duplicity (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

I have two machines scheduled to backup daily to Rackspace. One has a much larger backup set than the other. Besides that, there are no other relevant differences between the machines that I know of. The machine with the larger backup set is the one at issue. I've recently discovered that Deja Dup is silently failing to upload the backups. By that, I mean that Deja Dup gives every appearance of success (my network monitor even shows appropriate network traffic), and I can't find anything amiss in the debug log (though since it's > 3m lines long, I might have missed something). However, despite the lack of error messages, there's also no success confirmation, and the files are demonstrably not on Rackspace.

I suspect that there's some error, but is there appropriate error checking? It would appear that there isn't.

Here's some additional info. I don't know whether it belongs here or in a separate bug. According to the Rackspace Cloud Files documentation, a single file listing will return a maximum of 10,000 files and states that multiple requests are necessary if there are more than 10,000 files. But Deja Dup apparently only makes a single request and suspiciously retrieves exactly 10,000 files from that request.

I've previously reported bug 824583 and bug 824570. I think that this bug goes a long way in explaining the previous bugs. The only part of the requested standard info I'm including with this bug is a full log file showing the upload apparently succeeding (although the files aren't on Rackspace). I'm not duplicating the other info since you can find it in the other bugs I reported.

Finally, I contacted Rackspace to see if they had any relevant logs. They don't.

Related branches

Revision history for this message
Scott Severance (scott.severance) wrote :
Revision history for this message
Michael Terry (mterry) wrote :

Great research, Scott! I bet you're right, that the duplicity rackspace backend is not making multiple requests. I'll look at it.

Revision history for this message
Michael Terry (mterry) wrote :

Can you try this new version of cloudfilesbackend.py and let me know if it fixes things for you?

Revision history for this message
Michael Terry (mterry) wrote :

(put it in /usr/share/pyshared/duplicity/backends/)

Revision history for this message
Scott Severance (scott.severance) wrote :

So far, so good. Thanks. But I think I should wait a couple days before making any definitive statements, because I've gotten a few tracebacks and odd behaviors since using the new file. My theory is that they're caused by the effects of the previous bugs, but as I'm not yet certain I prefer to wait a bit longer until I have more data.

Revision history for this message
Scott Severance (scott.severance) wrote :

After several days of successful backups, I can confirm that the patch fixes this bug. Thanks.

Changed in duplicity:
importance: Undecided → Medium
milestone: none → 0.6.16
status: New → Fix Committed
Michael Terry (mterry)
Changed in deja-dup:
status: New → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package duplicity - 0.6.15-0ubuntu2

---------------
duplicity (0.6.15-0ubuntu2) oneiric; urgency=low

  * Backport some upstream fixes
  * debian/patches/06_use_passhprase.dpatch:
    - Don't prompt for passphrase if PASSPHRASE is set (LP: #836467)
  * debian/patches/07_large_rackspace_list.dpatch:
    - Support listing more than 10,000 files from Rackspace servers
      (LP: #832149)
  * debian/patches/08_check_volumes.dpatch:
    - Attempt to prevent a data corruption bug that can't be reproduced
      currently by detecting it up front and stopping the backup.
 -- Michael Terry <email address hidden> Thu, 08 Sep 2011 09:10:57 -0400

Changed in duplicity (Ubuntu):
status: New → Fix Released
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.