only full backups done on webdav

[SanskritFritz]

Duplicity version 0.6.05
Python version 2.6.4
OS Distro and version: ArchLinux, Kernel 2.6.31-BFS patch
Type of target filesystem: webdav

The command below just doesnt find previous backups on the remote server, regardless of how many times I run it. Doing the same on local file:// target works ok.

~/tmp> duplicity . webdav://user:<email address hidden>/DuplicityBackup
GnuPG passphrase:
Synchronizing remote metadata to local cache...
Deleting local /home/user/.cache/duplicity/364814ad3a0ebc24a84de943b696544f/duplicity-full.20091122T005517Z.manifest (not authoritative at backend).
Last full backup date: none
No signatures found, switching to full backup.
Retype passphrase to confirm:
--------------[ Backup Statistics ]--------------
StartTime 1258851492.44 (Sun Nov 22 01:58:12 2009)
EndTime 1258851492.45 (Sun Nov 22 01:58:12 2009)
ElapsedTime 0.01 (0.01 seconds)
SourceFiles 2
SourceFileSize 36910 (36.0 KB)
NewFiles 2
NewFileSize 36910 (36.0 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 46 (46 bytes)
TotalDestinationSizeChange 268 (268 bytes)
Errors 0
-------------------------------------------------

[SanskritFritz (sanskritfritz+launchpad)]

The problem persists with Duplicity 0.6.06

[Smoerk (smoerk)]

Exact same problem with Duplicity 0.5.18-0ubuntu1 on Ubuntu Karmic. Tested with is SabreDAV 1.01.

[Kenneth Loafman (kenneth-loafman)]

Can you list the contents of the WebDAV directory you're using for backup? I cannot reproduce this.

[SanskritFritz (sanskritfritz+launchpad)]

Thanks for investigating this.

Here is the filelist after two runs:

duplicity-full.20100218T183756Z.manifest.gpg
duplicity-full.20100218T183756Z.vol1.difftar.gpg
duplicity-full-signatures.20100218T183756Z.sigtar.gpg
duplicity-full.20100218T183733Z.manifest.gpg
duplicity-full-signatures.20100218T183733Z.sigtar.gpg
duplicity-full.20100218T183733Z.vol1.difftar.gpg

You can freely register here to reproduce the problem:
https://www.mydrive.ch/en

Is there a simple local webdav server to install, I'll try to reproduce it this way.

[daniel_L (daniel-das-grauen)]

I'm only getting this bug when using webdav on mydrive.ch whereas using webdav on gmx.net (you get a "mediacenter" with webdav access when creating an email account) works without any problems. I'm attaching logs for both services with verbosity=9 after this comment.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 8.10
Release: 8.10
Codename: intrepid

$ uname -r
2.6.27-17-generic

[daniel_L (daniel-das-grauen)]

$ duply test-mydrive backup

[daniel_L (daniel-das-grauen)]

$ duply test-mydrive status

[daniel_L (daniel-das-grauen)]

$ duply test-mydrive incr

[daniel_L (daniel-das-grauen)]

$ duply test-gmx backup

[daniel_L (daniel-das-grauen)]

$ duply test-gmx status

[daniel_L (daniel-das-grauen)]

$ duply test-gmx incr

[SanskritFritz (sanskritfritz+launchpad)]

Thanks daniel_L for testing.
One more thing: when I use davfs on mydrive.ch like this, it works ok:
cd /mnt/MyDrive.ch/DuplicityBackup/
duplicity /home/user/tmp/source/ file://.

[Florian Wiesweg (flow)]

Can cofirm it. MyDrive.ch doesn't work, gmx mediacenter does.

[Florian Wiesweg (flow)]

Sry, forgot adding version and all that stuff:
Ubuntu 10.04 Lucid RC
Python 2.6.5
duplicity 0.6.08b
Hope someone can solve this issue, it's somewhat a show-stopper if you want to do an encrypted backup to mydrive.ch's free space.

[SanskritFritz (sanskritfritz+launchpad)]

"This bug report was marked for expiration"
I hope this will bump it back, please dont let this expire. Even though i use duplicity happily with mydrive.ch mounted wia davfs, I really would like to have a solution.

[Kenneth Loafman (kenneth-loafman)]

I'm not sure there is a solution. The logs show the problem lies in mydrive.ch -- when asked for a list of files on the remote, it is supposed to return the list, MyDrive returns nothing. As good as duplicity may be, nothing means nothing.

[SanskritFritz (sanskritfritz+launchpad)]

I see. But on the other hand, we assumed that duplicity has a bug, for a reason: every other client I tried works well, including davfs, KDE kioslave, webdrive in windows.
Can I have that logfile, or can I generate one for myself? I will contact the support at mydrive.ch.

[Kenneth Loafman (kenneth-loafman)]

Sure, you can use the logfile(s) or make your own.

Duplicity works on most of the Webdav servers that we know about, but
I'm sure that's not all of them. I'm also fairly sure that the Webdav
protocol now, like the FTP protocol did, is diverging as it is
implemented by more programmers.

Let me know what you find. I appreciate you taking the initiative on this.

SanskritFritz wrote:
> I see. But on the other hand, we assumed that duplicity has a bug, for a reason: every other client I tried works well, including davfs, KDE kioslave, webdrive in windows.
> Can I have that logfile, or can I generate one for myself? I will contact the support at mydrive.ch.
>
>

[SanskritFritz (sanskritfritz+launchpad)]

I dont agree with you about the filelist being empty.

I did a test, and also checked the logfiles above sent by daniel_L. Both logs contain the filelist in the xml section, and then it says "0 files exist on backend". Odd. Maybe the xml format provided by sabredav is not standard, or the parsing routine has problems in duplicity? How can I check this? One thing that catches my attention, compared to the GMX log is that sabredav returns the xml on one line without linebreaks, causing one very long line. Maybe that confuses the parser? I miss the "webdav path decoding and translation" lines from the log as well.

<d:multistatus xmlns:d="DAV:" xmlns:s="http://www.rooftopsolutions.nl/NS/sabredav"><d:response><d:href>/Backups/test/</d:href><d:propstat><d:prop><d:getlastmodified xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" b:dt="dateTime.rfc1123">Thu, 08 Apr 2010 07:42:13 +0000</d:getlastmodified><d:resourcetype><d:collection/></d:resourcetype></d:prop><d:status>HTTP/1.1 200 Ok</d:status></d:propstat></d:response><d:response><d:href>/Backups/test/duplicity-full-signatures.20100408T081531Z.sigtar.gpg</d:href><d:propstat><d:prop><d:getlastmodified xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" b:dt="dateTime.rfc1123">Thu, 08 Apr 2010 08:15:35 +0000</d:getlastmodified><d:getcontentlength>1331</d:getcontentlength><d:resourcetype/><d:getetag>56c4f6ada945725180eaef05e37d23a0a09c23b2</d:getetag><d:getcontenttype>text/PGP</d:getcontenttype></d:prop><d:status>HTTP/1.1 200 Ok</d:status></d:propstat></d:response><d:response><d:href>/Backups/test/duplicity-full.20100408T081531Z.manifest.gpg</d:href><d:propstat><d:prop><d:getlastmodified xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" b:dt="dateTime.rfc1123">Thu, 08 Apr 2010 08:15:36 +0000</d:getlastmodified><d:getcontentlength>829</d:getcontentlength><d:resourcetype/><d:getetag>2ad1dd128ef12dc5bbdc235220c121bbdd86bb5c</d:getetag><d:getcontenttype>text/PGP</d:getcontenttype></d:prop><d:status>HTTP/1.1 200 Ok</d:status></d:propstat></d:response><d:response><d:href>/Backups/test/duplicity-full.20100408T081531Z.vol1.difftar.gpg</d:href><d:propstat><d:prop><d:getlastmodified xmlns:b="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/" b:dt="dateTime.rfc1123">Thu, 08 Apr 2010 08:15:35 +0000</d:getlastmodified><d:getcontentlength>14679</d:getcontentlength><d:resourcetype/><d:getetag>5b4276b2f23b7732e92bf16038f78208d8855ffe</d:getetag><d:getcontenttype>text/PGP</d:getcontenttype></d:prop><d:status>HTTP/1.1 200 Ok</d:status></d:propstat></d:response></d:multistatus>

0 files exist on backend
0 files exist in cache

[SanskritFritz (sanskritfritz+launchpad)]

What do you think about my comment above?

[Kenneth Loafman (kenneth-loafman)]

Looks like sabredev is using the tag name 'd:href' instead of 'D:href'. Would you mind making the following change in webdavbackend.py at line 184:

change:
        for href in dom.getElementsByTagName('D:href'):
to:
        for href in dom.getElementsByTagName('d:href'):

and let me know if this works. If it does, the fix is fairly straightforward to handle both cases.

[SanskritFritz (sanskritfritz+launchpad)]

This was worth the wait, thanks!
I patched the file as described above, tested the backup and the restore (including --restore-time) functions, everything worked flawlessly this way.
Thank you for the fix.

[Lutz Niggl (lutz-niggl)]

Same for webdav.mediencenter.t-online.de. Changing all D: into d: fixed the issue.
Thanks, works perfect now