Consolidate backup files into one file

[lunch]

When Deja Dup backs up to a remote location it creates a LOT of files (like 1,000+ files) on the remote folder. It is probably better to have Deja Dup back-up your files to one file on the remote folder.

[Simon Schenk (sschenk)]

True. My online storage provider does not handle large numbers of files well, so I often get timeouts.

[Michael Terry (mterry)]

It's all a trade-off. I'll quote from the source code:

    // Advantages of a smaller value:
    // * takes less temp space
    // * retries of a volume take less time
    // * quicker restore of a particular file (less excess baggage to download)
    // * we get feedback more frequently (duplicity only gives us a progress
    // report at the end of a volume) -- fixed by reporting when we're uploading
    // Downsides:
    // * less throughput:
    // * some protocols have large per-file overhead (like sftp)
    // * the network doesn't have time to ramp up to max tcp transfer speed per
    // file
    // * lots of files looks ugly to users

Deja Dup will not create one giant file. But I'm open to changing the size of the volumes we create. On remote shares, we currently make volumes of 10M. Simon, which storage provider do you use? I'm curious to hear reports of issues with many files.

[Wilhelm (wilhelm-mailaender)]

Maybe every backup session can store the data in a seperate folder?

[Launchpad Janitor (janitor)]

[Expired for Déjà Dup because there has been no activity for 60 days.]