Comment 3 for bug 385495

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote : Re: [Bug 385495] Re: Large backup signature and manifest files should be split with --volsize too

Ross Patterson wrote:
> I've got signature splitting working on a branch with some caveats
> documented in http://bazaar.launchpad.net/~rossp/duplicity/signature-
> volsize/revision/575. Since the duplicity code and I aren't getting
> along, I won't be working on this further or using duplicity, but I
> wanted to leave my contributions here in case someone can pick them up
> and do something with them. But they should certainly be fully reviewed
> and the problems addressed before merging.

Thanks for the effort. Its all appreciated and I'll take a look at it.
 Sorry to hear you won't be using duplicity. Maybe later.

I'm not sure how the signature volume size problem should be handled.
That's one of the issues to address later. So far, the restriction on
file sizes has only hit IMAP users. I don't know of any serious size
restrictions on other protocols, but there may be some out there. The
usage of duplicity has been moving up the food chain, stressing larger
volume sizes to support larger backups with fewer volumes.

That said, one quick fix to the volume size problem is to output a
signature file with every data volume. This has an absolute guarantee
that the signature size will be smaller than the volume size, and its
easy to implement, however, the number of files open during incrementals
go from 1 sigtar per incremental to N sigtars per incremental. That
would break the max file open limits on a lot of the folks that want to
live dangerously and have few full backups, but lots of incrementals.
Like most engineering problems, its all in the tradeoffs.