wishlist: differential backups

[az]

this is a forward/copy of debian bug #550698 (which lives here: http://bugs.debian.org/550698)
where the requester asks for differential backups to shorten the backup chain.

the requester, however, also asks for full control over which earlier backup
to use as base (for example a differential against an incremental backup)
which i don't believe makes much sense.

here's a copy of the request:
---cut---
I like how duplicity works, but having overlong backup chains to conserve
bandwidth/space puts too much pressure on the reliability of the remote
destination. I would like to keep the chains shorter, but can't due to space
constraints as the only alternative is a periodic full backup.

Is there a reason as of why "incremental" could not accept a "-t" argument to
specify which archive to use as the previous archive and let me manage the
storage by myself?

If I had a time or filename specifier like this, I could implement a
differential backup script on top of duplicity without any special handling on
duplicity's part, much in the same way as you implement differential tar
archives.

As long as "restore" walks the backup chain from the most recent volume, it
should also work unchanged.

I've seen some old comments in the duplicity mailing list, but no proper
feature request. I'm also filing this here for other Debian users to see,
since the inability to perform differential archives kinds-of defeats the
initial space gain you have with duplicicy.

---cut---

[Ole Martin Handeland (j-launchpad-olemartin-org)]

I second this. Currently, this is kind of a show-stopper for me, as having long chains of incremental backups increase complexity, restore time and will be more unreliable.

I don't really need the full control over which earlier backup to use as base (e.g. backing up changes since the beginning of this month: duplicity -t 2011-07-01T00:00:00+02:00 <source> <target>), but what i want is to specify a differential backup instead of full/incremental auto, as the default is now (e.g. backing up changes since last full: duplicity differential <source> <target>).

I'm backing up my home directory, with a lot of media, documents and stuff. This is ~50-100 GB, and a full remote backup takes time even with a >10gbit connection. Changes to these are pretty small, probably ~1 MB daily. Running for months with an ever increasing trail of incremental backups kind of scares me - what if one of them breaks? This guy/gal had that happening: http://forum.slicehost.com/comments.php?DiscussionID=3664

You could even have a "duplicity differential --full-if-larger-than 50M <source> <target>" and/or duplicity hinting that it might be time for a full backup.

I'll be on a trip for about a month now, but I'll come back and see what i can do to implement this after that.

[Ole Martin Handeland (j-launchpad-olemartin-org)]

Well, i ended up with using rsnapshot instead. It would be cool if rsnapshot also supported backup of just the changed parts of files, but as of now the lack of differential backups and thus my insecurity in whether or not a restore will fail when i need it make it more attractive to use another backup system instead of duplicity. Also, i just love rsnapshots use of hardlinks.. :-)