Deja Dup should use inotify or Zeitgeist to get the list of changed files

[Anders Wallenquist]

One of the most annoying things with Deja Dup on my laptop are that every time I boot it, for the first hour it works with deja dup / duplcity to check what was changed since last time before the actual backup takes place. During this time half of the CPU capacity and a large amount of IO are taken for the task. Backup are important and a wonderful peice of software as deja dup / duplicty takes a hit by this, and may be interupted to many times when the load stops normal work.

You have certenly got this question many times, inotify-system have been around for a long time now. With inotify we can get a message when a file are changed and take action. For example we can build a small list of what files are changed since last backup and use this instead of the time consuming task to check every file in the file system with the remote backup. With this list we can use --include-files-directive in dublicity and start copying directly. Or we can implement continuis backup, start backing up every changed file at the time the user has closed it (or wait a couple of minutes not to build a long queue of backups when the user changes the file eventy minute or so).

There are alot of pros using inotify (or incron), one of the bad thing that has bo be solved are that we have to let kernel listen at all directories in a directory structure. Now we only have to configure for example /home/user-directory to include all sub directories for that user. So we need a mechanism that builds a list of directories for inotify including changes over time.

Another solution would be to use Zeitgeist and listen for changed files using dbus. The upside whould be that deja dup can use the same privacy configuration that Zeitgeist uses not to distribute secret files to the backup site. Ive been programming with inotify and know there are no overhead and its very easy to handle, but I don't have the same experience of Zeitgeist api so I can't tell if its easy or not.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas E. (andreas-e)]

Usually scans take a multiple of the time of the actual copying phase (more than an hour). This is a severe reason that entices users not to run the backup tool more often.

I read somewhere an objection is that by relying on inotify deja-dup would not backup files that have been changed externally (dual boot, hard drive connected to another system). Probably a huge majority of users does not dual boot and would benefit from this feature without any risks.

For me personally, the chance of having missed a file is less likely than the chance of not having backed up often enough. I run deja-dup weekly. I am missing an adequate backup solution for hourly or immedite backups.