Can't prevent full backup from being performed instead of incremental one

[Milan Bouchet-Valat]

Without any apparent reason, Déjà Dup decided today that it would be better to perform a full backup instead of an incremental one, even if previous backups were detected. But this fails by lack of free disk space.

Anyway, I don't want to do a full backup when the main purpose of using duplicity is to to incremental backups. If it can be wise to perform full backups from time to time for safety, at least give the user the choice not to do them.

(Further, it would be good to have free space detection so that the program doesn't suggest an operation that will fails, while the incremental backup would have succeeded... ;-)

deja-dup 16.1.1-0ubuntu1
duplicity 0.6.10-0ubuntu1

[Michael Terry (mterry)]

Deja Dup does have free space detection logic. It will try to delete older backups as long as it can still keep two full backups on the disk. That's another reason for making full backups -- it lets DD delete old chains and save space.

But if it can't delete anything without losing on of your last two full backups, it will just try anyway and hope for the best. Sounds like that's what you're hitting. Is your backup location not especially large?

[Milan Bouchet-Valat (nalimilan)]

I don't remember exactly now, but I said in the bug report that there wasn't enough free space to perform a full backup, so the disk space must have been low when I got this bug. Anyway, I see several ways of improvement:
1) Tell the user what is happening: this is needed, even if you don't give the choice, show somewhere a reason. If it's because of lack of free space, people might prefer to free some space manually and run the backup again.
2) Check that there's enough free space before trying to do a full backup: if it fails while an incremental backup would have succeeded, there's obviously a problem! (But I'm not sure you know in advance how much space you'll need.) This leads to:
3) Looks like the full backup was added to the older ones, rather than replacing (part of) them: in these conditions, it was a waste of space instead of a gain. If the user is asked before (cf. point 1), then doing a full backup to replace the older ones is a smart behavior (but only in that case).

Thanks for your work anyway, I use DéjàDup in other cases and it's really a nice tool.

[Damon T (damon-tigronconsulting)]

Well I'll chime in just to help get some type of resolution to this. I completely agree with Milan Bouchet-Valat (nalimilan) on all three points. It is also very anoying to get a no space left message when you have more than enough space. The fact that it keeps trying to run full backups without telling you and not incremental is just bullocks. The CPU used during the process makes this an unusable solution for me clocking in at 13hrs for full backups only to fail with no space left messages.

[dg1727 (dg1727)]

It seems like there are about 3 kinds of issue being described in this bug. In the order they are mentioned in comment #2:

1. Notifying the user whether a full or incremental backup is about to be done, or is in progress. Allowing the user to postpone a full backup (that is, do an incremental backup this time instead).
2. Free-space detection logic.
3. Backup rotation logic.

I am interested in item 1 from that list. Some features not stated yet:
 a. Allowing the user to change a full backup to incremental even while the full backup is in progress. This could simply involve deleting the full backup in progress and restarting the backup as an incremental.
 b. Telling the user how long it took for the previous full backup and, say, the average & max of the last 5 incremental backups (and the date of the one that took the max time), in case the user normally doesn't pay attention while the backup is running, but today they have a time constraint & want to make a backup quickly.
 c. Having a preference for whether to pause before every full backup to let the user postpone the backup, or just start without prompting the user and rely on item (a.) if the user notices that this backup will take more time than they have today.
 d. Having a schedule of maybe 3 increasingly severe warnings for when the user postpones a full backup. Something like: show 1 dialog with a fairly bland warning; then after a couple of postponements, add a 2nd dialog with a more alarming warning (larger font, boldface, exclamation-symbol-inside-triangle icon); then after a couple more postponements, add a 3rd dialog which requires an admin password (or the user can click a button to do a full backup without the password). The password prompt should allow the password of any admin user on the machine, as can be done via "pkexec."

Please let me know if I should open a separate bug for this issue 1, to separate it from 2 & 3 (even though the title of this bug is closest to issue 1).

[Von (daaxix)]

I have backups set to once a week, but I missed last weeks, so now it is "creating a first backup" yet again, which takes 22+ hours.

It also just created a "first backup" the week prior to the one that I missed, so I don't understand why it is doing it yet again, only 3 weeks later.

[Michael Terry (mterry)]

I made a discussion ticket on GitLab to gather the various suggestions we’ve received around more advanced scheduling options. I’m going to close this in favor of tracking the suggestions there.

https://gitlab.gnome.org/World/deja-dup/-/issues/111