Incremental backup prevents "restore-missing" from restoring files, that were deleted before

[Vej]

If I do the following, I get an empty list.

1. Perform a fresh backup to a new folder (set a password for encryption).
2. Create the folder test and the file test/testfile.txt.
3. Restart the Backup (perform an incremental Backup)
4. Delete the file.
5. Restart the Backup (perform an incremental Backup)
6. Try to restore the file using LC_ALL=C deja-dup --restore-missing test/

Result: The list is empty.

I expect to see the file testfile.txt in the list, so I can restore it using the backup created in Step 3.

Some other users of the German community verified this bug (see https://forum.ubuntuusers.de/topic/fragen-zu-d-j-dup/ for an example - in German).

I already mentioned this in bug #1377873, but that one is marked as "Fix Released", so ML suggested to open a new one.

Ubuntu 16.04.1 LTS
deja-dup 34.2-0ubuntu1
duplicity 0.7.06-2ubuntu2

gsettings list-recursively org.gnome.DejaDup:

org.gnome.DejaDup last-restore '2015-11-01T21:37:49.910024Z'
org.gnome.DejaDup periodic true
org.gnome.DejaDup full-backup-period 75
org.gnome.DejaDup backend 'file'
org.gnome.DejaDup last-run '2016-08-07T10:21:31.709004Z'
org.gnome.DejaDup nag-check '2016-07-31T21:55:33.643340Z'
org.gnome.DejaDup prompt-check '2013-01-27T20:24:01.762209Z'
org.gnome.DejaDup root-prompt true
org.gnome.DejaDup include-list <A list including my home, /etc and a few additional locations>
org.gnome.DejaDup exclude-list <A list excluding Trash and a few directories>
org.gnome.DejaDup last-backup '2016-08-07T10:21:31.709004Z'
org.gnome.DejaDup periodic-period 1
org.gnome.DejaDup delete-after 730
[...]
org.gnome.DejaDup.File path 'file:///media/user/extern/Deja-Backup'
org.gnome.DejaDup.File short-name 'extern'
org.gnome.DejaDup.File uuid <a valid UUID>
org.gnome.DejaDup.File icon '. GThemedIcon drive-harddisk-usb drive-harddisk drive'
org.gnome.DejaDup.File relpath b'Desktop/extern'
org.gnome.DejaDup.File name 'Vendor External USB 3.0: extern'
org.gnome.DejaDup.File type 'volume'

[Ben (benpine)]

I suspect this is a bug in the underlying Duplicity and not in Deja-Dup itself.
I seem to be affected by the same issue you describe with deleted files not being available for restore after an incremental backup.
I followed the same basic steps you mentioned above, but by calling duplicity directly in the terminal.

1. Perform a full backup of a test dir with two test files to a new bkup_dir. (file:///mnt/bkup_dir/)
2. Verify the backup has the two test files with the duplicity list-current-files command
3. Delete one of the two test files.
4. Restart the duplicity Backup (perform an incremental Backup)
5. Now the duplicity list-current-files command only shows the un-deleted file. The file deleted before the incremental backup is not available for restore.
(The terminal output below shows this process on my Ubuntu 16.04 LTS laptop)

Extra. If I delete the incremental backup files that duplicity created from step 4, leaving only the original full backup files, then duplicity will list both test files available for restore.

testuser@systemname:~$ mkdir TestOrigin
testuser@systemname:~$ cd TestOrigin
testuser@systemname:~/TestOrigin$ duplicity --version
duplicity 0.7.06
testuser@systemname:~/TestOrigin$ uname -a
Linux systemname 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:07:23 UTC 2017 i686 i686 i686 GNU/Linux
testuser@systemname:~/TestOrigin$ touch test1.txt test2.txt
testuser@systemname:~/TestOrigin$ ls -al
total 8
drwxrwxr-x 2 testuser testuser 4096 Apr 15 20:23 .
drwxr-xr-x 17 testuser testuser 4096 Apr 15 20:38 ..
-rw-rw-r-- 1 testuser testuser 0 Apr 15 20:23 test1.txt
-rw-rw-r-- 1 testuser testuser 0 Apr 15 20:23 test2.txt
testuser@systemname:~/TestOrigin$ duplicity --no-encryption ~/TestOrigin/ file:///mnt/bkup_dir/
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1492304293.46 (Sat Apr 15 20:58:13 2017)
EndTime 1492304293.47 (Sat Apr 15 20:58:13 2017)
ElapsedTime 0.01 (0.01 seconds)
SourceFiles 3
SourceFileSize 4096 (4.00 KB)
NewFiles 3
NewFileSize 4096 (4.00 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 3
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 145 (145 bytes)
Errors 0
-------------------------------------------------

testuser@systemname:~/TestOrigin$ duplicity list-current-files file:///mnt/bkup_dir/
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Apr 15 20:58:13 2017
Sat Apr 15 20:23:51 2017 .
Sat Apr 15 20:23:43 2017 test1.txt
Sat Apr 15 20:23:43 2017 test2.txt
testuser@systemname:~/TestOrigin$ rm test2.txt
testuser@systemname:~/TestOrigin$ ls -al
total 8
drwxrwxr-x 2 testuser testuser 4096 Apr 15 21:02 .
drwxr-xr-x 17 testuser testuser 4096 Apr 15 21:01 ..
-rw-rw-r-- 1 testuser testuser 0 Apr 15 20:23 test1.txt
testuser@systemname:~/TestOrigin$ duplicity --no-encryption ~/TestOrigin/ file:///mnt/bkup_dir/
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sat Apr 15 20:58:13 2017
--------------[ Backup Statistics ]--------------
St...