Incremental backup for unchanged files with updated mtime

[heinzi]

*edit*: Bug is invalid. Please disregard.

When I create a full backup with duplicity, then run `touch` on each file to update mtime (but don't change the file contents), and then run an incremental backup, the incremental backup is as large as the original one. Duplicity recognized all files as changed. Shouldn't deduplication prevent this?

The use case is a docker image I created which pulls data (for example emails over IMAP) and then backs the data up using duplicity to Amazon S3. The docker image is ephemeral, i.e. it is deleted and later re-run, re-fetches all the emails, and does an incremental backup (duplicity automatically downloads the metadata from S3 for this). The second run should be incremental, but it doesn't work because the fetched emails have newer timestamps.

Initial backup:

$ duplicity --verbosity notice --s3-use-ia --s3-use-multiprocessing --s3-use-server-side-encryption --file-prefix-manifest manifest- --file-prefix-archive archive- --file-prefix-signature signature- --asynchronous-upload --volsize=500 --par2-redundancy 10 --log-file /tmp/mylogfile /tmp/mycopy/ par2+s3://s3-eu-west-1.amazonaws.com/my.bucket/duplicity/emails
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1493217811.37 (Wed Apr 26 14:43:31 2017)
EndTime 1493219233.19 (Wed Apr 26 15:07:13 2017)
ElapsedTime 1421.82 (23 minutes 41.82 seconds)
SourceFiles 104757
SourceFileSize 9484368265 (8.83 GB)
NewFiles 104757
NewFileSize 9484368265 (8.83 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 104757
RawDeltaSize 9471297929 (8.82 GB)
TotalDestinationSizeChange 5017946103 (4.67 GB)
Errors 0
-------------------------------------------------

Directly running duplicity again yields to expected incremental behaviour:

$ duplicity --verbosity notice --s3-use-ia --s3-use-multiprocessing --s3-use-server-side-encryption --file-prefix-manifest manifest- --file-prefix-archive archive- --file-prefix-signature signature- --asynchronous-upload --volsize=500 --par2-redundancy 10 --log-file /tmp/mylogfile /tmp/mycopy/ par2+s3://s3-eu-west-1.amazonaws.com/my.bucket/duplicity/emails
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Apr 26 14:43:31 2017
--------------[ Backup Statistics ]--------------
StartTime 1493221370.01 (Wed Apr 26 15:42:50 2017)
EndTime 1493221397.21 (Wed Apr 26 15:43:17 2017)
ElapsedTime 27.20 (27.20 seconds)
SourceFiles 104757
SourceFileSize 9484368265 (8.83 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 0
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 110 (110 bytes)
Errors 0
-------------------------------------------------

But running duplicity after a "find . -exec touch {} \;" yields:

$ duplicity --verbosity notice --s3-use-ia --s3-use-multiprocessing --s3-use-server-side-encryption --file-prefix-manifest manifest- --file-prefix-archive archive- --file-prefix-signature signature- --asynchronous-upload --volsize=500 --par2-redundancy 10 --log-file /tmp/mylogfile /tmp/mycopy/ par2+s3://s3-eu-west-1.amazonaws.com/my.bucket/duplicity/emails
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Apr 26 14:43:31 2017
--------------[ Backup Statistics ]--------------
StartTime 1493221731.55 (Wed Apr 26 15:48:51 2017)
EndTime 1493222923.71 (Wed Apr 26 16:08:43 2017)
ElapsedTime 1192.15 (19 minutes 52.15 seconds)
SourceFiles 104757
SourceFileSize 9484368265 (8.83 GB)
NewFiles 340
NewFileSize 13070336 (12.5 MB)
DeletedFiles 0
ChangedFiles 104417
ChangedFileSize 9471297929 (8.82 GB)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 104757
RawDeltaSize 1443062 (1.38 MB)
TotalDestinationSizeChange 3365981 (3.21 MB)
Errors 0
-------------------------------------------------

[heinzi (heinzisoft)]

sorry, this is actually working fine, there's a different issue. Will investigate further and possibly file a different bug.