Including a folder with an --include ending in "/" does not include folder contents. Including the folder with the same --include without the "/" does include contents.
As reported by Claus in Bug #1479545:
"I had a command line similar to
--include ~/very/important/stuff/here/needs/backup/ --exclude '**'
Now of course I checked the backups when I set this up and everything was fine.
Recently I noticed that backups are very fast (too fast for the amount of data) and found that all files are missing in the backup!
By changing behavior back and forth you probably messed up a lot of backups everywhere and many users are probably not aware that their files are currently not in the backup!
My suggestion on how to cleanup this mess:
a) Change back once again (It is better there are to many files in backup than to little).
b) Introduce an explicit switch e.g. "--include-dirs-only <your-dir>" for the old behavior
At least point out this old/new very unintuitive behavior in the man page please.
Sorry if this sounds a little harsh, I am not writing this to yell at people who are only trying to help others. But I am very concerned about users/admins having a data loss and then noticing their backups are useless."
This bug was introduced at rev 1112 of the 0.7-series, though note that include and exclude lines including any globbing patterns (including *) were pretty fundamentally broken prior to rev 1110 (Bug #932482) anyway, so the main people who will be seeing a regression will be those who do not use globs in the relevant directory include/exclude line.
This is a challenge.
In many ways duplicity is doing exactly what it is being asked to do. If a user wants a folder and everything in it, they could "--include folder/**". That said, in the example given, I completely see that a user doing "--include folder/" would expect duplicity to back up the folder and its contents, so that should ideally be the behaviour.
The slash matching folders only is useful, too, as somebody doing:
--include fo*/
would expect it to match "folder/" but not "foo.txt", so I'm not keen to reverse this change back out to how things were before (completely ignoring trailing slashes).
I therefore think that the correct solution to this would be for a matched include folder to include all of that folder's contents unless trumped by a higher-priority exclude. This is in essence how excludes already work for most cases, as the file traverser does not descend into excluded folders. It also seems consistent with Ken's comments in this email:
http://
The necessary consequence of that may be some of the more niche things, e.g.:
--include /var/log/**/
--exclude /var/log/**
including only the directory skeleton but no files would cease to work. On balance I think that is the lesser of two evils and I did mention in Bug #1479545 that I was not making the change to solve the directory skeleton use case. We can try to address these use cases from 0.8 onwards with specific flags (such as that suggested by Claus) or an equivalent of the "ignorecase:" prefix (so that it works in include-filelists).
I would appreciate any thoughts.