--exclude-if-present gives OSError looking for tag in locked folders

[Aaron Whitehouse]

As per:
http://lists.nongnu.org/archive/html/duplicity-talk/2016-08/msg00009.html

"I want to send a backup with --exclude-if-present=TAG and
--include-filelist.
The folder /src/folder is not accessible (mode 750, not owner, not
member of group). This folder is excluded by way of '- **/folder' in
the include filelist. Does the selection process resolve
exclude-if-present=TAG before it looks at include/exclude filelists
anyway ?

duplicity 0.7.10 (python 2.7.12) fails with

OSError: [Errno 13] Permission denied: '/src/folder/TAG'

duplicity 0.7.06 prints

Error accessing possibly locked file /src/folder

but it isn't a showstopper."

I can confirm this is the case. The reason is that --exclude-if-present looks in each folder within the backup tree looking for the excluded tag, even if that folder is later excluded.

The difference between version 0.7.06 and 0.7.10 is likely caused by my rev 1224:
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/revision/1224
which fixed Bug #1089131 by removing the readability check when the directory was first scanned and instead moved it to when the file comes to be included by the Select function. The issue here is that the function that deals with --exclude-if-present checks for the tag by simply trying to access it, without checking if it has access, giving an OSError.

[Aaron Whitehouse (aaron-whitehouse)]

Update from Richard McGraw on mailing list:
"Since my update to duplicity 0.7.11, I haven't reproduced the problem."

Great, thanks for the update Richard and good to hear it is working for you.

[Launchpad Janitor (janitor)]

This bug was fixed in the package duplicity - 0.7.06-2ubuntu3

---------------
duplicity (0.7.06-2ubuntu3) zesty; urgency=medium

  * d/p/backport-fix-tags-in-locked-folders.patch:
    - Fix OSError looking for tag in locked folders with exclude-if-present
      (LP: #1620085)
  * d/p/backport-crash-on-restore.patch:
    - Guard a chown call to avoid a stacktrace (LP: #1642813)
  * d/p/backport-ignore-excluded-locked-files.patch:
    - Ignore excluded locked files rather than giving noisy errors
      (LP: #1089131)
  * d/p/backport-downgrade-error.patch:
    - Make locked file messages be warnings not errors, again. This avoids
      confusing deja-dup. (LP: #1605939)
  * d/p/backport-flush-after-write.patch:
    - Flush after writing a temp file, to keep disk in sync (LP: #1538333)
  * d/p/backport-gpg2.patch:
    - Improve support for gpg2 (fixing FTBFS in process)

 -- Michael Terry <email address hidden> Tue, 07 Mar 2017 18:48:01 -0500

[Jesse Hines (jrsh1)]

It seems this bug has resurfaced? I'm running duplicity 0.8.19 on Ubuntu 20.04.

If I have the file tree:

test
  - root
    - ...
  - other stuff

And run
duplicity --exclude **/root --exclude-if-present .nobakup test file:///.../backup

duplicity prints a non-fatal error:
Error accessing possibly locked file test/root

No error occurs if I don't include --exclude-if-present

[Michael Terry (mterry)]

Interesting. Jesse, that might be due to https://gitlab.com/duplicity/duplicity/-/merge_requests/31 which rejiggered how `--exclude-if-present` checks files in order to avoid a fatal error that occurred in some edge cases. That change was released in 0.8.17.

Maybe we would need a smarter fix to avoid both the non-fatal and fatal errors.

[Jesse Hines (jrsh1)]

I've tried both the snap distribution which is at 0.8.19 and the apt distribution which is at 0.8.12 and both print the same error.

Everything still works fine so its only a cosmetic issue though.