MultiBackend: failed to delete a backup from file and s3

[Igor Gaisin]

Hi community,

I have a problem with my backups: it can create a backup, but doesn't delete old ones.

I use duplicity via duply:
duply --version
  duply version 2.2.2
  (http://duply.net)

  Using installed duplicity version 0.8.13, python 3.8.3 (/usr/bin/python3), gpg 2.2.20 (Home: /root/.gnupg), awk 'awk: warning: option -W is ignored', grep 'BusyBox v1.31.1 () multi-call binary.', bash '5.0.17(1)-release (x86_64-alpine-linux-musl)'.

The duply cleanup command results in:

TMPDIR='/tmp' PASSPHRASE='PASS' /usr/bin/python3 /usr/bin/duplicity 'cleanup' --name duply_hourly --encrypt-key THEKEY --sign-key THEKEY --verbosity '9' --gpg-options '--pinentry-mode loopback --trust-model always --compress-algo none' '--allow-source-mismatch' '--archive-dir' /var/backups/cache/ '--tempdir' /var/backups/tmp '--volsize' 1024 '--progress' --force --log-file /var/backups/cache/cleanup.log 'multi:///etc/duply/hourly/target.json?mode=mirror&onfail=continue'

While target.json is:

[
    {
 "description": "Strato HiDrive Backend",
 "url": "s3://s3.hidrive.strato.com/BUCKET/hourly",
 "env": [
     {
  "name" : "AWS_ACCESS_KEY_ID",
  "value" : "KEY_ID"
     },
     {
  "name" : "AWS_SECRET_ACCESS_KEY",
  "value" : "KEY"
     }
 ]
    },
    {
 "url": "file:///mnt/backup_dir/another-dir/hourly"
    }
]

Strato is an S3 compatible storage and at lease upload files works well. But when I try to cleanup it fails with the messages:

NFO 1
. MultiBackend: failed to delete b'duplicity-full.20200616T065519Z.vol158.difftar.gpg' from s3://s3.hidrive.strato.com/BUCKET/hourly

INFO 1
. MultiBackend: failed to delete b'duplicity-full.20200616T065519Z.vol158.difftar.gpg' from file:///mnt/backup_dir/another-dir/hourly

And I don't know how to get more info, why it can't delete files.

The same is for the purge (remove) command.

[Kenneth Loafman (kenneth-loafman)]

I'd like a log file, please. This is going big and I only need a portion. Please run with -v9, redirect to a log file, then run:
$ head -n 200 < log > log.head
$ tail -n 200 < log > log.tail
Attach, don't copy/paste, log.head and log.tail to this bug report.

[Hans-Erik Floryd (hans-erik-floryd)]

I am seeing a similar problem, where remove-all-but-n-full using the multi backend is not working, but works when using the individual backends directly. That is

duplicity remove-all-but-n-full 1 --force "multi:///etc/duplicity.json?mode=mirror&onfail=abort"

removes no files in any backend, but e.g.

duplicity remove-all-but-n-full 1 --force rsync://backupuser@backupserver://tank/backups/data

works fine.

Logs provided as requested.

[Launchpad Janitor (janitor)]

[Expired for Duplicity because there has been no activity for 60 days.]

[Florent B (florent-z)]

I also have this problem with Duplicity 0.8.16 & MultiBackend !

MultiBackend: failed to delete b'duplicity-full.20200820T151141Z.vol562.difftar.gpg' from b2://xxx:xxx@backup-xxx/
MultiBackend: failed to delete b'duplicity-full.20200820T151141Z.vol563.difftar.gpg' from boto3+s3://backup-xxx/

Backends are growing and growing in size !