Comment 5 for bug 1674833

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote : Re: [Bug 1674833] Re: error uploading manifest file on incremental backup

Try removing the .part file. That should solve the problem.

...Ken

On Wed, Mar 22, 2017 at 2:22 PM, ljesmin <email address hidden> wrote:

> Upgraded it to 0.7.12, but it did not fix error, last backup is done with
> version 0.7.12 (.manifest.part is left behind):
>
> drwxr-xr-x. 2 root root 4096 22. märts 02:25 .
> drwxr-xr-x. 5 root root 4096 20. märts 23:59 ..
> -rw-r--r--. 1 root root 23925657 8. märts 21:20 duplicity-full.
> 20170307T173156Z.manifest
> -rw-r--r--. 1 root root 24376459 8. märts 21:20 duplicity-full.
> 20170308T161708Z.manifest
> -rw-------. 1 root root 24376160 12. märts 03:51 duplicity-full.
> 20170312T001508Z.manifest
> -rw-------. 1 root root 24557672 19. märts 03:51 duplicity-full.
> 20170319T001510Z.manifest
> -rw-r--r--. 1 root root 375650731 8. märts 21:19
> duplicity-full-signatures.20170307T173156Z.sigtar.gz
> -rw-r--r--. 1 root root 383838030 8. märts 21:20
> duplicity-full-signatures.20170308T161708Z.sigtar.gz
> -rw-------. 1 root root 383887812 12. märts 03:51
> duplicity-full-signatures.20170312T001508Z.sigtar.gz
> -rw-------. 1 root root 384573437 19. märts 03:51
> duplicity-full-signatures.20170319T001510Z.sigtar.gz
> -rw-r--r--. 1 root root 1134 10. märts 00:11 duplicity-inc.
> 20170308T161708Z.to.20170309T215557Z.manifest
> -rw-------. 1 root root 171 10. märts 02:26 duplicity-inc.
> 20170309T215557Z.to.20170310T001528Z.manifest
> -rw-------. 1 root root 818 11. märts 02:26 duplicity-inc.
> 20170310T001528Z.to.20170311T001554Z.manifest
> -rw-------. 1 root root 5425019 13. märts 23:30 duplicity-inc.
> 20170312T001508Z.to.20170313T212005Z.manifest
> -rw-------. 1 root root 675 14. märts 02:24 duplicity-inc.
> 20170313T212005Z.to.20170314T001450Z.manifest
> -rw-------. 1 root root 24876 15. märts 02:28 duplicity-inc.
> 20170314T001450Z.to.20170315T001504Z.manifest
> -rw-------. 1 root root 44210 16. märts 02:24 duplicity-inc.
> 20170315T001504Z.to.20170316T001514Z.manifest
> -rw-------. 1 root root 32855 17. märts 02:24 duplicity-inc.
> 20170316T001514Z.to.20170317T001449Z.manifest
> -rw-------. 1 root root 4985 18. märts 02:25 duplicity-inc.
> 20170317T001449Z.to.20170318T001452Z.manifest
> -rw-------. 1 root root 961525 21. märts 23:58 duplicity-inc.
> 20170319T001510Z.to.20170321T214444Z.manifest
> -rw-------. 1 root root 245 22. märts 00:45 duplicity-inc.
> 20170321T214444Z.to.20170321T223228Z.manifest
> -rw-------. 1 root root 249 22. märts 02:25 duplicity-inc.
> 20170321T223228Z.to.20170322T001524Z.manifest.part
> -rw-r--r--. 1 root root 1630 10. märts 00:11 duplicity-new-signatures.
> 20170308T161708Z.to.20170309T215557Z.sigtar.gz
> -rw-------. 1 root root 45 10. märts 02:26 duplicity-new-signatures.
> 20170309T215557Z.to.20170310T001528Z.sigtar.gz
> -rw-------. 1 root root 55294 11. märts 02:26 duplicity-new-signatures.
> 20170310T001528Z.to.20170311T001554Z.sigtar.gz
> -rw-------. 1 root root 9107227 13. märts 23:30 duplicity-new-signatures.
> 20170312T001508Z.to.20170313T212005Z.sigtar.gz
> -rw-------. 1 root root 907 14. märts 02:24 duplicity-new-signatures.
> 20170313T212005Z.to.20170314T001450Z.sigtar.gz
> -rw-------. 1 root root 112011 15. märts 02:28 duplicity-new-signatures.
> 20170314T001450Z.to.20170315T001504Z.sigtar.gz
> -rw-------. 1 root root 301068 16. märts 02:24 duplicity-new-signatures.
> 20170315T001504Z.to.20170316T001514Z.sigtar.gz
> -rw-------. 1 root root 99114 17. märts 02:24 duplicity-new-signatures.
> 20170316T001514Z.to.20170317T001449Z.sigtar.gz
> -rw-------. 1 root root 88013 18. märts 02:25 duplicity-new-signatures.
> 20170317T001449Z.to.20170318T001452Z.sigtar.gz
> -rw-------. 1 root root 1410726 21. märts 23:58 duplicity-new-signatures.
> 20170319T001510Z.to.20170321T214444Z.sigtar.gz
> -rw-------. 1 root root 155 22. märts 00:45 duplicity-new-signatures.
> 20170321T214444Z.to.20170321T223228Z.sigtar.gz
> -rw-------. 1 root root 158 22. märts 02:25 duplicity-new-signatures.
> 20170321T223228Z.to.20170322T001524Z.sigtar.gz
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1674833
>
> Title:
> error uploading manifest file on incremental backup
>
> Status in Duplicity:
> New
>
> Bug description:
> Have a problem with duplicity incremental backups to S3.
>
> I do backup from about 30 G of small files, with exclusion of some files
> by
> pattern to S3, eu-central-1 bucket.
>
> I use duplicity 0.7.11 from EPEL on CentOS 7.3. Python is 2.7.5.
>
> Full backups work fine, but incremental backups immediately after full
> backup
> generate errors.
>
> Full backup command line:
>
> duplicity full --no-encryption --s3-use-multiprocessing
> --s3-multipart-chunk-size 10 --num-retries 5 --exclude "**typo3temp/**"
> --exclude "**_temp/**" --exclude "**_processed_/**" /var/www
> "s3://s3.eu-central-1.amazonaws.com/backup-bucket-name/www"
>
> Incremental backup command line:
>
> duplicity incremental --no-encryption --s3-use-multiprocessing
> --s3-multipart-chunk-size 10 --num-retries 5 --exclude "**typo3temp/**"
> --exclude "**_temp/**" --exclude "**_processed_/**" /var/www
> "s3://s3.eu-central-1.amazonaws.com/backup-bucket-name/www"
>
> Output on erratic incremental backup:
>
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sun Mar 19 02:15:10 2017
> Giving up after 1 attempts. OSError: [Errno 2] No such file or directory:
> '/root/.cache/duplicity/652b29dc986d2c5850695bd80bf5261a/duplicity-inc.
> 20170319T001510Z.to.20170321T001455Z.manifest'
>
> After backup, in /root/.cache/duplicity/652b29dc986d2c5850695bd80bf5261a/
> there
> are such files:
>
> duplicity-full.20170319T001510Z.manifest
> duplicity-new-signatures.20170319T001510Z.to.20170321T001455Z.sigtar.gz
> duplicity-inc.20170319T001510Z.to.20170321T001455Z.manifest.part
>
> In AWS bucket are from latest backup such files:
>
> www/duplicity-inc.20170319T001510Z.to.20170321T001455Z.vol1.difftar.gz
> www/duplicity-new-signatures.20170319T001510Z.to.
> 20170321T001455Z.sigtar.gz
>
> And manifest file is missing.
>
> Seems that for some reason, manifest is created as .part file and upload
> function does not use this file.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1674833/+subscriptions
>