Backup failed with an unknown error. .... MemoryError

[Christopher Barrington-Leigh]

The backup application (GUI) failed a couple of hours into its first attempt to write to an external disk after I upgraded to 17.10. The failure is "Backup Failed: Failed with an unknown error" and the following details. Note that the machine has 20GB of RAM, and there is no sign of swap having been used.

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1546, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1540, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1391, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1416, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 710, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 836, in get_backup_chains
    add_to_sets(f)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 824, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 101, in add_filename
    self.set_manifest(filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 148, in set_manifest
    self.set_files_changed()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 128, in set_files_changed
    mf = self.get_manifest()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 250, in get_manifest
    return self.get_local_manifest()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 224, in get_local_manifest
    return manifest.Manifest().from_string(manifest_buffer)
  File "/usr/lib/python2.7/dist-packages/duplicity/manifest.py", line 214, in from_string
    for match in vi_iterator:
MemoryError

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: duplicity 0.7.12-1ubuntu1
ProcVersionSignature: Ubuntu 4.13.0-16.19-generic 4.13.4
Uname: Linux 4.13.0-16-generic x86_64
ApportVersion: 2.20.7-0ubuntu3.4
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Mon Nov 20 08:12:07 2017
EcryptfsInUse: Yes
InstallationDate: Installed on 2017-11-18 (2 days ago)
InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20171018)
SourcePackage: duplicity
UpgradeStatus: No upgrade log present (probably fresh install)

[Kenneth Loafman (kenneth-loafman)]

Please upgrade to the current version of duplicity. This will assure that any bugs fixed since your release are available and will fix your issue.

There are three options:

* Release tarball Install - https://launchpad.net/duplicity/+download
* Daily duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/daily
* Stable duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa

NOTE: UNinstall duplicity first if it was installed via the distribution repository. For Ubuntu, that would be "sudo apt-get purge duplicity".

[Christopher Barrington-Leigh (cpbl)]

@kenneth-loafman : Thanks; after adding the PPA my next backup worked.

So, Ubuntu, is this not a High priority bug for 17.10!?

[Kenneth Loafman (kenneth-loafman)]

Ummm, long discussion short, Ubuntu and I don't agree on releases. They release every six months and the version they pick up is always at least one behind mine. I release when things are ready, and up the bug version with each release. For some reason they refuse to work that way, so to get updates to the people I've had to build PPA's and get folks to upgrade.

[Christopher Barrington-Leigh (cpbl)]

Indeed, and you correctly marked the Duplicity bug invalid. But surely the Ubuntu(Duplicity) bug is not invalid!? Well, no one else has complained yet.
Thank you!

[Michael Terry (mterry)]

I've uploaded 0.7.15 to bionic (what will become 18.04).

[Launchpad Janitor (janitor)]

This bug was fixed in the package duplicity - 0.7.15-0ubuntu1

---------------
duplicity (0.7.15-0ubuntu1) bionic; urgency=medium

  * New upstream release
    - Fixes memory consumption issues (LP: #1733326)

 -- Michael Terry <email address hidden> Sat, 25 Nov 2017 17:27:26 -0500

[Matthias Klose (doko)]

> Ummm, long discussion short, Ubuntu and I don't agree
> on releases. They release every six months and the
> version they pick up is always at least one behind mine.

well, we can agree on releases based on Python3 ;-P Are there any?

[Kenneth Loafman (kenneth-loafman)]

Nope, not yet. We have one contributor working on it, but his time is
limited.

On Wed, Nov 29, 2017 at 3:35 AM, Matthias Klose <email address hidden> wrote:

> > Ummm, long discussion short, Ubuntu and I don't agree
> > on releases. They release every six months and the
> > version they pick up is always at least one behind mine.
>
> well, we can agree on releases based on Python3 ;-P Are there any?
>
> --
> You received this bug notification because you are subscribed to
> duplicity in Ubuntu.
> https://bugs.launchpad.net/bugs/1733326
>
> Title:
> Backup failed with an unknown error. .... MemoryError
>
> Status in Duplicity:
> Invalid
> Status in duplicity package in Ubuntu:
> Fix Released
>
> Bug description:
> The backup application (GUI) failed a couple of hours into its first
> attempt to write to an external disk after I upgraded to 17.10. The
> failure is "Backup Failed: Failed with an unknown error" and the
> following details. Note that the machine has 20GB of RAM, and there is
> no sign of swap having been used.
>
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1546, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1540, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1391, in main
> do_backup(action)
> File "/usr/bin/duplicity", line 1416, in do_backup
> globals.archive_dir).set_values()
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 710, in set_values
> self.get_backup_chains(partials + backend_filename_list)
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 836, in get_backup_chains
> add_to_sets(f)
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 824, in add_to_sets
> if set.add_filename(filename):
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 101, in add_filename
> self.set_manifest(filename)
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 148, in set_manifest
> self.set_files_changed()
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 128, in set_files_changed
> mf = self.get_manifest()
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 250, in get_manifest
> return self.get_local_manifest()
> File "/usr/lib/python2.7/dist-packages/duplicity/collections.py",
> line 224, in get_local_manifest
> return manifest.Manifest().from_string(manifest_buffer)
> File "/usr/lib/python2.7/dist-packages/duplicity/manifest.py", line
> 214, in from_string
> for match in vi_iterator:
> MemoryError
>
> ProblemType: Bug
> DistroRelease: Ubuntu 17.10
> Package: duplicity 0.7.12-1ubuntu1
> ProcVersionSignature: Ubuntu 4.13.0-16.19-generic 4.13.4
> Uname: Linux 4.13.0-16-generic x86_64
> ApportVersion: 2.20.7-0ubuntu3.4
> Architecture: amd64
> CurrentDesktop: ubuntu:GNOME
> Date: Mon Nov 20 08:12:07 2017
> EcryptfsInUse: Yes
> InstallationDate: Installed on 2017-11-18 (2 days ago)
> InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64
> (...