MemoryError, maybe a regression in 07.08?

[Wolfgang Rohdewald]

Using installed duplicity version 0.7.08, python 2.6.8, gpg 2.0.9 (Home: ~/.gnupg), awk 'GNU Awk 3.1.8', bash '3.2.51(1)-release (i586-suse-linux-gnu)'.

This always worked but failed for the first time last night. 07.08 is installed since July 4.

The failing code only exists since

* Fixed bug #1594780 with patches from B. Reitsma
  - Use re.finditer() to speed processing

so I think it might be a regression. But since I do not want to reproduce this at the customer's site, I wonder how to proceed. Can you propose some debugging code? I will have to revert this change because the customer wants a reliable backup. So maybe you can try the new method and if it
fails with a MemoryError print out debugging info and fall back to the old one? That would make it reliable and if the problem happens again hopefully tell you want went wrong.

There are 19 full manifests with about 47MB each and 108 incremental manifests.

Last full backup is too old, forcing full backup
...
 File "/usr/local/lib/python2.6/site-packages/duplicity/collections.py",
line 224, in get_local_manifest
    return manifest.Manifest().from_string(manifest_buffer)
  File "/usr/local/lib/python2.6/site-packages/duplicity/manifest.py", line
214, in from_string
    for match in vi_iterator:
MemoryError

[Kenneth Loafman (kenneth-loafman)]

I can't seem to reproduce this problem.

How much memory do you have available?

What is your command line?

How many manifest files do you have?

On Thu, Nov 17, 2016 at 4:52 AM, Wolfgang Rohdewald <email address hidden>
wrote:

> Public bug reported:
>
> Using installed duplicity version 0.7.08, python 2.6.8, gpg 2.0.9 (Home:
> ~/.gnupg), awk 'GNU Awk 3.1.8', bash '3.2.51(1)-release (i586-suse-
> linux-gnu)'.
>
> This always worked but failed for the first time last night. 07.08 is
> installed since July 4.
>
> The failing code only exists since
>
> * Fixed bug #1594780 with patches from B. Reitsma
> - Use re.finditer() to speed processing
>
> so I think it might be a regression. But since I do not want to reproduce
> this at the customer's site, I wonder how to proceed. Can you propose some
> debugging code? I will have to revert this change because the customer
> wants a reliable backup. So maybe you can try the new method and if it
> fails with a MemoryError print out debugging info and fall back to the old
> one? That would make it reliable and if the problem happens again hopefully
> tell you want went wrong.
>
> There are 19 full manifests with about 47MB each and 108 incremental
> manifests.
>
> Last full backup is too old, forcing full backup
> ...
> File "/usr/local/lib/python2.6/site-packages/duplicity/collections.py",
> line 224, in get_local_manifest
> return manifest.Manifest().from_string(manifest_buffer)
> File "/usr/local/lib/python2.6/site-packages/duplicity/manifest.py",
> line
> 214, in from_string
> for match in vi_iterator:
> MemoryError
>
> ** Affects: duplicity
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1642544
>
> Title:
> MemoryError, maybe a regression in 07.08?
>
> Status in Duplicity:
> New
>
> Bug description:
> Using installed duplicity version 0.7.08, python 2.6.8, gpg 2.0.9
> (Home: ~/.gnupg), awk 'GNU Awk 3.1.8', bash '3.2.51(1)-release (i586
> -suse-linux-gnu)'.
>
> This always worked but failed for the first time last night. 07.08 is
> installed since July 4.
>
> The failing code only exists since
>
> * Fixed bug #1594780 with patches from B. Reitsma
> - Use re.finditer() to speed processing
>
> so I think it might be a regression. But since I do not want to
> reproduce this at the customer's site, I wonder how to proceed. Can you
> propose some debugging code? I will have to revert this change because the
> customer wants a reliable backup. So maybe you can try the new method and
> if it
> fails with a MemoryError print out debugging info and fall back to the
> old one? That would make it reliable and if the problem happens again
> hopefully tell you want went wrong.
>
> There are 19 full manifests with about 47MB each and 108 incremental
> manifests.
>
> Last full backup is too old, forcing full backup
> ...
> File "/usr/local/lib/python2.6/site-packages/duplicity/collections.py",
> line 224, in get_local_manifest
> return manifest.Manifest().from_string(manifest_buffer)
> File "/usr/local/lib/python2.6/site-packa...

[Wolfgang Rohdewald (wolfgang-rohdewald)]

I cannot reproduce it either, sorry. The next run was successfull.

For the number of manifests please see me first comment. Or did I misunderstand your question?

Command line:

duply version 1.9.0

duply myprofile pre+bkp+verify+purge_status --force --compare-data

the duply profile:
GPG_KEY='disabled'
GPG_PW='_GPG_PASSWORD_'
MAX_AGE=3M
MAX_FULL_BACKUPS=5
MAX_FULLBKP_AGE=1W
SOURCE='/'
TARGET='file:///tmp/duplydisk1/duply'
VERBOSITY=4 VOLSIZE=500 DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE"
DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "

# ulimit -a
core file size (blocks, -c) 1
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 30816
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) 3360132
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 30816
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

[Kenneth Loafman (kenneth-loafman)]

Perhaps something else was using all the memory. The iterator makes things
a bunch faster and really does not use more memory, so if only one run
failed, I'm thinking that may be the problem.

On Sun, Nov 20, 2016 at 3:10 AM, Wolfgang Rohdewald <email address hidden>
wrote:

> I cannot reproduce it either, sorry. The next run was successfull.
>
> For the number of manifests please see me first comment. Or did I
> misunderstand your question?
>
> Command line:
>
> duply version 1.9.0
>
> duply myprofile pre+bkp+verify+purge_status --force --compare-data
>
> the duply profile:
> GPG_KEY='disabled'
> GPG_PW='_GPG_PASSWORD_'
> MAX_AGE=3M
> MAX_FULL_BACKUPS=5
> MAX_FULLBKP_AGE=1W
> SOURCE='/'
> TARGET='file:///tmp/duplydisk1/duply'
> VERBOSITY=4
> VOLSIZE=500
> DUPL_PARAMS="$DUPL_PARAMS
> --full-if-older-than $MAX_FULLBKP_AGE"
> DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "
>
> # ulimit -a
> core file size (blocks, -c) 1
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 30816
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) 3360132
> open files (-n) 1024
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 30816
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1642544
>
> Title:
> MemoryError, maybe a regression in 07.08?
>
> Status in Duplicity:
> New
>
> Bug description:
> Using installed duplicity version 0.7.08, python 2.6.8, gpg 2.0.9
> (Home: ~/.gnupg), awk 'GNU Awk 3.1.8', bash '3.2.51(1)-release (i586
> -suse-linux-gnu)'.
>
> This always worked but failed for the first time last night. 07.08 is
> installed since July 4.
>
> The failing code only exists since
>
> * Fixed bug #1594780 with patches from B. Reitsma
> - Use re.finditer() to speed processing
>
> so I think it might be a regression. But since I do not want to
> reproduce this at the customer's site, I wonder how to proceed. Can you
> propose some debugging code? I will have to revert this change because the
> customer wants a reliable backup. So maybe you can try the new method and
> if it
> fails with a MemoryError print out debugging info and fall back to the
> old one? That would make it reliable and if the problem happens again
> hopefully tell you want went wrong.
>
> There are 19 full manifests with about 47MB each and 108 incremental
> manifests.
>
> Last full backup is too old, forcing full backup
> ...
> File "/usr/local/lib/python2.6/site-packages/duplicity/collections.py",
> line 224, in get_local_manifest
> return manifest.Manifest().from_string(manifest_buffer)
> File "/usr/local/lib/pyt...

[Kenneth Loafman (kenneth-loafman)]

Marking this as invalid since no one can verify.

[Tim Waugh (twaugh)]

FWIW I see this a lot on Fedora with duplicity-0.7.14-2.fc26.x86_64, and there are other reports too:
https://bugzilla.redhat.com/show_bug.cgi?id=1491168
https://bugzilla.redhat.com/show_bug.cgi?id=1492288

[Olivier Crête (olivier-crete)]

I think this isn't a duplicate of #1720159, I can still see it wit 0.7.15 on Fedora.

I can reproduce this every time I try to do a backup with deja-dup

Here is the backtrace from inside deja-dup

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1559, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1545, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1394, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1526, in do_backup
    check_last_manifest(col_stats) # not needed for full backup
  File "/usr/bin/duplicity", line 1228, in check_last_manifest
    last_backup_set.check_manifests()
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 208, in check_manifests
    remote_manifest = self.get_remote_manifest()
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 249, in get_remote_manifest
    return manifest.Manifest().from_string(manifest_buffer)
  File "/usr/lib64/python2.7/site-packages/duplicity/manifest.py", line 201, in from_string
    for match in vi_iterator:
 MemoryError

[Olivier Crête (olivier-crete)]

Actually, forget last comment, re-marking as duplicate, the this exact stack trace is bug #1730451