backup fails in final step with MemoryError in gzip module

[FrejSoya]

Simple backup to usb harddisk, ~100GB homedir to backup(=many small files). Memory usage goes up to ~60-70% before crash.
2GB ram (32bit machine)

Ubuntu jaunty.
deja-dup 10.1-0jaunty1
duplicity 0.6.00-0jaunty2
python2.6 2.6.2-0ubuntu1

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1002, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 995, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 974, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 415, in full_backup
    sig_outfp.to_remote()
  File "/usr/lib/python2.6/dist-packages/duplicity/dup_temp.py", line 179, in to_remote
    size = sys.maxint / 2)
  File "/usr/lib/python2.6/dist-packages/duplicity/gpg.py", line 317, in GzipWriteFile
    gzip_file.write(new_block.data)
  File "/usr/lib/python2.6/gzip.py", line 197, in write
    self.fileobj.write( self.compress.compress(data) )
MemoryError

[Michael Terry (mterry)]

FrejSoya, thanks for the bug! This looks like it may be an error in the command line backup tool used, duplicity.

Ken, does this look familiar to you?

[Kenneth Loafman (kenneth-loafman)]

No, this is new. One thing to try is to run with -v9 and find the file last mentioned before the traceback and try to gzip that one manually. It may fail and give a memory error, in which case you can exclude that file and continue the backup. If not, post the full -v9 log as an attachment, along with the command line, and I'll take a look at it.

[Michael Terry (mterry)]

FrejSoya, in deja-dup-speak, you can make the relevant logs by running Deja Dup in a console like this:

  DEJA_DUP_DEBUG=1 deja-dup > /tmp/deja-dup.log

And attaching /tmp/deja-dup.log to this bug. If it's just too enormous, run the following command on the file:

  head -n 200 /tmp/deja-dup.log > /tmp/deja-dup-head.log
  tail -n 200 /tmp/deja-dup.log > /tmp/deja-dup-tail.log

And attach /tmp/deja-dup-head.log and /tmp/deja-dup-tail.log to this bug.

Thanks!