duplicity backup fails with OverflowError: size does not fit in an int

Bug #1612554 reported by Peter Van Rompaey
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Duplicity
Invalid
Medium
Unassigned
duplicity (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Ubuntu 14.04.4 LTS
duplicity 0.6.23-1ubuntu4.1
python2.7 2.7.6-8ubuntu0.2

Backup command used:
duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET"

Backups have been running fine for several months, but since last night it started giving following error:

---

Reading filelist /root/.duplicity_excludes

Sorting filelist /root/.duplicity_excludes

Local and Remote metadata are synchronized, no sync needed.

Last full backup date: Sun Aug 7 23:30:02 2016

Traceback (most recent call last):

  File "/usr/bin/duplicity", line 1494, in <module>

    with_tempdir(main)

  File "/usr/bin/duplicity", line 1488, in with_tempdir

    fn()

  File "/usr/bin/duplicity", line 1337, in main

    do_backup(action)

  File "/usr/bin/duplicity", line 1470, in do_backup

    incremental_backup(sig_chain)

  File "/usr/bin/duplicity", line 637, in incremental_backup

    globals.backend)

  File "/usr/bin/duplicity", line 405, in write_multivol

    at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)

  File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile

    new_block = block_iter.next()

  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next

    result = self.process(self.input_iter.next())

  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter

    (new_path, sig_path, sigTarFile))

  File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error

    return function(*args)

  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path

    delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))

  File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__

    self.maker = _librsync.new_deltamaker(sig_string)

OverflowError: size does not fit in an int

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :
Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

This has been fixed for a couple of years. Please upgrade to a newer version using the duplicity PPA or through the tarball in the download area. The 0.6 series has been deprecated in the upstream. The 0.7 series is current.

Please be aware that you will need to 'apt-get remove duplicity' before upgrading to the 0.7 series.

Changed in duplicity (Ubuntu):
status: New → Incomplete
status: Incomplete → Invalid
Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Hi,

after performing these steps:

- apt-get remove duplicity
- add-apt-repository ppa:duplicity-team/ppa
- apt-get update
- apt-get install duplicity

I am now running:
$ duplicity --version
duplicity 0.7.09

And I still get the Overflow errors:

ERROR 30 OverflowError
. Traceback (most recent call last):
. File "/usr/bin/duplicity", line 1546, in <module>
. with_tempdir(main)
. File "/usr/bin/duplicity", line 1540, in with_tempdir
. fn()
. File "/usr/bin/duplicity", line 1391, in main
. do_backup(action)
. File "/usr/bin/duplicity", line 1522, in do_backup
. incremental_backup(sig_chain)
. File "/usr/bin/duplicity", line 671, in incremental_backup
. globals.backend)
. File "/usr/bin/duplicity", line 434, in write_multivol
. at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
. File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 401, in GzipWriteFile
. new_block = block_iter.next()
. File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 523, in next
. result = self.process(self.input_iter.next())
. File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 218, in get_delta_iter
. (new_path, sig_path, sigTarFile))
. File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 38, in check_common_error
. return function(*args)
. File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 139, in get_delta_path
. delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
. File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 154, in __init__
. self.maker = _librsync.new_deltamaker(sig_string)
. OverflowError: size does not fit in an int

Changed in duplicity (Ubuntu):
status: Invalid → New
Changed in duplicity:
importance: Undecided → Medium
milestone: none → 0.7.10
Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Some more testing:

I installed a virtualenv with python 2.7.12 and duplicity 0.7.09, but the problem still remains.

Then I ran the little test mentioned in https://bugs.python.org/issue27130 which shows that zlib has a problem zipping files > 4Gb:

When looking at the timestamps of the comments, it seems this issue was fixed after 2.7.12 came out, so it will probably be in the next release.

I am currently running a backup with all files >4Gb excluded to see if that completes OK.

---

$ python --version
Python 2.7.6

$ python -c "import zlib;zlib.crc32('a'*(1<<31 - 1))"

$ python -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: size does not fit in an int

---

$ . venv.2.7.12/bin/activate

(venv.2.7.12) $ python --version
Python 2.7.12

(venv.2.7.12) $ python -c "import zlib;zlib.crc32('a'*(1<<31 - 1))"

(venv.2.7.12) $ python -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: size does not fit in an int

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Backup completed successfully with all >4Gb files excluded.

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

Marking as invalid since this problem is not with duplicity itself.

Changed in duplicity:
importance: Medium → Undecided
milestone: 0.7.10 → none
status: New → Invalid
Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Cried victory too soon, I'm afraid.

I have:
- compiled python 2.7.12 with the zlib patch
- created a virtualenv with that version
- installed duplicity in the virtualenv, along with lockfile and boto

The zlib test now succeeds:
(duplicity-venv) root@eniac:~# python -c "import zlib;zlib.crc32('a'*(1<<31 - 1))"
(duplicity-venv) root@eniac:~# python -c "import zlib;zlib.crc32('a'*(1<<31))"
(duplicity-venv) root@eniac:~#

But duplicity still throws the OverFlowError during Incremental backup, and it wasn't anywhere near a large file. The modules mentioned in the Traceback are all in the duplicity dir, btw.

---

DEBUG 1
. Selection: result: None from function: Command-line exclude glob: /var/lib/elasticsearch

DEBUG 1
. Selection: + including file

DEBUG 1
. Selecting /var/log/lastlog

DEBUG 1
. Comparing var/log/lastlog and var/log/lastlog

DEBUG 1
. Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg)

DEBUG 1
. Releasing lockfile /root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb92/lockfile.lock

DEBUG 1
. Removing still remembered temporary file /tmp/duplicity-WNi8WI-tempdir/mktemp-yuJbZJ-3

DEBUG 1
. Removing still remembered temporary file /tmp/duplicity-WNi8WI-tempdir/mkstemp-hiRlXx-1

ERROR 30 OverflowError
. Traceback (most recent call last):
. File "/opt/duplicity-venv/bin/duplicity", line 1546, in <module>
. with_tempdir(main)
. File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir
. fn()
. File "/opt/duplicity-venv/bin/duplicity", line 1391, in main
. do_backup(action)
. File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup
. incremental_backup(sig_chain)
. File "/opt/duplicity-venv/bin/duplicity", line 671, in incremental_backup
. globals.backend)
. File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol
. at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py", line 401, in GzipWriteFile
. new_block = block_iter.next()
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 523, in next
. result = self.process(self.input_iter.next())
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 218, in get_delta_iter
. (new_path, sig_path, sigTarFile))
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
. return function(*args)
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 139, in get_delta_path
. delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py", line 154, in __init__
. self.maker = _librsync.new_deltamaker(sig_string)
. OverflowError: size does not fit in an int

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote : Re: [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Download full text (5.6 KiB)

What version of librsync?

On Fri, Aug 19, 2016 at 3:52 AM, Peter Van Rompaey <email address hidden> wrote:

> Cried victory too soon, I'm afraid.
>
> I have:
> - compiled python 2.7.12 with the zlib patch
> - created a virtualenv with that version
> - installed duplicity in the virtualenv, along with lockfile and boto
>
> The zlib test now succeeds:
> (duplicity-venv) root@eniac:~# python -c "import
> zlib;zlib.crc32('a'*(1<<31 - 1))"
> (duplicity-venv) root@eniac:~# python -c "import
> zlib;zlib.crc32('a'*(1<<31))"
> (duplicity-venv) root@eniac:~#
>
> But duplicity still throws the OverFlowError during Incremental backup,
> and it wasn't anywhere near a large file. The modules mentioned in the
> Traceback are all in the duplicity dir, btw.
>
> ---
>
> DEBUG 1
> . Selection: result: None from function: Command-line exclude glob:
> /var/lib/elasticsearch
>
> DEBUG 1
> . Selection: + including file
>
> DEBUG 1
> . Selecting /var/log/lastlog
>
> DEBUG 1
> . Comparing var/log/lastlog and var/log/lastlog
>
> DEBUG 1
> . Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg)
>
> DEBUG 1
> . Releasing lockfile /root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb
> 92/lockfile.lock
>
> DEBUG 1
> . Removing still remembered temporary file /tmp/duplicity-WNi8WI-tempdir/
> mktemp-yuJbZJ-3
>
> DEBUG 1
> . Removing still remembered temporary file /tmp/duplicity-WNi8WI-tempdir/
> mkstemp-hiRlXx-1
>
> ERROR 30 OverflowError
> . Traceback (most recent call last):
> . File "/opt/duplicity-venv/bin/duplicity", line 1546, in <module>
> . with_tempdir(main)
> . File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir
> . fn()
> . File "/opt/duplicity-venv/bin/duplicity", line 1391, in main
> . do_backup(action)
> . File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup
> . incremental_backup(sig_chain)
> . File "/opt/duplicity-venv/bin/duplicity", line 671, in
> incremental_backup
> . globals.backend)
> . File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol
> . at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py",
> line 401, in GzipWriteFile
> . new_block = block_iter.next()
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 523, in next
> . result = self.process(self.input_iter.next())
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 218, in get_delta_iter
> . (new_path, sig_path, sigTarFile))
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py",
> line 38, in check_common_error
> . return function(*args)
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 139, in get_delta_path
> . delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
> . File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py",
> line 154, in __init__
> . self.maker = _librsync.new_deltamaker(sig_string)
> . OverflowError: size does not fit in an int
>
> --
> You received this bug notification because you are subs...

Read more...

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

I didn't install any specific version, since it was included in the duplicity tarball:

$ tar tf duplicity-0.7.09.tar.gz | grep -i rsync
duplicity-0.7.09/duplicity/_librsyncmodule.c
duplicity-0.7.09/duplicity/librsync.py
duplicity-0.7.09/duplicity/backends/rsyncbackend.py

$ pip list
boto (2.42.0)
duplicity (0.7.9)
lockfile (0.12.2)
pip (8.1.2)
setuptools (25.2.0)
wheel (0.29.0)

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

do this:
$ apt-cache show librsync1

That will display the version.

On Mon, Aug 22, 2016 at 3:16 AM, Peter Van Rompaey <email address hidden> wrote:

> I didn't install any specific version, since it was included in the
> duplicity tarball:
>
> $ tar tf duplicity-0.7.09.tar.gz | grep -i rsync
> duplicity-0.7.09/duplicity/_librsyncmodule.c
> duplicity-0.7.09/duplicity/librsync.py
> duplicity-0.7.09/duplicity/backends/rsyncbackend.py
>
> $ pip list
> boto (2.42.0)
> duplicity (0.7.9)
> lockfile (0.12.2)
> pip (8.1.2)
> setuptools (25.2.0)
> wheel (0.29.0)
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1612554
>
> Title:
> duplicity backup fails with OverflowError: size does not fit in an int
>
> Status in Duplicity:
> Invalid
> Status in duplicity package in Ubuntu:
> New
>
> Bug description:
> Ubuntu 14.04.4 LTS
> duplicity 0.6.23-1ubuntu4.1
> python2.7 2.7.6-8ubuntu0.2
>
> Backup command used:
> duplicity --exclude-filelist "/root/.duplicity_excludes"
> --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption
> --volsize 100 / "$S3BUCKET"
>
> Backups have been running fine for several months, but since last
> night it started giving following error:
>
> ---
>
> Reading filelist /root/.duplicity_excludes
>
> Sorting filelist /root/.duplicity_excludes
>
> Local and Remote metadata are synchronized, no sync needed.
>
> Last full backup date: Sun Aug 7 23:30:02 2016
>
> Traceback (most recent call last):
>
> File "/usr/bin/duplicity", line 1494, in <module>
>
> with_tempdir(main)
>
> File "/usr/bin/duplicity", line 1488, in with_tempdir
>
> fn()
>
> File "/usr/bin/duplicity", line 1337, in main
>
> do_backup(action)
>
> File "/usr/bin/duplicity", line 1470, in do_backup
>
> incremental_backup(sig_chain)
>
> File "/usr/bin/duplicity", line 637, in incremental_backup
>
> globals.backend)
>
> File "/usr/bin/duplicity", line 405, in write_multivol
>
> at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name,
> globals.volsize)
>
> File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383,
> in GzipWriteFile
>
> new_block = block_iter.next()
>
> File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
> 518, in next
>
> result = self.process(self.input_iter.next())
>
> File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
> 213, in get_delta_iter
>
> (new_path, sig_path, sigTarFile))
>
> File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line
> 37, in check_common_error
>
> return function(*args)
>
> File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
> 134, in get_delta_path
>
> delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
>
> File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line
> 150, in __init__
>
> self.maker = _librsync.new_deltamaker(sig_string)
>
> OverflowError: size does not fit in an int
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions
>

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Package: librsync1
Version: 0.9.7-10

Changed in duplicity:
status: Invalid → In Progress
assignee: nobody → Kenneth Loafman (kenneth-loafman)
importance: Undecided → Medium
Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

It's possible that the last incremental was corrupted. Try this:

- duplicity cleanup --force <target>
- remove the last incremental backup, easiest is by date in the filename
- duplicity inc -v9 <source> <target>

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

# duplicity cleanup --force --no-encryption $BUCKET
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Aug 17 08:38:55 2016
No extraneous files found, nothing deleted in cleanup.

Collection-status gives me:

Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed Aug 17 08:38:55 2016
Chain end time: Wed Aug 17 08:38:55 2016
Number of contained backup sets: 1
Total number of contained volumes: 1665
 Type of backup set: Time: Num volumes:
                Full Wed Aug 17 08:38:55 2016 1665
-------------------------
No orphaned or incomplete backup sets found.

s3cmd shows that there's no files with a timestamp later than duplicity-full.20160817T083855Z

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Re-ran the incremental backup, but it fails with the same error while adding /var/log/lastlog, which is a sparse file with a reported size of 36Gb, but actually only 56k on disk. Can this be the cause?

---

(duplicity-venv) # ls -l /var/log/lastlog
-rw-rw-r-- 1 root utmp 367219431264 Aug 23 09:57 /var/log/lastlog

# du -k /var/log/lastlog
56 /var/log/lastlog

---

Selecting /var/log/lastlog
Comparing var/log/lastlog and var/log/lastlog
Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg)
Releasing lockfile /root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb92/lockfile.lock
Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/mktemp-MvQFNN-128
Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/mkstemp-Lc2BZc-1
Traceback (most recent call last):
  File "/opt/duplicity-venv/bin/duplicity", line 1546, in <module>
    with_tempdir(main)
  File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir
    fn()
  File "/opt/duplicity-venv/bin/duplicity", line 1391, in main
    do_backup(action)
  File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup
    incremental_backup(sig_chain)
  File "/opt/duplicity-venv/bin/duplicity", line 671, in incremental_backup
    globals.backend)
  File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol
    at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py", line 401, in GzipWriteFile
    new_block = block_iter.next()
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 523, in next
    result = self.process(self.input_iter.next())
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 218, in get_delta_iter
    (new_path, sig_path, sigTarFile))
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 139, in get_delta_path
    delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
  File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py", line 154, in __init__
    self.maker = _librsync.new_deltamaker(sig_string)
OverflowError: size does not fit in an int

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :
Download full text (5.1 KiB)

Yep, exclude that and it should work.

On Tue, Aug 23, 2016 at 6:33 AM, Peter Van Rompaey <email address hidden> wrote:

> Re-ran the incremental backup, but it fails with the same error while
> adding /var/log/lastlog, which is a sparse file with a reported size of
> 36Gb, but actually only 56k on disk. Can this be the cause?
>
> ---
>
> (duplicity-venv) # ls -l /var/log/lastlog
> -rw-rw-r-- 1 root utmp 367219431264 Aug 23 09:57 /var/log/lastlog
>
> # du -k /var/log/lastlog
> 56 /var/log/lastlog
>
> ---
>
> Selecting /var/log/lastlog
> Comparing var/log/lastlog and var/log/lastlog
> Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg)
> Releasing lockfile /root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb
> 92/lockfile.lock
> Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/
> mktemp-MvQFNN-128
> Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/
> mkstemp-Lc2BZc-1
> Traceback (most recent call last):
> File "/opt/duplicity-venv/bin/duplicity", line 1546, in <module>
> with_tempdir(main)
> File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir
> fn()
> File "/opt/duplicity-venv/bin/duplicity", line 1391, in main
> do_backup(action)
> File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup
> incremental_backup(sig_chain)
> File "/opt/duplicity-venv/bin/duplicity", line 671, in
> incremental_backup
> globals.backend)
> File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol
> at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py",
> line 401, in GzipWriteFile
> new_block = block_iter.next()
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 523, in next
> result = self.process(self.input_iter.next())
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 218, in get_delta_iter
> (new_path, sig_path, sigTarFile))
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py",
> line 38, in check_common_error
> return function(*args)
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py",
> line 139, in get_delta_path
> delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
> File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py",
> line 154, in __init__
> self.maker = _librsync.new_deltamaker(sig_string)
> OverflowError: size does not fit in an int
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1612554
>
> Title:
> duplicity backup fails with OverflowError: size does not fit in an int
>
> Status in Duplicity:
> In Progress
> Status in duplicity package in Ubuntu:
> New
>
> Bug description:
> Ubuntu 14.04.4 LTS
> duplicity 0.6.23-1ubuntu4.1
> python2.7 2.7.6-8ubuntu0.2
>
> Backup command used:
> duplicity --exclude-filelist "/root/.duplicity_excludes"
> --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption
> --volsize 100 / "$S3BUCKET"
>
> Backups have been running fi...

Read more...

Revision history for this message
Peter Van Rompaey (blacky-z) wrote :

Incr backup ran successfully with the sparse file excluded.

---

NOTICE 1
. --------------[ Backup Statistics ]--------------
. StartTime 1471995038.09 (Tue Aug 23 23:30:38 2016)
. EndTime 1472019526.98 (Wed Aug 24 06:18:46 2016)
. ElapsedTime 24488.90 (6 hours 48 minutes 8.90 seconds)
. SourceFiles 307014
. SourceFileSize 934910254490 (871 GB)
. NewFiles 35356
. NewFileSize 493377240055 (459 GB)
. DeletedFiles 34498
. ChangedFiles 3956
. ChangedFileSize 1536875100 (1.43 GB)
. ChangedDeltaSize 0 (0 bytes)
. DeltaEntries 73810
. RawDeltaSize 494391455212 (460 GB)
. TotalDestinationSizeChange 355318722017 (331 GB)
. Errors 0
. -------------------------------------------------
.

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

Marking invalid because duplicity does not support sparse files, yet.

Changed in duplicity:
assignee: Kenneth Loafman (kenneth-loafman) → nobody
status: In Progress → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in duplicity (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.