error when doing 'full system' backup

[Eric G.]

duplicity --exclude /mnt --exclude /tmp --exclude /proc --exclude /sys --exclude /dev / scp://XXX@XXX//XXX/XXXX

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1205, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 294, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.4/site-packages/duplicity/gpg.py", line 279, in GPGWriteFile
    data = block_iter.next(min(block_size, bytes_to_go)).data
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 505, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 631, in process
    data, last_block = self.get_data_block(fp, size - 512)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 658, in get_data_block
    buf = fp.read(read_size)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 415, in read
    buf = self.infile.read(length)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 384, in read
    buf = self.infile.read(length)
IOError: [Errno 22] Invalid argument

I've seen a report saying it was /home/virtfs, I do not have a /home/virtfs.

Duplicity Version 0.6.08b
Python Version 2.4.3-27.el5
CentOS 5.4
Linux ext3 on both sides

[Eric G. (eric-baldfury)]

Selecting /var/named/chroot/etc/localtime
Comparing ('var', 'named', 'chroot', 'etc', 'localtime') and None
Getting delta of (('var', 'named', 'chroot', 'etc', 'localtime') /var/named/chroot/etc/localtime reg) and None
A var/named/chroot/etc/localtime
Selecting /var/named/chroot/etc/named.conf
Comparing ('var', 'named', 'chroot', 'etc', 'named.conf') and None
Getting delta of (('var', 'named', 'chroot', 'etc', 'named.conf') /var/named/chroot/etc/named.conf reg) and None
A var/named/chroot/etc/named.conf
Selecting /var/named/chroot/etc/named.conf.local
Comparing ('var', 'named', 'chroot', 'etc', 'named.conf.local') and None
Getting delta of (('var', 'named', 'chroot', 'etc', 'named.conf.local') /var/named/chroot/etc/named.conf.local reg) and None
A var/named/chroot/etc/named.conf.local
Selecting /var/named/chroot/etc/named.conf.options
Comparing ('var', 'named', 'chroot', 'etc', 'named.conf.options') and None
Getting delta of (('var', 'named', 'chroot', 'etc', 'named.conf.options') /var/named/chroot/etc/named.conf.options reg) and None
A var/named/chroot/etc/named.conf.options
Selecting /var/named/chroot/etc/rndc.key
Comparing ('var', 'named', 'chroot', 'etc', 'rndc.key') and None
Getting delta of (('var', 'named', 'chroot', 'etc', 'rndc.key') /var/named/chroot/etc/rndc.key reg) and None
A var/named/chroot/etc/rndc.key
Selecting /var/named/chroot/proc
Comparing ('var', 'named', 'chroot', 'proc') and None
Getting delta of (('var', 'named', 'chroot', 'proc') /var/named/chroot/proc dir) and None
A var/named/chroot/proc
Selecting /var/named/chroot/proc/1
Comparing ('var', 'named', 'chroot', 'proc', '1') and None
Getting delta of (('var', 'named', 'chroot', 'proc', '1') /var/named/chroot/proc/1 dir) and None
A var/named/chroot/proc/1
Selecting /var/named/chroot/proc/1/attr
Comparing ('var', 'named', 'chroot', 'proc', '1', 'attr') and None
Getting delta of (('var', 'named', 'chroot', 'proc', '1', 'attr') /var/named/chroot/proc/1/attr dir) and None
A var/named/chroot/proc/1/attr
Selecting /var/named/chroot/proc/1/attr/current
Comparing ('var', 'named', 'chroot', 'proc', '1', 'attr', 'current') and None
Getting delta of (('var', 'named', 'chroot', 'proc', '1', 'attr', 'current') /var/named/chroot/proc/1/attr/current reg) and None
A var/named/chroot/proc/1/attr/current
Removing still remembered temporary file /tmp/duplicity--j3F7L-tempdir/mkstemp-yeu79u-1
Removing still remembered temporary file /tmp/duplicity--j3F7L-tempdir/mktemp-xEXWMX-182
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1210, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 294, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.4/site-packages/duplicity/gpg.py", line 279, in GPGWriteFile
    data = block_iter.next(min(block_size, bytes_to_go)).data
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 505, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib/pyt...

[Eric G. (eric-baldfury)]

I have resolved this. It was the bind mount of proc to the chroot environment for BIND (named). I'm not sure if you wanted to make any adjustements to the product to handle this, but as far as my issue, I have been able to exclude it and it works fine.

[Guilherme Salgado (salgado)]

I've also been bitten by this, but duplicity should not crash like it did. It could skip the /proc partition (and warn the user) or just stop the backup and state why it can't continue.

[Guilherme Salgado (salgado)]

This seems to be the same issue reported on http://<email address hidden>/msg00580.html

But note that according to https://launchpad.net/duplicity/0.6-series/0.6.07, the issue above should've been fixed so there might be a different root cause that triggered it.

[Eric G. (eric-baldfury)]

I continue having to 'exclude' files even in 0.6.09 because duplicity doesn't handle them at all well and simply crashes out. I was hoping by this time it would be able to handle this error and simply put a warning in the logs that a file couldn't be backed up.

[Eric G. (eric-baldfury)]

To see this in 0.6.09 here is the output:

Using archive dir: /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76
Using backup name: dcdda6f917acf7fc4b2901dced258c76
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.botobackend Failed: invalid syntax (botobackend.py, line 190)
Import of duplicity.backends.ftpbackend Succeeded
Main action: inc
================================================================================
duplicity 0.6.09 (July 25, 2010)
Args: /usr/bin/duplicity -v9 --exclude-device-files --no-encryption /selinux file:///backups/temp
Linux oratgc1h1.pp.bcinfra.net 2.6.18-164.0.0.0.1.el5 #1 SMP Thu Sep 3 00:21:28 EDT 2009 x86_64 x86_64
/usr/bin/python 2.4.3 (#1, Jul 2 2009, 15:50:36)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)]
================================================================================
Using temporary directory /tmp/duplicity-b_tyaY-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-b_tyaY-tempdir/mkstemp-3AuUag-1
Temp has 6600269824 available, backup will use approx 34078720.
Local and Remote metadata are synchronized, no sync needed.
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
No signatures found, switching to full backup.
Using temporary directory /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76/duplicity-8cp0Ix-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76/duplicity-8cp0Ix-tempdir/mktemp-9tqRGJ-1
Using temporary directory /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76/duplicity-IIENjP-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/dcdda6f917acf7fc4b2901dced258c76/duplicity-IIENjP-tempdir/mktemp-qsMid4-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-b_tyaY-tempdir/mktemp-3Mxt1k-2
Selecting /selinux
Comparing () and None
Getting delta of (() /selinux dir) and None
A .
Selecting /selinux/access
Comparing ('access',) and None
Getting delta of (('access',) /selinux/access reg) and None
A access
Selecting /selinux/avc
Comparing ('avc',) and None
Getting delta of (('avc',) /selinux/avc dir) and None
A avc
Selecting /selinux/avc/cache_stats
Comparing ('avc', 'cache_stats') and None
Getting delta of (('avc', 'cache_stats') /selinux/avc/cache_stats reg) and None
A avc/cache_stats
Selecting /selinux/avc/cache_threshold
Comparing ('avc', 'cache_threshold') and None
Ge...