duplicity jobs not working

[lstewart]

Binary package hint: backupninja

I get these error messages in the logs when there is already some data there:
Jul 15 01:24:20 Debug: duplicity --no-print-statistics --scp-command 'scp ' --ssh-command 'ssh ' --encrypt-key 83AF64BC --sig
n-key 83AF64BC --remove-older-than 60D --include '/home/lstewart/bin/**' --exclude '**' / scp://<email address hidden>/
/home/lstewart/backup/nuzetta
Jul 15 01:24:50 Debug: Traceback (most recent call last): File "/usr/bin/duplicity", line 373, in <module> if __name__ == "__m
ain__": main() File "/usr/bin/duplicity", line 362, in main check_last_manifest(col_stats) File "/usr/bin/duplicity", line 342
, in check_last_manifest last_backup_set.check_manifests() File "/usr/lib/python2.5/site-packages/duplicity/collections.py", l
ine 128, in check_manifests if local_manifest: remote_manifest = local_manifest UnboundLocalError: local variable 'local_manif
est' referenced before assignment

And this one when we are looking at the initial backup:
Jul 15 01:26:59 Debug: duplicity --no-print-statistics --scp-command 'scp ' --ssh-command 'ssh ' --encrypt-key 83AF64BC --sign-key 83AF64BC --remove-older-than 60D --include '/home/lstewart/bin/**' --exclude '**' / scp://<email address hidden>//home/lstewart/backup/nuzetta
Jul 15 01:27:12 Debug: No signatures found, switching to full backup. Traceback (most recent call last): File "/usr/bin/duplicity", line 373, in <module> if __name__ == "__main__": main() File "/usr/bin/duplicity", line 366, in main if not sig_chain: full_backup(col_stats) File "/usr/bin/duplicity", line 142, in full_backup bytes_written = write_multivol("full", tarblock_iter, globals.backend) File "/usr/bin/duplicity", line 78, in write_multivol globals.gpg_profile) File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 213, in GPGWriteFile try: data = block_iter.next(bytes_to_go).data File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 407, in next result = self.process(self.input_iter.next(), size) File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 277, in get_delta_iter_w_sig (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.5/site-packages/duplicity/robust.py", line 31, in check_common_error try: return function(*args) File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 309, in get_delta_path_w_sig if not new_path.isreg(): sigTarFile.addfile(ti) File "/usr/lib/python2.5/site-packages/duplicity/tarfile.py", line 762, in addfile self.fileobj.write(full_headers) File "/usr/lib/python2.5/site-packages/duplicity/dup_temp.py", line 120, in write return self.fileobj.write(buf) File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 103, in write def write(self, buf): return self.gpg_input.write(buf) IOError: [Errno 32] Broken pipe close failed: [Errno 32] Broken pipe

I know what you're gonna say: its a duplicity problem. The problem with that is that duplicity works with the same arguments from the command line, and in a simple scheduled cron job (what I've fallen back to). Everyone knows duplicity is a little bit unstable, but I think that this is really some issue with the environment the duplicity command is being executed in or something.

[Marco Rodrigues (gothicx)]

Do you still have this problem ? You're using Gutsy ?

[lstewart (agrodellic)]

Nope. Now I get what you see below. I abandoned backup ninja a while back and went with my shell scripts, and then just last weekend I tried backupninja again, but had little luck.

Yep I'm using Gutsy... You want me to file a different report for this one?

Also I had gpg dump a list of available keys ( gpg --list-secret-keys ), and the required key shows up. Also, as with the previous problem, the duplicity command that backupninja spits out works fine from the command line, as-is.

Cheers!

Dec 12 23:14:37 Info: >>>> starting action /etc/backup.d/action.dup (because of --now)
Dec 12 23:14:37 Debug: yes
Dec 12 23:14:37 Debug: ssh -o IdentityFile=/home/lstewart/.ssh/id_dsa -o PasswordAuthentication=no mail.sevaserver.org -l lstewart 'echo -n 1'
Dec 12 23:14:38 Debug: Connected to mail.sevaserver.org as lstewart successfully
Dec 12 23:14:39 Debug: Data will be encrypted with the GnuPG key 83AF64BC.
Dec 12 23:14:39 Debug: Data won't be signed.
Dec 12 23:14:39 Debug: duplicity -v9 --allow-source-mismatch --no-print-statistics --scp-command 'scp -o IdentityFile=/home/lstewart/.ssh/id_dsa' --sftp-command 'sftp -o IdentityFile=/home/lstewart/.ssh/id_dsa' --ssh-options '-o IdentityFile=/home/lstewart/.ssh/id_dsa' --encrypt-key 83AF64BC --remove-older-than 60D --exclude '/' --include '/home/lstewart/bin/**' --exclude '**' / scp://<email address hidden>//home/lstewart/backup/nuzetta
Dec 12 23:14:42 Debug: Main action: inc Running 'sftp -o IdentityFile=/home/lstewart/.ssh/id_dsa -o IdentityFile=/home/lstewart/.ssh/id_dsa <email address hidden>' (attempt #1) Warning, found incomplete backup sets, probably left from aborted session Collection Status ----------------- Connecting with backend: sshBackend Archive dir: None No backup chains with active signatures found Found 0 backup chains without signatures. Also found 0 backup sets not part of any chain, and 3 incomplete backup sets. These may be deleted by running duplicity with the --cleanup option. No signatures found, switching to full backup. Selecting / Comparing () and None Getting delta of (() / dir) and None Generating delta - new file: . gpg: 83AF64BC: skipped: public key not found gpg: [stdin]: encryption failed: public key not found gpg: 83AF64BC: skipped: public key not found gpg: [stdin]: encryption failed: public key not found Traceback (most recent call last): File "/usr/bin/duplicity", line 394, in <module> if __name__ == "__main__": main() File "/usr/bin/duplicity", line 387, in main if not sig_chain: full_backup(col_stats) File "/usr/bin/duplicity", line 146, in full_backup bytes_written = write_multivol("full", tarblock_iter, globals.backend) File "/usr/bin/duplicity", line 82, in write_multivol globals.gpg_profile,globals.volsize) File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 208, in GPGWriteFile file.close() File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 123, in close self.gpg_process.wait() File "/var/lib/python-support/python2.5/GnuPGInterface.py", line 639, in wait raise IOError, "GnuPG exited non-zero, with code %d" % (e << 8) IOError: GnuPG exited non-zero, with code 131072
Dec 12 23:14:42 Fatal: Duplicity failed.
Dec 12 23:14:42...

[lstewart (agrodellic)]

"Also I had gpg dump a list of available keys ( gpg --list-secret-keys ), and the required key shows up. Also, as with the previous problem, the duplicity command that backupninja spits out works fine from the command line, as-is."

From within the dupl action, dupl.in, or w/e.

[Buttay (cyril-buttay)]

I also have a problem running duplicity via backupninja:

on its own, duplicity works:
$ export PASSPHRASE=my_passphrase
$ duplicity --full --no-print-statistics --encrypt-key XXXXXXX --sign-key XXXXXXXX --remove-older-than 60D --exclude '/home/*/.gnupg' --exclude '/home/*/.gnupg' --exclude '/home/*/.local/share/Trash' --exclude '/home/*/.Trash' --exclude '/home/*/.thumbnails' --exclude '/home/*/.beagle' --exclude '/home/*/.aMule' --exclude '/home/*/gtk-gnutella-downloads' --exclude '/home/cyril/backups' --include '/var/spool/cron/crontabs' --include '/var/backups' --include '/etc' --include '/root' --include '/home' --include '/usr/local/*bin' --include '/var/lib/dpkg/status*' --exclude '**' / scp://login@host//directory

(the duplicity command is a copy of the one generated by the backupninja script)

however, running backupninja fails (line breaks added for readability):

backupninja --debug --run /etc/backup.d/90.dup

Debug: check_perms /etc/backup.d
Debug: perms: drwxrwx---
Debug: gperm: rwx
Debug: wperm: ---
Debug: check_perms /etc/backup.d/90.dup
Debug: perms: -rw-------
Debug: gperm: ---
Debug: wperm: ---
Info: >>>> starting action /etc/backup.d/90.dup (because of --now)
Debug: yes
Debug: ssh -o PasswordAuthentication=no host -l login 'echo -n 1'
Debug: Connected to host as login successfully
Debug: Data will be encrypted with the GnuPG key XXXXXXXX.
Debug: Data will be signed will the GnuPG key XXXXXXXX.

duplicity --full --no-print-statistics --encrypt-key XXXXXXX --sign-key XXXXXXXX --remove-older-than 60D --exclude '/home/*/.gnupg' --exclude '/home/*/.gnupg' --exclude '/home/*/.local/share/Trash' --exclude '/home/*/.Trash' --exclude '/home/*/.thumbnails' --exclude '/home/*/.beagle' --exclude '/home/*/.aMule' --exclude '/home/*/gtk-gnutella-downloads' --exclude '/home/cyril/backups' --include '/var/spool/cron/crontabs' --include '/var/backups' --include '/etc' --include '/root' --include '/home' --include '/usr/local/*bin' --include '/var/lib/dpkg/status*' --exclude '**' / scp://login@host//directory

Debug: Warning, found incomplete backup sets, probably left from aborted session No signatures found, switching to full backup. Traceback (most recent call last):
File "/usr/bin/duplicity", line 394, in <module> if __name__ == "__main__": main()
File "/usr/bin/duplicity", line 387, in main if not sig_chain: full_backup(col_stats)
File "/usr/bin/duplicity", line 146, in full_backup bytes_written = write_multivol("full", tarblock_iter, globals.backend)
File "/usr/bin/duplicity", line 82, in write_multivol globals.gpg_profile,globals.volsize)
File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 198, in GPGWriteFile try: data = block_iter.next(bytes_to_go).data
File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 407, in next result = self.process(self.input_iter.next(), size)
File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 277, in get_delta_iter_w_sig (new_path, sig_path, sigTarFile))
File "/usr/lib/python2.5/site-packages/duplicity/robust.py", line 31, in check_common_error try: return function(*args)
File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line 309, in get_de...

[Buttay (cyril-buttay)]

I just noticed that the --ssh-command issue I mentionned at the end of the message above is already fixed by the latest version of backupninja upstream:

http://code.autistici.org/trac/backupninja/browser/trunk/ChangeLog

regards

Cyril

[Mark Edgington (edgimar)]

See http://bugs.debian.org/458816 . Backupninja 0.9.5-3 fixes the problem, and is in the Debian repository. The Ubuntu repository just needs to get updated with the latest Debian package.

[Greg Grossmeier (greg.grossmeier)]

Marking as fix released as Jaunty has 0.9.6.4 (which is newer than the version which is said to fix this issue).

If you still have problems, please feel free to open a new bug report.

Thanks!