BackendException: sftp put of [] failed: [Errno 2]

[Francis West]

Hello, I have a strange bug with v0.6.18

I get the following error:

Local and Remote metadata are synchronized, no sync needed.
Warning, found incomplete backup sets, probably left from aborted session
Last full backup left a partial set, restarting.
Last full backup date: Tue Mar 6 11:42:05 2012
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
RESTART: The first volume failed to upload before termination.
         Restart is impossible...starting backup from beginning.
Reading filelist /root/bin/filelist.txt
Sorting filelist /root/bin/filelist.txt
Local and Remote metadata are synchronized, no sync needed.
Warning, found incomplete backup sets, probably left from aborted session
Last full backup date: none
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
No signatures found, switching to full backup.
BackendException: sftp put of /var/tmp/duplicity-Y5cji0-tempdir/mktemp-0Cehis-2 (as duplicity-full.20120306T115112Z.vol1.difftar.gpg) failed: [Errno 2]

yes indeed the file it is trying to access does not exist,

it doesn't appear to affect all my systems i've tested duplicity 0.6.18 on, however the systems are lucid with the latest updates installed.

* lots of comparing files *

Comparing () and None
Getting delta of (() /opt/zimbra/aspell-0.60.6/lib/aspell-0.60/fr-60-only.rws reg) and None
A file.abc
Removing still remembered temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-YUYbEk-tempdir/mktemp-WapA7f-1
Cleanup of temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-YUYbEk-tempdir/mktemp-WapA7f-1 failed
Removing still remembered temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-aPa8gV-tempdir/mktemp-MeYX9j-1
Cleanup of temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-aPa8gV-tempdir/mktemp-MeYX9j-1 failed
AsyncScheduler: running task synchronously (asynchronicity disabled)
Removing still remembered temporary file /tmp/duplicity-Pa11IH-tempdir/mkstemp-tl22iM-1
Removing still remembered temporary file /tmp/duplicity-Pa11IH-tempdir/mktemp-R58OQe-2
Backend error detail: Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1391, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1384, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1359, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 500, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 399, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.6/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 398, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 296, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.6/dist-packages/duplicity/backends/sshbackend.py", line 191, in put
    raise BackendException("sftp put of %s (as %s) failed: %s" % (source_path.name,remote_filename,e))
BackendException: sftp put of /tmp/duplicity-VOemDY-tempdir/mktemp-VGFNvm-2 (as duplicity-full.20120306T120940Z.vol1.difftar.gpg) failed: [Errno 2]

BackendException: sftp put of /tmp/duplicity-VOemDY-tempdir/mktemp-VGFNvm-2 (as duplicity-full.20120306T120940Z.vol1.difftar.gpg) failed: [Errno 2]

yes i agree you are going to have an error there because that file/dir doesn't exist

the first few lines are:

Using archive dir: /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15
Using backup name: 763e12cfbcae8ceca538e33fee48bf15
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.u1backend Succeeded
Reading filelist /root/bin/filelist.txt
Sorting filelist /root/bin/filelist.txt
Main action: inc
================================================================================
duplicity 0.6.18 ($reldate)
Args: /usr/bin/duplicity -v 9 --encrypt-key A1C86AEB --sign-key A1C86AEB --exclude-filelist /root/bin/filelist.txt / scp://user_stuff_snip@remote_host/remotedir
Linux zimbra 2.6.32-37-server #81-Ubuntu SMP Fri Dec 2 20:49:12 UTC 2011 x86_64
/usr/bin/python2.6 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3]
================================================================================
================================================================================
Using temporary directory /tmp/duplicity-Pa11IH-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-Pa11IH-tempdir/mkstemp-tl22iM-1
Temp has 84592353280 available, backup will use approx 34078720.
Local and Remote metadata are synchronized, no sync needed.
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: SftpBackend
Archive dir: /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
Reuse configured PASSPHRASE as SIGN_PASSPHRASE
No signatures found, switching to full backup.
Using temporary directory /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-YUYbEk-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-YUYbEk-tempdir/mktemp-WapA7f-1
Using temporary directory /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-aPa8gV-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/763e12cfbcae8ceca538e33fee48bf15/duplicity-aPa8gV-tempdir/mktemp-MeYX9j-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-Pa11IH-tempdir/mktemp-R58OQe-2
Selecting /
Comparing () and None
Getting delta of (() / dir) and None
A .
Selecting /cdrom
Comparing ('cdrom',) and None
Getting delta of (('cdrom',) /cdrom dir) and None
A cdrom
Selecting /etc
Comparing ('etc',) and None
....

[Eugene Crosser (crosser)]

I get this error too. It looks to me as a bug in paramiko. Apparently it does successfully 'put' the file to the server but thinks that the operation fails.

Francis, can you check which versions of python-paramiko are installed on your hosts that work and that fail with this error?

[Jacob Henner (jacobhenner)]

After fixing https://bugs.launchpad.net/duplicity/+bug/965017 by updating to duplicity 0.6.19, this error occurs. (But only on Ubuntu, not on an Arch Linux Machine)

[Jacob Henner (jacobhenner)]

Strike that last comment, while the first bug does affect me, this error was due to a server-end problem

[Marco Patzer (mpfusion)]

I have a similar issue while running:

    duplicity remove-all-but-n-full 1 --force scp://<email address hidden>/backup

The output:

Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Apr 10 01:24:12 2013
Deleting backup chain at time:
Wed Apr 10 01:23:28 2013
Deleting complete signature chain Wed Apr 10 01:23:28 2013
Deleting complete signature chain Wed Apr 10 01:23:28 2013
sftp rm duplicity-new-signatures.20130409T232252Z.to.20130409T232328Z.sigtar.gpg failed: [Errno 2] No such file (Try 2 of 5) Will retry in 10 seconds.
sftp rm duplicity-new-signatures.20130409T232252Z.to.20130409T232328Z.sigtar.gpg failed: [Errno 2] No such file (Try 3 of 5) Will retry in 10 seconds.
sftp rm duplicity-new-signatures.20130409T232252Z.to.20130409T232328Z.sigtar.gpg failed: [Errno 2] No such file (Try 4 of 5) Will retry in 10 seconds.
sftp rm duplicity-new-signatures.20130409T232252Z.to.20130409T232328Z.sigtar.gpg failed: [Errno 2] No such file

Let me know if I can provide any more information.

    duplicity 0.6.20