duplicity hangs on sftp (no encryption)

Bug #1422778 reported by Robert Buchholz
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Duplicity
Invalid
Undecided
Unassigned

Bug Description

We are using Duplicity on RHEL7 (EPEL) and are experiencing hanging/stalling situations. This happened when using GPG (pub/priv key), but we were able to narrow it down to a hang issue even in this scenario:

--------------------------------------------------------------------------------------------------------------------
[root@localhost ~]# /usr/bin/python /usr/bin/duplicity --no-encrypt /usr/sbin/ sftp://user:password@backupserver/fnord/
Local and Remote metadata are synchronized, no sync needed.
Warning, found incomplete backup sets, probably left from aborted session
Last full backup date: none
No signatures found, switching to full backup.

[ hangs ]
--------------------------------------------------------------------------------------------------------------------

Hitting CTRL-C does not terminate the process.

On the remote storage, this file is created:
--------------------------------------------------------------------------------------------------------------------
drwxr-xr-x 1 0 0 3 17. Feb 16:08 .
drwxr-xr-x 1 0 0 4 17. Feb 16:08 ..
-rw-r--r-- 1 0 0 0 17. Feb 16:08 duplicity-full.20150217T151048Z.vol1.difftar.gz
--------------------------------------------------------------------------------------------------------------------

Versions:
--------------------------------------------------------------------------------------------------------------------
[root@localhost]# python -V
Python 2.7.5
[root@localhost]# duplicity -V
duplicity 0.6.24
[root@localhost]# yum info python-paramiko
...
Version : 1.15.1
Release : 1.el7
--------------------------------------------------------------------------------------------------------------------

Verbose log:
--------------------------------------------------------------------------------------------------------------------
[root@localhost ~]# /usr/bin/python /usr/bin/duplicity --no-encrypt -v9 /usr/sbin/ sftp://user:password@backupserver/fnord/
Using archive dir: /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898
Using backup name: 2f780b2947feb9f1de97bc51f312d898
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.~par2wrapperbackend Succeeded
ssh: starting thread (client mode): 0x172dd90L
ssh: Connected (version 2.0, client mod_sftp/0.9.9)
ssh: kex algos:[u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1', u'rsa1024-sha1'] server key:[u'ssh-rsa', u'ssh-dss'] client encrypt:[u'aes256-ctr', u'aes192-ctr', u'aes128-ctr', u'aes256-cbc', u'aes192-cbc', u'aes128-cbc', u'blowfish-ctr', u'blowfish-cbc', u'cast128-cbc', u'arcfour256', u'arcfour128', u'3des-ctr', u'3des-cbc'] server encrypt:[u'aes256-ctr', u'aes192-ctr', u'aes128-ctr', u'aes256-cbc', u'aes192-cbc', u'aes128-cbc', u'blowfish-ctr', u'blowfish-cbc', u'cast128-cbc', u'arcfour256', u'arcfour128', u'3des-ctr', u'3des-cbc'] client mac:[u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1', u'hmac-sha1-96', u'hmac-md5', u'hmac-md5-96', u'hmac-ripemd160', <email address hidden>'] server mac:[u'hmac-sha2-256', u'hmac-sha2-512', u'hmac-sha1', u'hmac-sha1-96', u'hmac-md5', u'hmac-md5-96', u'hmac-ripemd160', <email address hidden>'] client compress:[u'none'] server compress:[u'none'] client lang:[u''] server lang:[u''] kex follows?False
ssh: Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
ssh: using kex diffie-hellman-group14-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
ssh: Switch to new keys ...
ssh: userauth is OK
ssh: Authentication (password) successful!
ssh: [chan 1] Max packet in: 32768 bytes
ssh: [chan 1] Max packet out: 0 bytes
ssh: Secsh channel 1 opened.
ssh: [chan 1] Sesch channel 1 request ok
ssh: [chan 1] Opened sftp connection (server version 3)
ssh: [chan 1] stat('fnord')
ssh: [chan 1] mkdir('fnord', 511)
ssh: [chan 1] stat('fnord')
ssh: [chan 1] normalize('fnord')
Main action: inc
================================================================================
duplicity 0.6.24 (May 09, 2014)
Args: /usr/bin/duplicity --no-encrypt -v9 /usr/sbin/ sftp://user:password@backupserver/fnord/
Linux localhost.sntl-publishing.com 3.10.0-123.13.2.el7.x86_64 #1 SMP Thu Dec 18 14:09:13 UTC 2014 x86_64 x86_64
/usr/bin/python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)]
================================================================================
Using temporary directory /tmp/duplicity-FEINN7-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-FEINN7-tempdir/mkstemp-9nG9K1-1
Temp has 28068343808 available, backup will use approx 34078720.
ssh: [chan 1] listdir('/fnord/.')
Local and Remote metadata are synchronized, no sync needed.
ssh: [chan 1] listdir('/fnord/.')
0 files exist on backend
2 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: SSHParamikoBackend
Archive dir: /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
No signatures found, switching to full backup.
Using temporary directory /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-AWgmVf-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-AWgmVf-tempdir/mktemp-dir7rD-1
Using temporary directory /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-LwCC1O-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-LwCC1O-tempdir/mktemp-_Mxi_x-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-FEINN7-tempdir/mktemp-uaLzyb-2
Selecting /usr/sbin
Comparing . and None
Getting delta of (. dir) and None
A .
Selecting /usr/sbin/accessdb
Comparing accessdb and None
Getting delta of (accessdb reg) and None
A accessdb
...
Selecting /usr/sbin/zic
Comparing zic and None
Getting delta of (zic reg) and None
A zic
Removing still remembered temporary file /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-AWgmVf-tempdir/mktemp-dir7rD-1
Removing still remembered temporary file /root/.cache/duplicity/2f780b2947feb9f1de97bc51f312d898/duplicity-LwCC1O-tempdir/mktemp-_Mxi_x-1
AsyncScheduler: running task synchronously (asynchronicity disabled)
ssh: [chan 1] open('/fnord/duplicity-full.20150217T151048Z.vol1.difftar.gz', 'wb')
ssh: [chan 1] open('/fnord/duplicity-full.20150217T151048Z.vol1.difftar.gz', 'wb') -> 63343835363636643862346564363433
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
ssh: Sending global request "<email address hidden>"
[repeats]
--------------------------------------------------------------------------------------------------------------------

Revision history for this message
Dennis Schridde (devurandom) wrote :

I am having the same issue. Did you figure this one out?

What struck me was "Max packet out: 0 bytes". I wonder where Paramiko gets that information from and whether that literally means: I will not send anything, which would explain the situation.

Revision history for this message
Dennis Schridde (devurandom) wrote :

Version 1.15.4 works and reports max-packet-out > 0.

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

This problem has been fixed in the latest version at https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa

Changed in duplicity:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.