pssh (parallel-scp) invalid argument

[Shane D]

Binary package hint: pssh

Whenever running parallel-scp, it looks like it always yields an "invalid argument" for each client

swdixon(17884)% parallel-scp -h ~/hosts/hosts -l testuser ~/test /tmp/test
[1] 08:31:57 [FAILURE] host1 (22, 'Invalid argument')
[2] 08:31:57 [FAILURE] host2 (22, 'Invalid argument')

I've been able to get parallel-ssh to work. It looks like it could be something in the python code that's trying to put an argument in the wrong place. Scp works on all these hosts and testuser has public/private key authentication to each host (which has been tested and works).

ii ssh 1:4.7p1-8ubuntu1.2 secure shell client and server (metapackage)
ii pssh 1.3.1-5 Parallel versions of SSH-based tools

amd64

[Brendan Martens (shrift)]

I have also experienced this; when running parallel-scp, or parallel-rsync I get a bunch of "[1] 16:28:58 [FAILURE] server.com (22, 'Invalid argument')". Parallel-ssh seems to work just fine, I'm using the same hosts file as for parallel-ssh.

[Brendan Martens (shrift)]

Confirming this bug as I have the same problem.

[Brendan Martens (shrift)]

Sorry for so many extra comments.... I am also using amd64.

[Panagiotis Astithas (pastith)]

The following patch fixes it for me:

--- /usr/bin/parallel-scp 2008-05-07 04:54:39.000000000 +0300
+++ parallel-scp 2009-02-19 17:43:37.000000000 +0200
@@ -21,7 +21,7 @@
 from psshlib.basethread import BaseThread

 _DEFAULT_PARALLELISM = 32
-_DEFAULT_TIMEOUT = sys.maxint # "infinity" by default
+_DEFAULT_TIMEOUT = 60

 def print_usage():
     print "Usage: pscp [OPTIONS] -h hosts.txt local remote"

[Panagiotis Astithas (pastith)]

This is probably a better solution:

--- /usr/bin/parallel-scp 2008-05-07 04:54:39.000000000 +0300
+++ parallel-scp 2009-02-20 13:14:44.000000000 +0200
@@ -21,7 +21,10 @@
 from psshlib.basethread import BaseThread

 _DEFAULT_PARALLELISM = 32
-_DEFAULT_TIMEOUT = sys.maxint # "infinity" by default
+if sys.maxint >> 33:
+ _DEFAULT_TIMEOUT = sys.maxint >> 33
+else:
+ _DEFAULT_TIMEOUT = sys.maxint # "infinity" by default

 def print_usage():
     print "Usage: pscp [OPTIONS] -h hosts.txt local remote"

[Brendan Martens (shrift)]

Should I be trying to apply this directly to /usr/bin/parallel-scp? When I do this I get an error:

brendanmartens@hairball:~$ sudo patch /usr/bin/parallel-ssh psh.diff
patching file /usr/bin/parallel-ssh
Hunk #1 FAILED at 21.
1 out of 1 hunk FAILED -- saving rejects to file /usr/bin/parallel-ssh.rej
brendanmartens@hairball:~$
brendanmartens@hairball:~$ cat psh.diff
--- /usr/bin/parallel-scp 2008-05-07 04:54:39.000000000 +0300
+++ parallel-scp 2009-02-20 13:14:44.000000000 +0200
@@ -21,7 +21,10 @@
 from psshlib.basethread import BaseThread

 _DEFAULT_PARALLELISM = 32
-_DEFAULT_TIMEOUT = sys.maxint # "infinity" by default
+if sys.maxint >> 33:
+ _DEFAULT_TIMEOUT = sys.maxint >> 33
+else:
+ _DEFAULT_TIMEOUT = sys.maxint # "infinity" by default

 def print_usage():
     print "Usage: pscp [OPTIONS] -h hosts.txt local remote"
brendanmartens@hairball:~$

brendanmartens@hairball:~$ cat /usr/bin/parallel-scp
#!/usr/bin/python
# -*- Mode: python -*-
#
# Usage: pscp [OPTIONS] -h hosts.txt local remote
#
# Parallel scp to the set of nodes in hosts.txt. For each node,
# we essentially do a scp [-r] local user@host:remote. This program
# also uses the -q (quiet) and -C (compression) options. Note
# that remote must be an absolute path.
#
# Created: 16 August 2003
#
# $Id: pscp,v 1.1.1.1 2005/12/31 10:03:33 bnc Exp $
#
import fcntl, os, popen2, pwd, select, signal, sys, threading, time

basedir, bin = os.path.split(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append("%s" % basedir)

import psshlib
from psshlib.basethread import BaseThread

_DEFAULT_PARALLELISM = 32
_DEFAULT_TIMEOUT = sys.maxint # "infinity" by default

def print_usage():
    print "Usage: pscp [OPTIONS] -h hosts.txt local remote"
    print
    print " -r --recursive recusively copy directories (OPTIONAL)"
    print " -h --hosts hosts file (each line \"host[:port] [login]\")"
    print " -l --user username (OPTIONAL)"
    print " -p --par max number of parallel threads (OPTIONAL)"
    print " -o --outdir output directory for stdout files (OPTIONAL)"
    print " -e --errdir output directory for stderr files (OPTIONAL)"
    print " -t --timeout timeout in seconds to do scp to a host (OPTIONAL)"
    print " -O --options SSH options (OPTIONAL)"
    print
    print "Example: pscp -h hosts.txt -l irb2 foo.txt /home/irb2/foo.txt"
    print

def read_envvars(flags):
    if os.getenv("PSSH_HOSTS"):
        flags["hosts"] = os.getenv("PSSH_HOSTS")
    if os.getenv("PSSH_USER"):
        flags["user"] = os.getenv("PSSH_USER")
    if os.getenv("PSSH_PAR"):
        flags["par"] = int(os.getenv("PSSH_PAR"))
    if os.getenv("PSSH_OUTDIR"):
        flags["outdir"] = os.getenv("PSSH_OUTDIR")
    if os.getenv("PSSH_ERRDIR"):
        flags["errdir"] = os.getenv("PSSH_ERRDIR")
    if os.getenv("PSSH_TIMEOUT"):
        flags["timeout"] = int(os.getenv("PSSH_TIMEOUT"))
    if os.getenv("PSSH_OPTIONS"):
        flags["options"] = os.getenv("PSSH_OPTIONS")

def parsecmdline(argv):
    import getopt
    shortopts = "rh:l:p:o:e:t:O:"
    longopts = [ "recursive", "hosts" , "user", "par", "outdir", "errdir",
                 "timeout", "options" ]
    flags = {...

[Panagiotis Astithas (pastith)]

Before applying the patch try to:

$ cd /usr/bin

[Panagiotis Astithas (pastith)]

Apparently this bug was fixed in version 1.4.0:
http://www.theether.org/pssh/ChangeLog
Jaunty has 1.4.3, so this should be already fixed there. Now if only we could get a backport...