ssshuttle server fails to connect endpoints with python 3.8

Bug #1873368 reported by Dan Hill
50
This bug affects 8 people
Affects Status Importance Assigned to Milestone
Sshuttle
Fix Released
Unknown
sshuttle (Debian)
Fix Released
Unknown
sshuttle (Ubuntu)
Fix Released
High
Unassigned
Xenial
Fix Released
High
Unassigned
Bionic
Fix Released
High
Unassigned
Focal
Fix Released
High
Felipe Reyes
Groovy
Fix Released
High
Unassigned

Bug Description

[Impact]

sshuttle fails to connect to a remote system with python >= 3.8, or (after initial patch) to remote system with python <= 3.4.

[Test Case]

connect from a system with sshuttle installed to a remote system, in all combinations (t/x/b/f/g with sshuttle to remote t/x/b/f/g system). All combinations should work.

The first failure, connecting to remote systems with python >= 3.8, will fail like:

$ sshuttle -r ubuntu@{ip-addr} {subnet-1}
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 38, in <module>
  File "sshuttle.server", line 298, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

The second failure, connecting to remote systems with python <= 3.4, will fail like:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 39, in <module>
  File "sshuttle.server", line 400, in main
  File "sshuttle.ssnet", line 598, in runonce
  File "sshuttle.ssnet", line 488, in callback
  File "sshuttle.ssnet", line 437, in flush
AttributeError: 'module' object has no attribute 'set_blocking'
client: fatal: server died with error code 1

[Regression Potential]

any regression would likely cause problems at connection initialization, i.e. when connecting from the sshuttle system to the remote system. it's unlikely this would cause any regression that occurs after the initial setup has been completed.

[scope]

First, I'll note this regression illustrates the importance of the [scope] section, and why I always include it in my SRUs...

tl;dr for scope is 2 fixes are needed (work with remote py >= 3.8 and work with remote py <= 3.4), and both fixes are needed in sshuttle for all releases.

details:

this is needed for all releases; x, b, f, and g. However there are 2 parts to fixing this; the first part is fixing sshuttle connecting from any release to a system with python >= 3.8. That is done for g, and in proposed for f, and not done for b or x. The second part is to correct the first patch's regression to allow sshuttle connecting from any release to a system with python <= 3.4. That is needed for x, b, f, and g.

a good scope table from @smoser is in comment 26.

[Other Info]

https://github.com/sshuttle/sshuttle/issues/381
https://bugs.python.org/issue39685
https://bugs.python.org/issue35415

[Original Description]

Client
$ python3 --version
Python 3.8.2
$ lsb_release -rd
Description: Ubuntu Focal Fossa (development branch)
Release: 20.04
$ apt-cache policy sshuttle
sshuttle:
  Installed: 0.78.5-1
  Candidate: 0.78.5-1

Server
$ python3 --version
Python 3.8.2
$ lsb_release -rd
Description: Ubuntu 20.04 LTS
Release: 20.04
$ apt-cache policy openssh-server
openssh-server:
  Installed: 1:8.2p1-4
  Candidate: 1:8.2p1-4

$ sshuttle -r ubuntu@{ip-addr} {subnet-1} {subnet-2}
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 38, in <module>
  File "sshuttle.server", line 298, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

The sshuttle upstream tracker is issue#381 [0]. They are waiting on a response to bpo#39685 [1].

This regression was introduced in python 3.8 by bpo#35415 [2], which restricts socket.fromfd() calls to provide valid socket family file descriptors.

[0] https://github.com/sshuttle/sshuttle/issues/381
[1] https://bugs.python.org/issue39685
[2] https://bugs.python.org/issue35415

Related branches

Revision history for this message
Dan Hill (hillpd) wrote :

The python docs [0] indicate that the file descriptor should be a socket:
"The file descriptor should refer to a socket, but this is not checked — subsequent operations on the object may fail if the file descriptor is invalid."

The docs do need to be corrected. bpo#35415 now explicitly checks fd to ensure they are sockets.

The fix for this issue likely needs to be in sshuttle, not python.

[0] https://docs.python.org/3/library/socket.html#socket.fromfd

Dan Hill (hillpd)
tags: added: seg
Revision history for this message
Dave Walker (davewalker) wrote :

Not ideal - but if the remote system has multiple versions of python, you can work around it with:
sshuttle --python=/usr/bin/python3.7 -r ..

Dan Hill (hillpd)
Changed in sshuttle (Ubuntu):
importance: Undecided → High
Eric Desrochers (slashd)
Changed in sshuttle (Ubuntu):
status: New → Confirmed
Changed in sshuttle (Ubuntu Focal):
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Pres-Gas (presgas) wrote :

I did attempt to use "sudo pip3 install sshuttle" after "apt purge sshuttle" with the same issue.

For me, this started happening when I upgraded my target server to 20.04. I did keep the sshd_config file the same. Not sure what that means. Let me know if I can provide any info on that.

Additionally, I changed my target server I connect to a RHEL server. That works.

Revision history for this message
Pres-Gas (presgas) wrote :

I found an upstream bug and added details to that. If I point to python2, client 20.04 and target 20.04 works. If I do NOT point sshuttle to python2, but keep it at python3 it fails.

Keeping client at python3 while target is RHEL 7.7 works. Please see my post in this upstream bug:

https://github.com/sshuttle/sshuttle/issues/381#issuecomment-624607821

Perhaps this can be coordinated with upstream?

Revision history for this message
Dan Hill (hillpd) wrote :

RHEL 7.7 packages python 3.6. That's why it works.

If the host is running python 3.8+, then you have to work around this issue by pointing to a different version of python that's available on the host.

Revision history for this message
Dan Hill (hillpd) wrote :

This is fixed by pr#431 [0], which landed in v1.0.1.

[0] https://github.com/sshuttle/sshuttle/pull/431

Dan Hill (hillpd)
Changed in sshuttle (Ubuntu Focal):
assignee: nobody → Dan Hill (hillpd)
Changed in sshuttle (Ubuntu Groovy):
assignee: nobody → Dan Hill (hillpd)
Revision history for this message
Eric Desrochers (slashd) wrote :

Here's my recommendation

Let's bump Debian unstable to the version which contains the fix upstream (which seems to be v1.0.1)

Then let's sync groovy from Debian to be on the same version (Taking benefit of the development cycle to bump the version is best)

Then let's cherry-pick the commit into v0.78 (if feasible) through SRU for Focal and any other impacted and supported releases.

- Eric

Revision history for this message
Felipe Reyes (freyes) wrote :

I created a ppa that contains the backported patch in case anyone wants to test it - https://launchpad.net/~freyes/+archive/ubuntu/lp1873368

Revision history for this message
Eric Desrochers (slashd) wrote :

I wrote to the debian maitainer which also seems to be the author of the desired upstream patch.

Please see debbugs #953621

I offered him 2 options

* (A) Bump "sshuttle" to upstream release "1.0.2" and take benefit of that to modernize the package to a let's say more modern dh compat version, ...

* (B) Cherry-pick the 3.8 fix[0] into Debian so that Ubuntu can then follows.

- Eric

Changed in sshuttle (Debian):
status: Unknown → Confirmed
Revision history for this message
Scott Moser (smoser) wrote :

@slashd,

Are you suggesting your 'A' (version 1.0.2) for focal? Unless there are other reasons than this bug, that feels like not the right path for SRU.

So that means 'B' is the path forward for focal. So I might suggest just doing 'B' for groovy now (adding ubuntu delta). That would then allow ubuntu to move forward, get some easier real-world testing, and satisfy the "fixed in development release" requirement for SRU.

Presumably the next upload to debian will have a fix, so that would be a short-term delta and we can do a sync of whatever debian does.

Revision history for this message
Eric Desrochers (slashd) wrote :

@smoser,

Not for focal, but for the active development release and debian unstable I suggest we bump to 1.0.2.

As part of the SRU, just cherry-pick the desired fix.

- Eric

Changed in sshuttle (Debian):
status: Confirmed → Fix Released
Revision history for this message
Eric Desrochers (slashd) wrote :

[Groovy]

# Launchpad
https://launchpad.net/ubuntu/+source/sshuttle

# rmadison
 sshuttle | 1.0.3-1 | groovy/universe | source, all

# changelog
sshuttle (1.0.3-1) unstable; urgency=medium

  [ Brian May ]
  * Update to latest upstream version. Closes: #953621.
  * Upstream has dropped Python 2 support.

  [ Scott Talbert ]
  * Remove unused Python 2 dependencies. Closes: #943238.
  * Remove ancient X-Python3-Version field
  * Add missing ${sphinxdoc:Depends}
  * Update debhelper-compat version to 13
  * Update Standards-Version to 4.5.0 (no changes needed)

 -- Brian May <email address hidden> Sun, 12 Jul 2020 19:21:26 +1000

Changed in sshuttle (Ubuntu Groovy):
status: Confirmed → Fix Released
Revision history for this message
Eric Desrochers (slashd) wrote :

Next step is to introduce the fix through SRU into sshuttle version "0.78.5-1".

Changed in sshuttle (Ubuntu Groovy):
assignee: Dan Hill (hillpd) → nobody
Changed in sshuttle (Ubuntu Focal):
status: Confirmed → In Progress
Revision history for this message
Eric Desrochers (slashd) wrote :

@freyes

Do you have your focal debdiff handy still ?
I'll be happy to sponsor the work you did for sshuttle focal.

- Eric

Eric Desrochers (slashd)
description: updated
description: updated
Eric Desrochers (slashd)
Changed in sshuttle (Ubuntu Focal):
assignee: Dan Hill (hillpd) → Felipe Reyes (freyes)
Revision history for this message
Felipe Reyes (freyes) wrote :

Attaching debdiff for focal that contains a backport of commit https://github.com/sshuttle/sshuttle/commit/9c873579348e3308123d1bf2a917b0c2f82b9dae

Revision history for this message
Scott Moser (smoser) wrote :

@Felipe,

Two things I think should be changed in your debdiff:
 a.) version should be 0.78.5-1ubuntu0.1 instead of 0.78.5-1ubuntu1 per
https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging . I'm not certain on that though.
 b.) the referenced commit (9c873579) is not the commit that was added to upstream master. Per your link "This commit does not belong to any branch on this repository." The correct value is 6c21addde97c4582b3dccb22bca64c33af3e5eff [1] which landed in upstream/master. You should update the value in debian/patches/lp1873368.patch .

--
[1] https://github.com/sshuttle/sshuttle/commit/6c21addde97c4582b3dccb22bca64c33af3e5eff

Eric Desrochers (slashd)
tags: added: sts-sponsor-slashd
Revision history for this message
Eric Desrochers (slashd) wrote :

[sts-sponsor]

Thanks for your contribution Felipe.

The debdiff has been sponsored with the following minor changes:

- Renamed the quilt patch to "d/p/0001-compatibility-with-python38.patch"
- Removed the py2 build dependencies as they are no longer used and prevent to build properly
(python-pytest-runner, python-pytest-cov
- Added a DEP3 header in the quilt patch.
- Slighly modified d/changelog to reflect the minor changes.

- Eric

Eric Desrochers (slashd)
tags: added: sts
removed: sts-sponsor-slashd
Eric Desrochers (slashd)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

I am not super happy with the version number chosen, since we have 0.78.5-1 in both focal and eoan, but then again eoan is going EOL (in theory) today, so I'll accept it as-is.

Changed in sshuttle (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Dan, or anyone else affected,

Accepted sshuttle into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sshuttle/0.78.5-1ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Chris Johnston (cjohnston) wrote :

Prior to updating:

$ sshuttle -r sts-bastion 10.5.0.0/16
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 38, in <module>
  File "sshuttle.server", line 298, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

Upgrading:

Preparing to unpack .../sshuttle_0.78.5-1ubuntu1_all.deb ...
Unpacking sshuttle (0.78.5-1ubuntu1) over (0.78.5-1) ...
Setting up sshuttle (0.78.5-1ubuntu1) ...
Processing triggers for man-db (2.9.1-1) ...

After upgrade:

$ sshuttle -r sts-bastion 10.5.0.0/16
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
client: warning: closed channel 4 got cmd=TCP_DATA len=2048
client: warning: closed channel 4 got cmd=TCP_DATA len=1961
^Cclient:
client: Keyboard interrupt: exiting.

Works well here

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Eric Desrochers (slashd) wrote :

Thanks Chris ! Will ping SRU team to release it sometime this or next week.

Revision history for this message
William Grant (wgrant) wrote :

This breaks connecting to Python 3.4 hosts, e.g. trusty:

client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 39, in <module>
  File "sshuttle.server", line 400, in main
  File "sshuttle.ssnet", line 598, in runonce
  File "sshuttle.ssnet", line 488, in callback
  File "sshuttle.ssnet", line 437, in flush
AttributeError: 'module' object has no attribute 'set_blocking'
client: fatal: server died with error code 1

tags: added: regression-proposed verification-failed-focal
removed: verification-done verification-done-focal
William Grant (wgrant)
tags: removed: regression-proposed
Revision history for this message
Scott Moser (smoser) wrote :

@William,

The fix here does break python < 3.5 (as advertised in the debian/patches/ patch). It is mostly trivial to make this change in a backwards compatible way, so we should probably do that.

I wonder what the official policy is on breaking this "client"s interaction with an ubuntu release version that is now under ESM. I didn't see anything that obviously covered this in https://wiki.ubuntu.com/StableReleaseUpdates .

Revision history for this message
Eric Desrochers (slashd) wrote :

I wrote to sil2100 (SRU verification team member)

He'll take a look at it when time permit, and he should update the bug anytime soon with his 2 cents about the current situation.

- Eric

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hey! So I guess I'd like to know a bit more before we proceed. Since generally I do not like introducing regressions from previous behavior, but I would first like to understand a bit more the use-case that is now broken by this SRU.

@wgrant could you give us a bit more of an info which case in focal is broken? Does this mean that by using sshuttle on focal you can no longer connect to any trusty hosts?

Revision history for this message
Scott Moser (smoser) wrote :

@Lukasz

sshuttle does not require itself to be installed on the server. Rather it "pushes" itself from the client to the server. So the code that executes on client is the same as that on the server.

Maybe the table below will help:

client | server | comment
trusty | focal | broken due to py3.8 on focal (would need SRU to trusty)
bionic | focal | broken due to py3.8 on focal (would need SRU to bionic)
focal | focal | was broken, is fixed by 0.78.5-1ubuntu1
focal | bionic | works before and after
focal | trusty | broken by 0.78.5-1ubuntu1
trusty | trusty | works

Revision history for this message
Scott Moser (smoser) wrote :

@all,

Please review a MP at https://code.launchpad.net/~smoser/ubuntu/+source/sshuttle/+git/sshuttle/+merge/388062 that will fix this for focal -> trusty and keep focal -> focal working.

Revision history for this message
Eric Desrochers (slashd) wrote :

@scott,

Would it be worth submitting it upstream for them to stay backward compatible within the python3 series ?

That way we won't have to carry the patch over and over, if this become part of the upstream code.

- Eric

Revision history for this message
Scott Moser (smoser) wrote :

@Eric,
Well... We'd only be carring the change for 20.04. I would not suggest to carry it for 20.10 or beyond. As my change is right now I dont' think I would accept it for upstream. It should be sufficient for a stable release though.

I really suspect that if upstream cared about python < 3.5, they'd have maintained it.

Revision history for this message
William Grant (wgrant) wrote :

IMO we should carry it until trusty ESM expires. There are still trusty systems around, and groovy users who need to connect to those trusty systems.

Revision history for this message
Eric Desrochers (slashd) wrote :

IMO, we should not assume that upstream cared or not about py < 3.5.
As William and I pointed out, we will most likely have to carry the patch to future release due to ESM.

I think that opening the discussion upstream to get their official position on this might be insightful, and then we can see which course of action to follow next.

(a) Upstream to merge a patch to remain backward compatible that we can cherry-pick
(b) Us to carry a "UBUNTU SAUCE:" patch for sshuttle
or else ....

Revision history for this message
Scott Moser (smoser) wrote :

@Eric, William,

I think you're taking a very narrow view on this when a more thoughtful review is needed.

Ubuntu should identify a general policy on 'client <-> server' version backwards and forwards compatibility guarantees. That policy should be implemented here. The fact this particular client-server mismatch is somewhat trivial to fix should not really be individually be considered.

It is simply not feasible for Ubuntu to support and carry delta from upstream for all clients shipping in 23.10 that do not interoperate with servers shipped in 14.04.

@Eric,
You are welcome to float the patch upstream. But they *have* intentionally and specifically dropped python 3.4 support. That doesn't seem unreasonable since python 3.4 went end of life in March of 2019.

https://github.com/sshuttle/sshuttle/commit/1b50d364c67ae1eb9dc831e312fc11b75f4ad43e

Revision history for this message
Eric Desrochers (slashd) wrote :
Changed in sshuttle:
status: Unknown → New
Dan Streetman (ddstreet)
description: updated
Changed in sshuttle (Ubuntu Groovy):
status: Fix Released → In Progress
Changed in sshuttle (Ubuntu Focal):
status: Fix Committed → In Progress
Changed in sshuttle (Ubuntu Bionic):
status: New → In Progress
Changed in sshuttle (Ubuntu Xenial):
status: New → In Progress
Dan Streetman (ddstreet)
Changed in sshuttle (Ubuntu Bionic):
importance: Undecided → High
Changed in sshuttle (Ubuntu Xenial):
importance: Undecided → High
Revision history for this message
Dan Streetman (ddstreet) wrote :

I don't think the upstream fix for this is actually complete; MuxWrapper passes the files up to the SockWrapper constructor, but SockWrapper still thinks they are sockets and uses them as such, e.g. when calling self.rsock.setblocking(False) which will fail when SockWrapper.rsock is a file, not a socket.

Revision history for this message
Dan Streetman (ddstreet) wrote :

> I don't think the upstream fix for this is actually complete

this seems to only be an issue in bionic and just requires upstream commit 0dba8a8beb5e2a6f16042421455004c2e7587ed4 to simply ignore the failure.

I'm still concerned about upstream's casual use of stdin/stdout fd in place of sockets, as MuxWrapper() passes the rfile/wfile up to SocketWrapper without changing SocketWrapper's handling of the 'sockets' at all. I suppose that's an issue for upstream to work out, however.

Revision history for this message
Dan Streetman (ddstreet) wrote :
Changed in sshuttle:
status: New → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote :

So, what's actually happening here? My understanding of the current state is that:
1) sshuttle 0.78.5-1ubuntu1 in focal-proposed introduces a regression in connecting to trusty hosts
2) There's a patch merged upstream which fixes that regression
3) There's a desire for an explicit policy on when SRUs can regress connecting to old hosts

It seems that since we don't currently have an exception to the normal “no regressions” policy for (3) that the thing to move this forward is for someone to fold in the fix in (2) and upload a new version that fixes this bug while *not* regressing the ability to connect to trusty hosts.

Revision history for this message
Dan Streetman (ddstreet) wrote :

To follow up, with my test builds from comment 36:

FROM trusty:
WORKS TO python2 in t/x/b/g
FAILS to python2 in f
FAILS to python3 in t/x/b/f/g

FROM xenial:
WORKS TO python2 in t/x/b/g
FAILS to python2 in f
FAILS to python3 in t/f
WORKS TO python3 in x/b/g

FROM bionic:
WORKS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

FROM focal:
WORKS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

FROM focal:
FAILS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

Revision history for this message
Dan Streetman (ddstreet) wrote :

Sorry the last entry in that table should be FROM *groovy* not focal.

Revision history for this message
Dan Streetman (ddstreet) wrote :

Failure detail:

FROM trusty TO focal python2:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 26, in <module>
  File "server.py", line 170, in main
  File "server.py", line 71, in list_routes
  File "server.py", line 51, in _list_routes
  File "ssubprocess.py", line 606, in __init__
  File "ssubprocess.py", line 1117, in _execute_child
OSError: [Errno 2] No such file or directory
client: fatal: server died with error code 1

FROM trusty TO any python3:
  File "<string>", line 1
    import sys; skip_imports=1; verbosity=0; exec compile(sys.stdin.read(764), "assembler.py", "exec")
SyntaxError: invalid syntax

FROM xenial TO focal python2:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 36, in <module>
  File "sshuttle.server", line 229, in main
  File "sshuttle.server", line 84, in list_routes
  File "sshuttle.server", line 63, in _list_routes
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
client: fatal: server died with error code 1

FROM xenial TO trusty python3:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 36, in <module>
  File "sshuttle.server", line 243, in main
TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'
client: fatal: server died with error code 1

FROM xenial TO focal python3:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 36, in <module>
  File "sshuttle.server", line 229, in main
  File "sshuttle.server", line 84, in list_routes
  File "sshuttle.server", line 63, in _list_routes
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'netstat'
client: fatal: server died with error code 1

FROM groovy TO any python2:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 18, in <module>
TypeError: module.__init__() argument 1 must be string, not unicode
client: fatal: server died with error code 1

Revision history for this message
Dan Streetman (ddstreet) wrote :

My test builds include the patch from:
https://github.com/sshuttle/sshuttle/pull/507#

This upstream bug may still apply:
https://github.com/sshuttle/sshuttle/issues/508

Revision history for this message
Dan Streetman (ddstreet) wrote :

The failure from t/x->f with python2, and x->f with python3, is due to the net-tools package not being installed by default in focal (sshuttle fails trying to remotely run 'netstat'), while it IS installed by default in t/x/b/g. Not sure if that's an error that should be corrected or not.

In any case, once net-tools is installed in the focal test instance, the updated matrix is:

FROM trusty:
WORKS TO python2 in t/x/b/f/g
FAILS to python3 in t/x/b/f/g

FROM xenial:
WORKS TO python2 in t/x/b/f/g
FAILS to python3 in t
WORKS TO python3 in x/b/f/g

FROM bionic:
WORKS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

FROM focal:
WORKS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

FROM groovy:
FAILS TO python2 in t/x/b/f/g
WORKS TO python3 in t/x/b/f/g

Revision history for this message
Dan Streetman (ddstreet) wrote :

Note that the t/x failures are *not* regressions related to this bug; they already failed with the existing versions in -release or -updates.

FROM trusty:
FAILS to python3 in t/x/b/f/g

FROM xenial:
FAILS to python3 in t

As t is ESM only now, I'm going to just ignore its failure connecting to any remote system with python3.

Since x works for py2 and py3 for all remote releases except t, and it does work with py2 in t, and the default python in t is py2, and the failure is not new (not caused by any change for this bug), I'm ignoring this failure as well.

Leaving the only remaining failures being from g to any remote release with py2:

FROM groovy:
FAILS TO python2 in t/x/b/f/g

Revision history for this message
Dan Streetman (ddstreet) wrote :

Looks like the g failures to py2 remote is fixed upstream by c12d2ba5c6d0c7287b1e316fec4f9c9bff63c328, so I think we're almost at the happy place.

Revision history for this message
Dan Streetman (ddstreet) wrote :

uploaded to x/b/f/g

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sshuttle - 1.0.4-1ubuntu1

---------------
sshuttle (1.0.4-1ubuntu1) groovy; urgency=medium

  * d/p/lp1873368/0001-Fix-python2-server-compatibility.patch,
    d/p/lp1873368/0002-Fix-flake8-line-too-long.patch,
    d/p/lp1873368/0003-Fix-python2-client-compatibility.patch:
    - fix compatibility with remote py2 (LP: #1873368)
  * d/t/control, d/t/cross-release:
    - add autopkgtest for cross-release compatibility checks

 -- Dan Streetman <email address hidden> Fri, 18 Sep 2020 13:57:01 -0400

Changed in sshuttle (Ubuntu Groovy):
status: In Progress → Fix Released
Revision history for this message
Dan Streetman (ddstreet) wrote :

uploaded fixed versions for x/b/f as well as creating an autopkgtest to check inter-release operation, added starting in groovy:
https://autopkgtest.ubuntu.com/packages/sshuttle

Revision history for this message
Chris Halse Rogers (raof) wrote :

Hello Dan, or anyone else affected,

Accepted sshuttle into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sshuttle/0.78.5-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sshuttle (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
removed: verification-failed-focal
Revision history for this message
Chris Halse Rogers (raof) wrote :

Hello Dan, or anyone else affected,

Accepted sshuttle into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sshuttle/0.78.3-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in sshuttle (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Changed in sshuttle (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Chris Halse Rogers (raof) wrote :

Hello Dan, or anyone else affected,

Accepted sshuttle into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sshuttle/0.76-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Dan Streetman (ddstreet) wrote :

root@lp1896299-x:~# dpkg -l|grep sshuttle
ii sshuttle 0.76-1ubuntu1 all Transparent proxy server for VPN over SSH

root@lp1896299-x:~# sshuttle --python python3 -r lp1896299-g.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 36, in <module>
  File "sshuttle.server", line 239, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

root@lp1896299-x:~# dpkg -l|grep sshuttle
ii sshuttle 0.76-1ubuntu1.1 all Transparent proxy server for VPN over SSH

root@lp1896299-x:~# sshuttle --python python3 -r lp1896299-g.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Revision history for this message
Dan Streetman (ddstreet) wrote :

root@lp1896299-b:~# dpkg -l|grep sshuttle
ii sshuttle 0.78.3-1ubuntu1 all Transparent proxy server for VPN over SSH
root@lp1896299-b:~# sshuttle --python python3 -r lp1896299-f.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 37, in <module>
  File "sshuttle.server", line 281, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

root@lp1896299-b:~# dpkg -l|grep sshuttle
ii sshuttle 0.78.3-1ubuntu1.1 all Transparent proxy server for VPN over SSH
root@lp1896299-b:~# sshuttle --python python3 -r lp1896299-f.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Dan Streetman (ddstreet) wrote :

root@lp1896299-f:~# dpkg -l|grep sshuttle
ii sshuttle 0.78.5-1 all Transparent proxy server for VPN over SSH
root@lp1896299-f:~# sshuttle -r lp1896299-f.lxd 1.2.3.4/24
The authenticity of host 'lp1896299-f.lxd (10.202.51.43)' can't be established.
ECDSA key fingerprint is SHA256:PydUWn39X3KKq0obuNhd7vf7oll1BehZGAPynxDNIdI.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'lp1896299-f.lxd,10.202.51.43' (ECDSA) to the list of known hosts.
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 38, in <module>
  File "sshuttle.server", line 298, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

root@lp1896299-f:~# dpkg -l|grep sshuttle
ii sshuttle 0.78.5-1ubuntu1.1 all Transparent proxy server for VPN over SSH
root@lp1896299-f:~# sshuttle -r lp1896299-f.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

Hello Dan, or anyone else affected,

Accepted sshuttle into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sshuttle/0.76-1ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed verification-needed-xenial
removed: verification-done verification-done-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sshuttle - 0.78.5-1ubuntu1.1

---------------
sshuttle (0.78.5-1ubuntu1.1) focal; urgency=medium

  * d/p/lp1873368/0001-allow-Mux-flush-fill-to-work-with-python-3.5.patch,
    d/p/lp1873368/0002-Fix-python2-server-compatibility.patch:
    - fix interoperability with remote py2 and/or py3
      follow-on to last commit for (LP: #1873368)

sshuttle (0.78.5-1ubuntu1) focal; urgency=medium

  * d/p/0001-compatibility-with-python38.patch:
    Backport patch to make sshuttle compatible
    with python-3.8. (LP: #1873368). Closes: #953621.
  * d/control: Remove unused Python 2 build
    dependencies since its source package
    dropped python2 support. Closes: #943238.

 -- Dan Streetman <email address hidden> Fri, 18 Sep 2020 08:15:24 -0400

Changed in sshuttle (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for sshuttle has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sshuttle - 0.78.3-1ubuntu1.1

---------------
sshuttle (0.78.3-1ubuntu1.1) bionic; urgency=medium

  [ Dan Streetman ]
  * d/p/lp1592853-Use-versions-of-python3-greater-than-3.5-when-availa.patch:
    - detect if remote python is 'python' or 'python3' (LP: #1592853)
  * d/p/lp1873368/0001-Don-t-crash-if-we-can-t-look-up-peername.patch,
    d/p/lp1873368/0002-allow-Mux-flush-fill-to-work-with-python-3.5.patch,
    d/p/lp1873368/0003-Fix-python2-server-compatibility.patch,
    - fix interoperability with other releases py2/py3
      (same LP bug as below)

  [ Felipe Reyes ]
  * d/p/0001-compatibility-with-python38.patch:
    Backport patch to make sshuttle compatible
    with python-3.8. (LP: #1873368). Closes: #953621.

 -- Felipe Reyes <email address hidden> Sat, 19 Sep 2020 13:54:06 -0400

Changed in sshuttle (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Dan Streetman (ddstreet) wrote :

root@sshuttle-x:~# dpkg -l | grep sshuttle
ii sshuttle 0.76-1ubuntu1 all Transparent proxy server for VPN over SSH
root@sshuttle-x:~# sshuttle --python python3 -r sshuttle-g.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "assembler.py", line 36, in <module>
  File "sshuttle.server", line 239, in main
  File "/usr/lib/python3.8/socket.py", line 544, in fromfd
    return socket(family, type, proto, nfd)
  File "/usr/lib/python3.8/socket.py", line 231, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 88] Socket operation on non-socket
client: fatal: server died with error code 1

root@sshuttle-x:~# dpkg -l | grep sshuttle
ii sshuttle 0.76-1ubuntu1.2 all Transparent proxy server for VPN over SSH
root@sshuttle-x:~# sshuttle --python python3 -r sshuttle-g.lxd 1.2.3.4/24
assembler.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
client: Connected.

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package sshuttle - 0.76-1ubuntu1.2

---------------
sshuttle (0.76-1ubuntu1.2) xenial; urgency=medium

  * d/p/lp1897987-Backward-compatibility-with-Python-2.4-server.patch:
    - Fix compatibility with remote py3.4 (LP: #1897987)
  * d/p/lp1897961-Fix-shell-quoting.patch:
    - Add missing import needed by previous commit backport
      (LP: #1897961)

sshuttle (0.76-1ubuntu1.1) xenial; urgency=medium

  [ Dan Streetman ]
  * d/p/lp1896299/0001-Add-support-for-iproute2.patch,
    d/p/lp1896299/0002-Small-refactoring-of-netstat-iproute-parsing.patch,
    d/p/lp1896299/0003-Backward-compatibility-with-Python-2.4-server.patch:
    - allow connecting to remote without netstat (LP: #1896299)
  * d/p/lp1592853-Use-versions-of-python3-greater-than-3.5-when-availa.patch:
    - detect if remote python is 'python' or 'python3' (LP: #1592853)
  * d/p/lp1873368/0001-Don-t-crash-if-we-can-t-look-up-peername.patch,
    d/p/lp1873368/0002-allow-Mux-flush-fill-to-work-with-python-3.5.patch,
    d/p/lp1873368/0003-Fix-python2-server-compatibility.patch:
    - fix interoperability with remote py2 and/or py3
      (same bug as below)

  [ Felipe Reyes ]
  * d/p/0001-compatibility-with-python38.patch:
    Backport patch to make sshuttle compatible
    with python-3.8. (LP: #1873368). Closes: #953621.

 -- Dan Streetman <email address hidden> Wed, 30 Sep 2020 20:34:45 -0400

Changed in sshuttle (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Peter De Sousa (pjds) wrote :

Seeing this bug again on a Focal -> Focal system sshuttle 0.78.5

Revision history for this message
Dan Streetman (ddstreet) wrote :

@pjds this bug is closed, and also I see no failure on my focal system.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.