1.23-beta websocket incompatibility

[Marco Ceppi]

When trying to use the python-jujuclient library against an environment bootstrapped using 1.23-beta2 from ppa:juju/devel the client errors with the following traceback:

websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.

Reproduction:

On a clean 14.04 install:

```
sudo add-apt-repository ppa:juju/devel
sudo apt-get update
sudo apt-get install juju juju-core juju-local python-virtualenv

juju init
juju switch local
juju bootstrap
juju deploy trusty/ubuntu
juju status

virtualenv ~/ilovejuju
. ~/ilovejuju/bin/activate
pip install jujulcient pyyaml
```

The output of the above: http://paste.ubuntu.com/10722464/

$ juju version
1.23-beta2-trusty-amd64
$ dpkg -l | grep juju
ii juju-0.7 0.7+bzr628+bzr632~raring1 all next generation service orchestration system
ii juju-core 1.23-beta2-0ubuntu1~14.04.1~juju1 amd64 Juju is devops distilled - client
ii juju-deployer 0.4.3-0ubuntu1~ubuntu14.04.1~ppa1 all Deploy complex stacks of services using Juju
ii juju-local 1.23-beta2-0ubuntu1~14.04.1~juju1 all dependency package for the Juju local provider
ii juju-mongodb 2.4.9-0ubuntu3 amd64 MongoDB object/document-oriented database for Juju
ii juju-quickstart 2.0.1+bzr124+ppa32~ubuntu14.04.1 all Easy configuration of Juju environments
ii python-jujuclient 0.50.1-2 amd64 Python API client for juju-corez
>>> from jujuclient import Environment
>>> env = Environment.connect('local')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 460, in connect
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 142, in run
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 150, in connect_env
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 448, in __init__
  File "build/bdist.linux-x86_64/egg/jujuclient.py", line 162, in connect_socket
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_core.py", line 102, in create_connection
    websock.connect(url, **options)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_core.py", line 253, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_handshake.py", line 54, in handshake
    send(sock, header_str)
  File "/home/marco/Projects/benchmarking/collector-web/.venv/local/lib/python2.7/site-packages/websocket_client-0.28.0-py2.7.egg/websocket/_socket.py", line 109, in send
    raise WebSocketConnectionClosedException("socket is already closed.")
websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.

[Marco Ceppi (marcoceppi)]

Using "1.23-beta3" from gh:juju/1.23 branch (ref: 9710c29) this issue does not present itself. I suspect this will be resolved in the beta3 release of 1.23 if all commits up to 9710c29 are included. Can't really pinpoint which commit addresses this.

[Adam Collard (adam-collard)]

Just hit a very similar issue with juju-deployer and 1.23b3, filed as a duplicate.

https://bugs.launchpad.net/juju-deployer/+bug/1442723

[David Britton (davidpbritton)]

I tried this locally with 1.23b3 and didn't get it, there must be a race/flakiness of some kind where the connection is getting closed on us. Other than retrying the connection, I'm not sure there is anything to do in python-jujuclient. Marked incomplete while we wait for core to sound in.

[Curtis Hovey (sinzui)]

Our deployer tests of the landscape bundle with beta1-4 against maas (1.5, 1.7 and 1.8), joyent, aws, and Hp don't see this.

[Adam Collard (adam-collard)]

@Curtis - specifically I see it with "juju-deployer -T" after having the bundle deployed (to cleanup)

[Curtis Hovey (sinzui)]

@Adam, thank you. I can reproduce this issue for easily. Using a 1.23 beta3 or beta4
    juju bootstrap
    juju --deploy-delay 10 --config my-bundles.yaml
    juju --debug deployer -T

2015-04-10 20:05:24 INFO juju.cmd supercommand.go:37 running juju [1.23-beta4-trusty-amd64 gc]
2015-04-10 16:05:24 Resetting environment...
Traceback (most recent call last):
  File "/usr/bin/juju-deployer", line 9, in <module>
    load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
    run()
  File "/usr/lib/python2.7/dist-packages/deployer/cli.py", line 171, in run
    watch=options.watch)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 82, in reset
    self.wait_for_units(timeout, goal_state='removed', watch=watch)
  File "/usr/lib/python2.7/dist-packages/deployer/env/go.py", line 227, in wait_for_units
    services=services, on_errors=on_errors).run(callback)
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 1918, in run
    for change_set in self.watch:
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 404, in next
    return super(TimeoutWatcher, self).next()
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 346, in next
    'Id': self.watcher_id})
  File "/usr/lib/python2.7/dist-packages/jujuclient.py", line 273, in _rpc
    raw = self.conn.recv()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 663, in recv
    opcode, data = self.recv_data()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 678, in recv_data
    frame = self.recv_frame()
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 753, in recv_frame
    frame_buffer.recv_header(self._recv_strict)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 257, in recv_header
    header = recv_fn(2)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 860, in _recv_strict
    bytes = self._recv(shortage)
  File "/usr/lib/python2.7/dist-packages/websocket/_core.py", line 842, in _recv
    bytes = self.sock.recv(bufsize)
  File "/usr/lib/python2.7/ssl.py", line 341, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 260, in read
    return self._sslobj.read(len)
socket.error: [Errno 104] Connection reset by peer
2015-04-10 20:05:45 ERROR juju.cmd supercommand.go:430 subprocess encountered error code 1

[Marco Ceppi (marcoceppi)]

This was due to an issue with websocket-client 0.28.0 (thanks William!) using websocket-client 0.29.0 and above resolves this issue.

[William Reade (fwereade)]

Resolved by updating websocket-client python package to 0.29.0 or later (which can do SSL websocket connections).

https://github.com/liris/websocket-client/compare/v0.28.0...v0.29.0

[David Britton (davidpbritton)]

2 Things:

1) Andreas Hasenack has started a PPA for python-websocket:

    https://code.launchpad.net/~ahasenack/+archive/ubuntu/websocket-client-daily

This fix apparently works on Trusty.

2) On Vivid, I still get the error, not sure why, but I filed an upstream bug:

    https://github.com/liris/websocket-client/issues/174