bzr fails with python2.6 and https missing sendall

[Saša Janiška]

Hi!

Attempt to pull the code for bzr-push-and-update plugin crashes bzr with:

[gour@nitai plugins] bzr branch https://launchpad.net/bzr-push-and-update bzr_push_and_update
/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib2_wrappers.py:254: DeprecationWarning: socket.ssl() is deprecated. Use ssl.wrap_socket() instead.
  ssl = socket.ssl(self.sock, self.key_file, self.cert_file)
/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib2_wrappers.py:255: DeprecationWarning: FakeSocket is deprecated, and won't be in 3.x. Use the result of ssl.wrap_socket() directly instead.
  self.sock = httplib.FakeSocket(self.sock, ssl)
bzr: ERROR: exceptions.AttributeError: sendall

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 839, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/site-packages/bzrlib/builtins.py", line 953, in run
    from_location)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 888, in open_tree_or_branch
    bzrdir = klass.open(location)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 766, in open
    return BzrDir.open_from_transport(t, _unsupported=_unsupported)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 803, in open_from_transport
    redirected)
  File "/usr/lib/python2.6/site-packages/bzrlib/lazy_import.py", line 125, in __call__
    return obj(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/__init__.py", line 1634, in do_catching_redirections
    return action(transport)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 780, in find_format
    transport, _server_formats=_server_formats)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 1634, in find_format
    return format.probe_transport(transport)
  File "/usr/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 2617, in probe_transport
    server_version = medium.protocol_version()
  File "/usr/lib/python2.6/site-packages/bzrlib/smart/medium.py", line 593, in protocol_version
    client_protocol.query_version()
  File "/usr/lib/python2.6/site-packages/bzrlib/smart/protocol.py", line 769, in query_version
    self.call('hello')
  File "/usr/lib/python2.6/site-packages/bzrlib/smart/protocol.py", line 617, in call
    self._request.finished_writing()
  File "/usr/lib/python2.6/site-packages/bzrlib/smart/medium.py", line 421, in finished_writing
    self._finished_writing()
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/__init__.py", line 575, in _finished_writing
    data = self._medium.send_http_smart_request(self._buffer)
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/__init__.py", line 552, in send_http_smart_request
    code, body_filelike = t._post(bytes)
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib.py", line 138, in _post
    accepted_errors=[200, 403]))
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib.py", line 78, in _perform
    response = self._opener.open(request)
  File "/usr/lib/python2.6/urllib2.py", line 383, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 401, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 361, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib2_wrappers.py", line 634, in https_open
    return self.do_open(HTTPSConnection, request)
  File "/usr/lib/python2.6/site-packages/bzrlib/transport/http/_urllib2_wrappers.py", line 528, in do_open
    headers)
  File "/usr/lib/python2.6/httplib.py", line 911, in _send_request
    self.endheaders()
  File "/usr/lib/python2.6/httplib.py", line 868, in endheaders
    self._send_output()
  File "/usr/lib/python2.6/httplib.py", line 740, in _send_output
    self.send(msg)
  File "/usr/lib/python2.6/httplib.py", line 719, in send
    self.sock.sendall(str)
AttributeError: sendall

bzr 1.10dev on python 2.6 (linux2)
arguments: ['/usr/bin/bzr', 'branch', 'https://launchpad.net/bzr-push-and-update', 'bzr_push_and_update']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bookmarks /home/gour/.bazaar/plugins/bookmarks [unknown]
  gnulog /home/gour/.bazaar/plugins/gnulog.py [unknown]
  gtk /usr/lib/python2.6/site-packages/bzrlib/plugins/gtk [0.94.0]
  interactive /home/gour/.bazaar/plugins/interactive [1.2.0]
  launchpad /usr/lib/python2.6/site-packages/bzrlib/plugins/launchpad [unknown]
  search /home/gour/.bazaar/plugins/search [1.7.0dev0]
  stats /home/gour/.bazaar/plugins/stats [unknown]
  upload /home/gour/.bazaar/plugins/upload [0.1.1]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

Sincerely,
Gour

[John A Meinel (jameinel)]

So it seems we have a compatibility issue with python2.6.

I don't see a function called "ssl.wrap_socket()" it python2.5 and early. There is socket.ssl(sock)

so perhaps they mean that instead of using:
ssl = socket.ssl(self.sock, self.key_file, self.cert_file)
self.sock = httplib.FakeSocket(self.sock, ssl)

we should be doing
self.sock = socket.ssl(self.sock, self.key_file, self.cert_file)

Anyway, it seems that not only did they deprecate FakeSocket, they also broke it by removing the 'sendall()' function.

[Vincent Ladeuil (vila)]

The most amazing is that the test suite didn't catch that :-/

Or may be my python-2.6 built from sources is not up-to-date (I branched from trunk, mau be some divergence occurred since then).

Gour, what linux are you using ?

[Saša Janiška (gour)]

Hi!

Archlinux (x86_64)....
[gour@gaura-nitai gour] python
Python 2.6 (r26:66714, Oct 27 2008, 22:48:57)
[GCC 4.3.2] on linux2

Sincerely,
Gour

[Vincent Ladeuil (vila)]

Bah, of course it can't be caught by the test suite, I have a branch to implement an https test server !

Time to work on it again.

[Vincent Ladeuil (vila)]

A fix is available in the associated branch.

As mentioned in the comments, this bug wasn't caught by the test suite because we lack an https test server.

An https test server needs a certificate, and we want to be able to generate certificates for test purposes, etc.

This is still a work in progress that I can't submit for inclusion in core without enough tests.

And as this bug demonstrated even a single test can't be written without an https server.

Anyway, lp:~vila/bzr/https-test-server from which the fix has been sitting for months (sorry :-/) has enough support to run all the tests usually using an http server against an https server.

So the available fix has been tested again after I merged from bzr.dev.

[Robert Collins (lifeless)]

On Wed, 2008-11-05 at 20:57 +0000, vila wrote:
>
> An https test server needs a certificate, and we want to be able to
> generate certificates for test purposes, etc.

Could we just bundle a test certificate that certifies 127.0.0.1 ?

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

[Vincent Ladeuil (vila)]

>>>>> "robert" == Robert Collins <email address hidden> writes:

    robert> On Wed, 2008-11-05 at 20:57 +0000, vila wrote:
    >>
    >> An https test server needs a certificate, and we want to be able to
    >> generate certificates for test purposes, etc.

    robert> Could we just bundle a test certificate that certifies 127.0.0.1 ?

Sure, I'd look into lp:~vila/bzr/https-test-server on how to
extract such a simplification (but the certificate needs to be
approved by an authority which itself needs a certificate :0)

I can go with a self-certified certificate too but that needs
more modifications on the client side :)

But since I found some other problems with osx/python-2.6 I'll
fix that first and try to submit a reduced patch for inclusion in
1.9 first (without automated tests).

Then I'd look harder at what can be merged for 1.10 (with tests).