pycurl and bzr-svn breaks branching from https://launchpad.net/bzr with http 405 error

[Björn Tillenius]

When I have both python-pycurl and bzr-svn installed, I can't create a branch from https://launchpad.net/bzr. Uninstalling either python-pycurl or bzr-svn makes it possible to branch from such a URL:

ixia:/devel/tmp> bzr branch https://launchpad.net/bzr
bzr: ERROR: Invalid http response for https://launchpad.net/bzr/.bzr/branch-format: Unable to handle http code 405: expected 200 or 404 for full response.:

This is a problem since goinstall (to install Go packages) uses such URL, so it's not possible to easily install packages.

Installed versions:
  bzr: 2.4.0~bzr6024~ppa3954.3938~natty1
  bzr-svn: 1.1.0~bzr3759~ppa389~natty1
  python-pycurl: 7.19.0-3build1

Traceback:

Mon 2011-07-18 09:39:45 +0300
0.057 bazaar version: 2.5.0dev1
0.057 bzr arguments: [u'branch', u'https://launchpad.net/bzr']
0.087 looking for plugins in /home/bjorn/.bazaar/plugins
0.126 Unable to load plugin 'lpmerge'. It requested API version (2, 3, 0) of module <module 'bzrlib' from '/usr/lib/python2.7/dist-packages/bzrlib/__init__.pyc'> but the minimum exported version is (2, 4, 0), and the maximum is (2, 5, 0)
0.127 looking for plugins in /usr/lib/python2.7/dist-packages/bzrlib/plugins
0.142 encoding stdout as sys.stdout encoding 'UTF-8'
1.180 Transferred: 3kB (5.5kB/s r:2kB w:1kB)
1.186 Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 946, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 1150, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 699, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 721, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/builtins.py", line 1263, in run
    from_location)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 918, in open_tree_or_branch
    bzrdir = klass.open(location)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 828, in open
    return BzrDir.open_from_transport(t, _unsupported=_unsupported)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 858, in open_from_transport
    redirected)
  File "/usr/lib/python2.7/dist-packages/bzrlib/lazy_import.py", line 129, in __call__
    return obj(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/__init__.py", line 1663, in do_catching_redirections
    return action(transport)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 845, in find_format
    transport, _server_formats=_server_formats)
  File "/usr/lib/python2.7/dist-packages/bzrlib/controldir.py", line 749, in find_format
    return prober.probe_transport(transport)
  File "/usr/lib/python2.7/dist-packages/bzrlib/bzrdir.py", line 1344, in probe_transport
    format_string = transport.get_bytes(".bzr/branch-format")
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/__init__.py", line 606, in get_bytes
    f = self.get(relpath)
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/http/__init__.py", line 126, in get
    code, response_file = self._get(relpath, None)
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/http/_pycurl.py", line 171, in _get
    return self._get_full(relpath)
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/http/_pycurl.py", line 215, in _get_full
    curl, 'expected 200 or 404 for full response.')
  File "/usr/lib/python2.7/dist-packages/bzrlib/transport/http/_pycurl.py", line 327, in _raise_curl_http_error
    % (code, msg, plaintext_body))
InvalidHttpResponse: Invalid http response for https://launchpad.net/bzr/.bzr/branch-format: Unable to handle http code 405: expected 200 or 404 for full response.:

1.186 return code 3

[Björn Tillenius (bjornt)]

Not sure whether this is a bug in bzr or bzr-svn (or pycurl maybe), but the problem is that bzr-svn sets pycurl.CUSTOMREQUEST and nothing unsets it before doing the next GET request. For example, the following patch solves the problem.

=== modified file '__init__.py'
--- __init__.py 2011-06-12 20:30:12 +0000
+++ __init__.py 2011-07-18 07:12:44 +0000
@@ -206,6 +206,7 @@
                 conn.setopt(pycurl.WRITEFUNCTION, data.write)
                 transport._curl_perform(conn, header)
                 code = conn.getinfo(pycurl.HTTP_CODE)
+ conn.unsetopt(pycurl.CUSTOMREQUEST)
                 if code == 404:
                     raise NoSuchFile(transport._path)
                 if code in (403, 405):

[Jelmer Vernooij (jelmer)]

bzr-hg uses a similar trick

[Jelmer Vernooij (jelmer)]

The problem appears to be that a curl http transport is "tainted" after it's returned a InvalidHttpResponse once. After that single invalid response it will continue raising that error.