http decorators incorrectly handled when authentication is used

[Vincent Ladeuil]

bzr use urlparse.urlsplit or urlparse.urlparse (which calls urlsplit) to extract scheme, host, path etc from the url

urlparse.urlsplit fails to recognize http+urllib or http+pycurl as valid schemes and returns an empty netloc (user + password + host + port).

As the netloc is prepended to the returned path and bzr concatenate both when needed, this has not been noticed until now.

While no authentication is used, there is no consequences, but failing to extract the user and password from the netloc causes problems when authentication is needed.

But again, this occurs only when a user needs to use an http implementation explicitly, if the user relies on the default implementation by using no decorators, all is fine.

It's arguably a python bug but we have to deal with it.

[John A Meinel (jameinel)]

I think a simple "bzrlib.transport.register_urlparse_netloc_protocol('http+urllib')" et al will suffice (you need +urllib and +pycurl as well as https).

And then some tests that "get_transport(https+pycurl://user@host...)" is properly parsed for the different variants.

[Vincent Ladeuil (vila)]

We alreday have the appropriate register_urlparse_netloc_protocol :

grep -nH -e register_urlparse_netloc_protocol -r .
./transport/__init__.py:153:def register_urlparse_netloc_protocol(protocol):
./transport/http/_pycurl.py:47:from bzrlib.transport import register_urlparse_netloc_protocol
./transport/http/_pycurl.py:78:register_urlparse_netloc_protocol('http+pycurl')
./transport/http/_urllib.py:23:from bzrlib.transport import register_urlparse_netloc_protocol
./transport/http/_urllib.py:35:register_urlparse_netloc_protocol('http+urllib')
./transport/remote.py:38: transport.register_urlparse_netloc_protocol(scheme)
./transport/sftp.py:54: register_urlparse_netloc_protocol,
./transport/sftp.py:73:register_urlparse_netloc_protocol('sftp')

Unfortunately it's not enough .

[Ian Clatworthy (ian-clatworthy)]

Following a review of some proposed changes to bzrdir.py, here are some associated comments on IRC:

(17:00:30) poolie_: it's about, say, if it was http+pycurl then we should preserve the 'pycurl' bit
(17:00:30) igc: when vila get's to it
(17:01:15) poolie_: hm
(17:01:28) poolie_: i think the main thing is that it ought to be a call to something like
(17:01:48) poolie_: target = urlutils.copy_url_qualifiers(original, target)
(17:01:57) poolie_: and then that can be tested separately
(17:02:08) poolie_: rather than doing string slicing inline here

The code in question (which was previously executed but the result was not used), has been removed and replaced with a comment.

[Vincent Ladeuil (vila)]

The above comment does not strictly apply to this bug (redirection instead of authentication), but both are linked to decorators handling.

The two problems *can* be addressed separately but that would be nice to solve them both in one shot.