HTTP Client mistakenly send unicode in request start line and headers

[Ugo Riboni]

To reproduce:

- using the ElisaHttpClient.request method, pass as uri or method parameter a python unicode object
- attach callback and errback and wait

What happens:

- either you get back a response with an http 408 (timed out) code after a while, or
- neither the callback nor the errback are ever called and you're stuck waiting forever

The issue is that we send down to the server the unicode objects as-is.
This is in violation of the HTTP specification (http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2) as the first line of the HTTP request should only contain US-ASCII characters.
Besides being in violation of the spec, the server will never recognize the CR+LF that should terminate the first line so it will keep waiting for it (or timeout with a 408, depening on the implementation).

[Olivier Tilloy (osomon)]

Note that the API documentation of the request method explicitely requires str objects, not unicode (http://www.moovida.com/documentation/api/elisa.plugins.http_client.http_client.ElisaHttpClient-class.html#request).

Note that there was also a failing unit test for the HTTP client that had been commented out until someone would take the time to understand the issue, which you did.

I'm not sure this should be considered as a bug, the documentation being clear about the expected value types.

[Olivier Tilloy (osomon)]

And that failing unit test is called "test_unicode_fails_for_request".

[Ugo Riboni (uriboni)]

Well, you have a point there, the docs say that we need str objects already. So yes, it is technically a misuse of the API.

However, with most other functions passing the wrong type in most cases will result in exceptions or in very identifiable bad behaviour that can be very easily traced back to a misuse in the API.

In this case the bad behaviour is confusing and hard to trace back to API misuse. It took me various hours to find out what was really going on, and I had to debug into the http client and use wireshark.

Also, sending unicode is a violation of the HTTP specification. In my opinion a good implementation of HTTP should never allow the user to do things that violate the specification. Therefore, the client should either do an implicit conversion like I implemented in my patch, or it should raise an exception early when the user try to pass something to the API that would result in a violation of the spec and not allow him to continue.

[Olivier Tilloy (osomon)]

Agreed.