updown OAuth fails with valid credentials due to delayed timestamp verification

[Roman Yepishev]

OAuth headers used to check the validity of the request contain the timestamp of the request to prevent reply attacks. It is working well (unless the client's time is completely off, but we are working around that in Ubuntu, Windows and Android clients). Usually all requests take less than 15 minutes (the default for oauth in updown).

Now client initiates PUT request which (depending on the client's bandwidth) may take much more than 15 minutes. By the time request is consumed by the server and gets processed, the timestamp is already off and request is discarded with OAuth error (which at the moment is handled incorrectly by redirecting to a html login page)

This is a cause of bug 879342 in deja-dup.

For deja-dup the speed needs to be at least 217kbps for 25Mb volume to complete uploading within 15 minutes.

[Roman Yepishev (rye)]

The reason for this behavior was found (thanks karni!).

OAuth headers used to check the validity of the request contain the timestamp of the request to prevent reply attacks. It is working well (unless the client's time is completely off, but we are working around that in Ubuntu, Windows and Android clients). Usually all requests take less than 5 minutes (the default for oauth).

Now client initiates PUT request which (depending on the client's bandwidth) may take much more than 5 minutes. By the time request is consumed by the server and gets processed, the timestamp is already off and request is discarded with OAuth error (which at the moment is handled incorrectly by redirecting to a html login page)

[Vincenzo Di Somma (vds)]

For what I can see, we don't store the nonces, once the oauth lib asks us if we have a nonce U1BaseOAuthDataStore.lookup_nonce always returns None.
So we are not using this security feature of OAuth.
There are two possible ways to solve the issue we are having:

1) Increase the timestamp_threshold in web.oauth.views.get_oauth_server to something bigger than 900 (ie 7200), if some upload will take more than 7200 secs the problem will still occur.

2) monkey patch python-oauth so that the nonce/timestamp check is always passed, is a small patch. If, in the future, we'll want to do a proper nonce/timestamp check we can always remove the monkey patch and go for option 1)

[Vincenzo Di Somma (vds)]

While my comments above are still valid, I'm not so sure the reason for the redirect is the timestamp threshold. All the oauth checks are done when the request is started and before reading the content of the PUT.

[Michał Karnicki (karni)]

I had two issues when uploading large files. Either it was socket timeout exception, or the request has completed (sent all the data), but the response was HTTP 302. Although it's not an elegant way to test it, you should be able to reproduce by uploading a large file (say, 40MB or 80MB) and check the response code. It be best if you could throttle that upload, so it actually takes time (it could equally be small file on a terribly slow network connection). OAuth authorization header is correct, still instead of HTTP 200, you get a HTTP 302 with a redirect to the login page, after a lengthy upload.

[Roman Yepishev (rye)]

Reproduced with trickle on 10KB/s with 12Mb openfire package:

ubuntu@lab:~/ubuntuone-scripts$ trickle -u 10 curl -v -T ../openfire_3.7.1_all.deb `./oauth-sign -m PUT https://files.one.ubuntu.com/content/~/Ubuntu%20One/Test.deb -f url`
trickle: Could not reach trickled, working independently: No such file or directory
* About to connect() to files.one.ubuntu.com port 443 (#0)
* Trying 174.129.230.205... connected
* Connected to files.one.ubuntu.com (174.129.230.205) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* subject: O=files.one.ubuntu.com; OU=Domain Control Validated; CN=files.one.ubuntu.com
* start date: 2011-10-25 21:35:27 GMT
* expire date: 2012-10-27 01:38:03 GMT
* subjectAltName: files.one.ubuntu.com matched
* issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certificates.godaddy.com/repository; CN=Go Daddy Secure Certification Authority; serialNumber=07969287
* SSL certificate verify ok.
> PUT /content/~/Ubuntu%20One/Test.deb?oauth_nonce=04417916&oauth_timestamp=1336043735&oauth_consumer_key=HI MOM!&oauth_signature_method=HMAC-SHA1&oauth_version=1.0&oauth_token=REDACTED&oauth_signature=REDACTED HTTP/1.1
> User-Agent: curl/7.21.6 (i686-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: files.one.ubuntu.com
> Accept: */*
> Content-Length: 12460336
> Expect: 100-continue
>
* Done waiting for 100-continue
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 302 FOUND
< Date: Thu, 03 May 2012 11:35:46 GMT
< Server: TwistedWeb/10.0.0
< Vary: Cookie,Accept-Encoding
< Etag: "d41d8cd98f00b204e9800998ecf8427e"
< X-Bzr-Revision-Number: 5413
< Location: https://files.one.ubuntu.com/auth/login/?next=/content/%7E/Ubuntu%20One/Test.deb%3Foauth_nonce%3D04417916%26oauth_timestamp%3D1336043735%26oauth_consumer_key%HELLO%26oauth_signature_method%3DHMAC-SHA1%26oauth_version%3D1.0%26oauth_token%REDACTED%26oauth_signature%REDACTED%253D
< Content-Type: text/html; charset=utf-8
< Via: 1.1 ip-10-84-67-0.ec2.internal
< Content-Length: 0
<
* Connection #0 to host files.one.ubuntu.com left intact
* Closing connection #0

[Sidnei da Silva (sidnei)]

@Rye: would be awesome if you could try the changes in the attached branch. There's some XXX that need fixing before we actually land this, but should be enough for you to test.

[Roman Yepishev (rye)]

This needs some more work because at the moment the OAuth check fails with invalid base. We need to re-create original URL properly. I will find the time for this next week.