SSL connection issues

[Michael Nelson]

We've started getting a bunch of oopses with tracebacks like

https://pastebin.canonical.com/49034/
https://pastebin.canonical.com/49036/
https://pastebin.canonical.com/49035/

21 for /reviews/api/1.0/reviews/3919/recommendations/ (Unknown) 2002rangpur39041, 2002rangpur39044, 2002rangpur39046, 2002rangpur39047, 2002rangpur39049 on 2011-06-25.

Not sure if they're related, but we can split them out into separate bugs when investigating.

[Michael Nelson (michael.nelson)]

mvo just pointed out bug 797281 - which might be worth checking out as a possible cause of the ssl-related errors (if it only happens with oneric clients).

[Michael Nelson (michael.nelson)]

Looking at the logs, I don't think this can be related to bug 797281 (as the errors are when our *server* is contacting SSO, and there have been no updates to lucid's lazr.restfulclient afaics).

Just summarising the various errors:
  * All occur while sca contacts sso using lazr.restfulclient calling sso.authentications.validate_token()
  * It seems it was only on 2011-06-25 that we experienced SSL issues (4 on rangpur and 2 on satsuma) like:
        https://pastebin.canonical.com/49100/ - decryption failed or bad record mac
        https://pastebin.canonical.com/49034/ - block cypher pad is wrong
        https://pastebin.canonical.com/49036/ - wrong version number
 * There were other errors which seem related but were not SSL errors on 2011-06-25:
        https://pastebin.canonical.com/49035/ - httplib Bad status line
 * We continue to have issues (perhaps unrelated) calling sso.accounts.me():
        https://pastebin.canonical.com/49096/ (on rangpur with the frequencies per day: 6 on 27th, 4 on 26th, 14 on 25th, 0 on 24th, 2 on 23rd, 0 on 22nd) Looking at the lazr.restfulclient code, we'd get such an exception for any non 2xx status code returned by SSO (I'll look into that separately and perhaps create a new bug).