OpenID sessions accumulate in the session forever

[William Grant]

There exists on production a session with more than 100MB of data, and a great many with more than 1MB. It seems that each OpenID request adds its data to the session, and it's never removed, eg. in _process_openid_request and _handle_user_response.

We should probably move the OpenID bits of the session into a subdict, store a timestamp alongside each one, and expire old (>24h?) ones when we add new ones.

[Daniel Manrique (roadmr)]

I've checked empirically with a local SSO and local openid consumer, and I see this happening only when the consumer is an "untrusted" one (i.e. not one we have an openid rp configuration item for). We could check the logs in production SSO to try to understand which unknown peer (which I think might be in the referrer) is using us to authenticate. This is totally allowed by how openid works and doesn't contradict the solution William proposed.

[Daniel Manrique (roadmr)]

Top 12 openid peers hiting /decide endpoints:

https://pastebin.canonical.com/p/wgCszRdYn4/

interestingly login.launchpad.net and bugs.launchpad net are not in SSO's list of trusted OpenID RPs (but I imagine given their special relationship that might be OK?).

review.openstack.org is also not in the list.
Finally, of the top 12, summit.linuxplumbersconf.org which is kind of suspicious since that site is for a 2013 conference apparently and is not https or something, so I wonder why we're getting auth requests from it. I checked and sure enough, the "login" thing hits Launchpad.

If we find an offending session with zillions of tokens we could learn more by either dissecting session data or correlating the tokens with the sso web logs.

[Daniel Manrique (roadmr)]

Here's a proposal that keeps the openid stuff (which is stored in the session dict under a "token" key; the storage of many data items and thus many tokens is what seems to be ballooning the sessions) where it is, and stores each token's creation timestamp separately within the session. The benefit is that it's more backwards-compatible with existing sessions.

https://code.launchpad.net/~roadmr/canonical-identity-provider/openid-token-reaper/+merge/349080

I still need to add true backwards compatibility to add an expiration date to existing tokens (currently this works only for newly-added ones) and tests obviously, but wanted to show the general approach to ensure I'm not being silly.

[Maximiliano Bertacchini (maxiberta)]

New openid tokens now have a 24h TTL in production. Old tokens will also be dropped 24h after first seen in an openid dance (/+openid). As this has just been deployed, the effects be visible starting tomorrow ~noon.