[meta] Split remaining DB dependencies from LP

[Stuart Metcalfe]

As part of https://dev.launchpad.net/LEP/OpenIdRoadmap

We currently replicate several tables from LP to support our teams implementation. We should be able to use the LP API for this. A few considerations:

 * Some sites depend on the high availability of teams information and would be intolerant of LP downtime.
 * LP doesn't expose an immutable user identifier so would things get a bit sticky if the user changed their email *and* username?
 * Using the LP API could incur longer load times on SSO.
 * We'd need a user-friendly and unobtrusive method of attaching your LP account to your SSO account.

Tag child bugs: proj-splitit-final

See also
========

Bug 629169

[Robert Collins (lifeless)]

On Tue, Sep 7, 2010 at 5:34 AM, Stuart Metcalfe
<email address hidden> wrote:
> Public bug reported:
>
> As part of https://dev.launchpad.net/LEP/OpenIdRoadmap
>
> We currently replicate several tables from LP to support our teams
> implementation.  We should be able to use the LP API for this.  A few
> considerations:
>
>  * Some sites depend on the high availability of teams information and would be intolerant of LP downtime.

We're working on downtime - I consider reducing / eliminating downtime
a critical component in reducing the cost of launchpad development. In
the short term DB schema changes will still require downtime but we
hope to eliminate all other causes of downtime by christmas.

>  * LP doesn't expose an immutable user identifier so would things get a bit sticky if the user changed their email *and* username?

We should be able to do what SSO did for this - please file a new bug?

>  * Using the LP API could incur longer load times on SSO.

Thats true; I am thinking about organising a dedicated APIs server
cluster (just partitioning off some of the appservers for APIs) so
that APIs are not affected (positively or negatively) by load spikes
on the web cluster. This might help mitigate SSO concerns. We could,
of course, do a dedicated FE cluster for SSO but I don't think that
that would be needed. We're also driving our response times down very
successfully at the moment, but we can and should put the specific
API's you need into a nagio alert.

>  * We'd need a user-friendly and unobtrusive method of attaching your LP account to your SSO account.

Definitely; worthy of a new bug perhaps? We should prepopulate this as
part of the migration, no need for existing users to have to take
manual action.

-Rob

[Stuart Bishop (stub)]

Given we cannot guarantee 100% Launchpad availability, or even 100% Launchpad availability when the SSO is available, we may find that replication of information is the best solution either via database tables and Slony or some other mechanism. Point 4 on the LEP is because the replicated information the SSO sees at the moment is incorrect - as of the upcoming rollout Launchpad supports multiple OpenId Identifiers linked to an Account, but is faking this in the data the SSO sees with one identifier per Account. This might cause some edge cases where if people have multiple accounts in the SSO, only the one they most recently used to log into Launchpad is linked to their Launchpad information.

[Stuart Metcalfe (stuartmetcalfe)]

On Mon, 2010-09-06 at 23:14 +0000, Robert Collins wrote:
> On Tue, Sep 7, 2010 at 5:34 AM, Stuart Metcalfe
> > * LP doesn't expose an immutable user identifier so would things get
> a bit sticky if the user changed their email *and* username?
>
> We should be able to do what SSO did for this - please file a new bug?

I've filed bug #655565 for this.

[Robert Collins (lifeless)]

This is a blocker for moving away from slony, stopping impacting on SSO etc.

Current thinking is that we definitely want an API server for team lookups, but as part of LP refactoring we may be moving to a separate team server *anyway* - so we could bring that up as a microservice, delegate through it to LP itself, and do our data model splitting for LP as a separate project.

Long term I think we should stop issuing team info through SSO, because its really a separate dimension.

Regardless, the API we bring up needs some though on what keys to look up via (e.g. openid url, something special, etc etc).