Deadlock in login tests

[Dave Cheney]

The session copying fix triggers a deadlock in apiserver tests (login test) > 50% of the time. Fixing the deadlock will solve the problem.

[Ian Booth (wallyworld)]

I found an unrelated deadlock in the login tests. It's very much timing related so it is plausible that underlying changes to state may lead to the deadlock occurring more frequently.

The problematic test is TestLoginRateLimited on loginSuite. This starts N login attempts, and then starts attempt N+1. The expectation in that N+1 will error out immediately with a try again error. But, all N+1 login attempts are racing via the rpc layer to reach the login on the server. And attempt N+1 may beat one of the N attempts to get there, thus one of the N attempts is the one to get the retry error. This stuff up the tests because attempt N+1 now blocks and the code to unblock the pending login attempts is never run.

[Andrew Wilkins (axwalk)]

I've marked the older lp:1323357 as a duplicate of this, as there's some more analysis and activity here.