Make pgcreate robust against race conditions

[Francis J. Lacoste]

Our database initialization process sometimes fails because of race conditions. This race seems to happen more often in EC2:

Traceback (most recent call last):
  File "../../utilities/pgcreate.py", line 42, in ?
    sys.exit(main())
  File "../../utilities/pgcreate.py", line 27, in main
    cur.execute(
psycopg2.OperationalError: source database "launchpad_empty" is being accessed by other users

When this error happens, the script should sleep and retry a few times.

[Stuart Bishop (stub)]

I just checked pgcreate.py and we have already implemented this behavior. I don't know what is causing the ec2 failures we see occasionally.

[Stuart Bishop (stub)]

Some instrumentation and possible improvements is on lp:~stub/launchpad/pgmassacre, awaiting review. It increases the createdb timeout to 20 seconds, which should be plenty even if the ec2 box is loaded.

[Stuart Bishop (stub)]

lp:~stub/launchpad/trivial contains the fix, but I can't land it on lp:~launchpad-pqm/launchpad/devel due to the db check failing. The branch is safe as far as edge goes.

[testfix][r=jtv][ui=none][bugs=353865] Disable autovacuum on template databases to avoid conflicting with our build scripts

[Stuart Bishop (stub)]

Its been landed on lp:~launchpad-pqm/launchpad/db-devel so some spurious failures should stop, but still needs to land on lp:~launchpad-pqm/launchpad/devel