requestMirror timing out

[Michael Hudson-Doyle]

For example:

https://devpad.canonical.com/~matsubara/oops.cgi/2008-09-19/XMLP10

And probably the same issue: https://answers.edge.launchpad.net/launchpad-bazaar/+question/45717.

[Michael Hudson-Doyle (mwhudson)]

Seems to be deep database stuff:

<mwhudson> lifeless: so the problem we have now and then with branches is that 'update branch set next_mirror_time = NOW where id = %d' times out
<mwhudson> lifeless: my guess is that this could be because the scanner has a transaction open adding branchrevision rows that reference this branch
<mwhudson> does that make sense?
<lifeless> mwhudson: yes, sounds likely
<mwhudson> and if the fkey wasn't on delete cascade this wouldn't happen?
<lifeless> mwhudson: yes
<mwhudson> ok
<mwhudson> very interesting :)

[Stuart Bishop (stub)]

Inserts into the BranchRevision table do block updates on the referenced Branch rows though.

I can't reproduce the ON DELETE behavior though. I don't think ON DELETE CASCADE makes any difference to locking behavior with PostgreSQL.

[Stuart Bishop (stub)]

Changing the foreign key constraint to deferred could be an easy fix so the lock is only taken when attempting to commit. This will cause the BranchRevision inserting transaction to block until the Branch updating transaction has committed instead (reversed).

[Michael Hudson-Doyle (mwhudson)]

A deferred constraint sounds entirely appropriate here. Do you want me to write a database patch? It seems kinda pointless as I'll have to get you to tell me what to type :)

[Stuart Bishop (stub)]

Yup. I'll handle it. We won't see this solution until the next production rollout - is that ok?

(Well... we could cowboy it in if it is a problem we need to address now, although it might require some bits being shutdown and possibly some timeouts).

[Michael Hudson-Doyle (mwhudson)]

If cowboying is a complex process, I think waiting until the rollout is OK.

If people start hitting this more often, we might have to think again.