BzrDirFormat.initialize_ex_1.16 on Launchpad wastes time making Twisted failures

[Martin Pool]

I'm just going to split out one aspect of bug 740759, which is that one cause of branch creation on Launchpad being slow seems to be that raising exceptions within twisted is unreasonably slow.

[John A Meinel (jameinel)]

Didn't you get this landed in Launchpad (as an update to twisted)

[John A Meinel (jameinel)]

It looks like you did land the first of your 2 patches, but are still missing the second revision, and my one follow up revision.
http://bazaar.launchpad.net/~jameinel/launchpad/faster-branch-vfs-740759/revision/12687
http://bazaar.launchpad.net/~jameinel/launchpad/faster-branch-vfs-740759/revision/12688

I'll try to put together a proposal for these, along with timing showing the improvement at least locally.

I just ran 10 'bzr push' to new branches on Launchpad, and this is the timing I see. Note that it doesn't show where the time is spent on the server.
1.764s
1.968s
2.125s
2.079s
4.733s
1.531s
1.352s
1.449s
1.910s
1.648s

So there can still be a lot of variability (peak of 4.7s here), the fastest time was 1.3s, the average is 2.0s.

[John A Meinel (jameinel)]

Robert mentions that this is probably not rolled out onto codehosting yet, so it still can establish a baseline for us to compare against later.

[Andrew Bennetts (spiv)]

John A Meinel wrote:
> It looks like you did land the first of your 2 patches, but are still missing the second revision, and my one follow up revision.
> http://bazaar.launchpad.net/~jameinel/launchpad/faster-branch-vfs-740759/revision/12687

Your measurements showed this had no significant effect, so I dropped it
from my branch. It doesn't seem worth adding such an ugly hack for no
measureable benefit.

[…]
> I just ran 10 'bzr push' to new branches on Launchpad, and this is the timing I see. Note that it doesn't show where the time is spent on the server.
> 1.764s
> 1.968s
> 2.125s
> 2.079s
> 4.733s
> 1.531s
> 1.352s
> 1.449s
> 1.910s
> 1.648s
>
> So there can still be a lot of variability (peak of 4.7s here), the
> fastest time was 1.3s, the average is 2.0s.

If the speed up is evident through all that variability we'll know we've
done a great job :)

[Andrew Bennetts (spiv)]

Although this issue isn't actually fixed yet, the work remaining to be done is being tracked in other bugs now:
 * bug 751200 for the workaround patch to lp:launchpad
 * Twisted's #5011 for the proper fix to Twisted itself.

When 5011 is fixed upstream we should apply that fix to Launchpad's version of Twisted too. But in the meantime there's nothing much more worth doing in this bug.

[Andrew Bennetts (spiv)]

John: bug 751200 is now marked as Fix Released. Could you try running those timings again?

[Robert Collins (lifeless)]

Pretty sure we've upgraded to latest twisted.