status performance regression compared to 0.17?

[Martin Pool]

http://benchmark.bazaar-vcs.org/usertest/usertest.log

shows that status in 0.91dev and 0.18 is approximately 3x slowerthan it was in 0.17:

2.99 30.466 30.289 10.202 script_common.AddTask:status

[John A Meinel (jameinel)]

I'm very curious what is going on here.

I did some personal testing and my times are:

~/dev/bzr/releases/bzr.0.17/bzr --no-plugins status 4.56s user 1.00s system 99% cpu 5.566 total
~/dev/bzr/releases/bzr.0.18/bzr --no-plugins status 4.56s user 1.08s system 99% cpu 5.644 total
~/dev/bzr/releases/bzr.0.90/bzr --no-plugins status 3.98s user 1.01s system 99% cpu 5.023 total
~/dev/bzr/bzr.dev/bzr --no-plugins status 3.91s user 1.04s system 99% cpu 4.983 total

Which shows bzr.dev to be about 10% faster than 0.17, not 3x slower.

As near as I can tell, that 10% comes from the pyrex implementation of the dirstate parser. (Since that landed in 0.90 and it nicely coincides with that release).

So we should certainly figure out why it seems that on some machines it is performing significantly worse rather than better.

[John A Meinel (jameinel)]

You know 1 think it could be... If your LANG value isn't set to .UTF-8

So if you have "LANG=en_US" rather than "LANG=en_US.UTF-8" then it will have to convert all paths to Unicode internally (and then encode them back into utf-8).

That could possibly count for a 3x performance hit.

[Andrew Bennetts (spiv)]

http://benchmark.bazaar-vcs.org/usertest/usertest.log looks quite stale to me. It says near the top:

+ echo usertest run starting
usertest run starting
+ date -u
Tue Aug 7 07:23:31 UTC 2007

So maybe this problem isn't current?

[Martin Pool (mbp)]

On 9/6/07, Andrew Bennetts <email address hidden> wrote:
> http://benchmark.bazaar-vcs.org/usertest/usertest.log looks quite stale
> to me. It says near the top:
>
> + echo usertest run starting
> usertest run starting
> + date -u
> Tue Aug 7 07:23:31 UTC 2007
>
> So maybe this problem isn't current?

You may be right; I've just restarted it.
--
Martin

[Martin Pool (mbp)]

The problem is still visible in the current bzr.dev. This job is
running with LANG=en_GB.UTF-8 -- John suggested it may be related to
changes that try to handle filename encoding better.

--
Martin

[Martin Pool (mbp)]

I've just tested this on my machine grace on the
gcc 4.1.0 tree, with LANG=en_AU.UTF-8

/home/mbp/bzr/bzr.0.17/bzr --no-plugins add -q 4.82s user 0.35s system 99% cpu 5.184 total
bzr status > /tmp/st2 2.54s user 0.34s system 99% cpu 2.882 total
/home/mbp/bzr/bzr.0.17/bzr --no-plugins ci -m 'initial commit' 99.69s user 6.54s system 85% cpu 2:04.80 total

/home/mbp/bzr/trunk/bzr --no-plugins add -q 5.20s user 0.31s system 99% cpu 5.511 total
/home/mbp/bzr/trunk/bzr --no-plugins status 2.33s user 0.36s system 67% cpu 3.989 total
/home/mbp/bzr/trunk/bzr --no-plugins status > /tmp/status.out 2.59s user 0.32s system 99% cpu 2.922 total
/home/mbp/bzr/trunk/bzr --no-plugins commit -q -m initial\ commit 74.60s user 5.52s system 87% cpu 1:31.09 total
~/bzr/trunk/bzr --no-plugins status > status.out 3.04s user 0.63s system 99% cpu 3.683 total
~/bzr/trunk/bzr --no-plugins status > status.out 1.34s user 0.36s system 100% cpu 1.698 total

So there's no clear indication yet...

[Martin Pool (mbp)]

When i tested by hand the sequence of

  init
  add -q
  commit -q
  mkdir zzz; touch zzz/zzfile
  add
  status

there was no significant difference between the performance of 0.17 and 0.91.

[Martin Pool (mbp)]

I note that just before doing the add/status part of the benchmark, it's done a non-repository branch of all the trees, which is going to read 300MB of repository data, write out 300MB of repository data, plus building a 300MB tree. That's 2700MB of data in transit in total altogether, so we're likely to still be under memory pressure when we start the next operation. I'm speculating that by the time we start the next one, we'll have more clean buffer pages.

I think we really need to change usertest to do all the operations on the first tool, then everything on the second, and so on. Then the memory and buffer usage generated by any run of bzr will mostly penalize its own operations. I'm not sure how to account for the followon cost from one to the next; possibly running 'sync' and waiting for a while.

[Martin Pool (mbp)]

I changed the cron job to run usertest repeatedly, so tests for the different versions of bzr are not overlapped. The status operation after add is now just about 10s. So I'm going to mark this closed. It might be good if usertest had builtin support for this, because the default ordering invites problems.