Buildmaster has performance issues

[Celso Providelo]

Currently buildmaster handles the queue entries in a very uneficient manner: huge fetches in BQ table, ordering and sorting in python code. It needs to be solved before Soyuz Rollout.

[Celso Providelo (cprov)]

It will become serious when buiding PPA.

[Christian Reis (kiko)]

What methods do this fetching? I could take a look at helping with this if you point out the code to me.

[Celso Providelo (cprov)]

Ok,

createMissingBuilds & addMissingBuildQueueEntries are the critical methods, nonetheless changes there will affect the entire module, for instance: scoreBuildQueueEntry, sortAndSplitByProcessor, dispatchByProcessor, etc

The way we reorganize the DB information in python domain (dicts) is the problem itself.

[Julian Edwards (julian-edwards)]

This is a very old bug. How much of it is still pertinent in regards to today's code base?

[Celso Providelo (cprov)]

The code steps seem to be fast enough in our current codebase. The main cause of poor performance is the fact that the builder-scanning is a serialized procedure. It check for builders processing jobs and deal with their results one at time, this clearly gets worse when we add more builders.

Ideally we should be able to parallelize the scanning procedure and then be able to collect results quickly.

Another major cause of the delays in are cause by waiting sockets to timeout (120s) in case of failures. This mechanism should be redesign.

[Julian Edwards (julian-edwards)]

RF 7848