file-failures doesn't scale with a large number of build failures

[Matthias Klose]

the script currently builds up a complete list of packages to examine before actually starting to do anything. this doesn't scale well with a large number of build failures (or think about scanning all successful build logs). so it would be better to apply *all* the logic for one build record completely before examining the next one, i.e:

  for b in getBuildRecords (FAILED_BUILDS)
    - check if superseded
    - check if built on reference arch
    - check if bug already filed
    - file bug

[Mattias Backman (mabac)]

Hi,

You're right about that. I'll work on rearranging this as soon as some ongoing basic changes are done.

There probably are a few more possible improvements now that I know more about how you're actually using the script. For instance now all passed archives are treated equal, all are checked for failed builds and cross referenced for later successful builds. I thought this was clever but the benefits might be purely academic while there is a real performance penalty.

A less generic solution would be to be explicit about which archive serves which purpose like
    - target-archive - assumed to contain only builds for target-architecture and is scanned only for failed builds. Only file bugs on packages found to fail to build in here.
    - reference-archive - assumed to only contain builds for reference-architectures and is scanned only for failed builds
    - main-archive - scanned only for successful builds for interesting packages on target-architecture

Perhaps that's closer to the actual use case and also would make using the script more obvious?

What you would loose is the possibility to run the script on only a mixed archive and let it use itself as reference, but that's a use case I just assumed and might not be interesting?

Thanks,

Mattias

[Mattias Backman (mabac)]

Performance issue.