Comment 2 for bug 1903913

The discussion on IRC about this bug's details started here:

https://irclogs.ubuntu.com/2020/11/11/%23ubuntu-release.html#t21:04

2020-11-11 22:03:59 rbalint vorlon, Laney, juliank, doko about 5-10% of load reduction would be the result of skipping never passing tests LP: #1903913
2020-11-11 22:04:01 ubot5 Launchpad bug 1903913 in Auto Package Testing "Don't run tests that never pass (or don't run tests with force-badtest)" [Undecided,New] https://launchpad.net/bugs/1903913
2020-11-11 22:05:06 juliank rbalint: we have to try them eventually otherwise we won't notice once they start working
2020-11-11 22:05:48 rbalint juliank, please see the bug, also there is not much to loose not noticing that quickly
2020-11-11 22:06:47 juliank rbalint: force-badtest ignoring is ok IMO, though we should try running the tests at least once a month or so
2020-11-11 22:07:01 rbalint juliank, yes, i'm proposing that
2020-11-11 22:07:06 juliank Well if we run them at least once a month, we can ignore all always failed tests
2020-11-11 22:07:27 juliank Without having to rely on continuous baseline retesting
2020-11-11 22:07:32 rbalint juliank, yes
2020-11-11 22:07:56 juliank Just have britney only schedule the test for an always-failed if it had no results for a month, essentially
2020-11-11 22:08:03 rbalint retesting everything is ~1200h on amd64 and ~3000h on arm64
2020-11-11 22:08:20 rbalint juliank, yes, and having a separate queue for that
2020-11-11 22:08:31 juliank rbalint: not a separate queue, no
2020-11-11 22:08:36 rbalint juliank, why?
2020-11-11 22:08:44 rbalint then which queue
2020-11-11 22:08:48 juliank The separate queue for the baseline retest
2020-11-11 22:09:02 juliank But I'm talking about the normal tests
2020-11-11 22:09:09 juliank e.g. foo has always failed
2020-11-11 22:09:25 juliank Bar triggers foo gets scheduled if we haven't run foo tests in a month
2020-11-11 22:09:58 juliank This avoids having to deal with the complicated continuous baseline retesting bits and extra queues
2020-11-11 22:10:35 juliank As an intermediate solution until we have baseline retesting, at which point we don't need that :(
2020-11-11 22:10:39 juliank Um :)
2020-11-11 22:10:40 rbalint if this is easier to implement, i like that
2020-11-11 22:10:54 rbalint bileto can always skip those
2020-11-11 22:11:01 juliank Depends on whether britney has the data for it
2020-11-11 22:11:26 rbalint juliank, imo the force-badtest data is good for that
2020-11-11 22:11:43 juliank That one is easy I guess
2020-11-11 22:12:06 rbalint do we have a plan? ;-)
2020-11-11 22:12:18 juliank But also doing that for any test that is always failed and not run in a month would be more effective.
2020-11-11 22:12:34 juliank The question here is whether britney knows when a test last ran
2020-11-11 22:12:51 juliank Or whether it only knows that the test always failed
2020-11-11 22:12:54 rbalint autopkgtest.db does not seem to have dates
2020-11-11 22:13:03 juliank I think it downloads the database from autopkgtesr
2020-11-11 22:13:09 juliank Hmm
2020-11-11 22:13:27 juliank Id expect there to be a timestamp given that we see dates in the histor
2020-11-11 22:14:22 rbalint juliank, hm, it is encoded in run_id
2020-11-11 22:14:50 rbalint so that would work :-)
2020-11-11 22:15:33 juliank Now write a merge proposal :)
2020-11-11 22:16:09 juliank Also I have to have another look at britney triggers
2020-11-11 22:16:22 rbalint i hopes someone more familiar with britney would do it ;-)
...