The discussion on IRC about this bug's details started here:

https://irclogs.ubuntu.com/2020/11/11/%23ubuntu-release.html#t21:04


2020-11-11 22:03:59     rbalint vorlon, Laney, juliank, doko about 5-10% of load reduction would be the result of skipping never passing tests LP: #1903913 
2020-11-11 22:04:01     ubot5   Launchpad bug 1903913 in Auto Package Testing "Don't run tests that never pass (or don't run tests with force-badtest)" [Undecided,New] https://launchpad.net/bugs/1903913
2020-11-11 22:05:06     juliank rbalint: we have to try them eventually otherwise we won't notice once they start working
2020-11-11 22:05:48     rbalint juliank, please see the bug, also there is not much to loose not noticing that quickly
2020-11-11 22:06:47     juliank rbalint: force-badtest ignoring is ok IMO, though we should try running the tests at least once a month or so
2020-11-11 22:07:01     rbalint juliank, yes, i'm proposing that
2020-11-11 22:07:06     juliank Well if we run them at least once a month, we can ignore all always failed tests
2020-11-11 22:07:27     juliank Without having to rely on continuous baseline retesting
2020-11-11 22:07:32     rbalint juliank, yes
2020-11-11 22:07:56     juliank Just have britney only schedule the test for an always-failed if it had no results for a month, essentially
2020-11-11 22:08:03     rbalint retesting everything is ~1200h on amd64 and ~3000h on arm64
2020-11-11 22:08:20     rbalint juliank, yes, and having a separate queue for that
2020-11-11 22:08:31     juliank rbalint: not a separate queue, no
2020-11-11 22:08:36     rbalint juliank, why?
2020-11-11 22:08:44     rbalint then which queue
2020-11-11 22:08:48     juliank The separate queue for the baseline retest
2020-11-11 22:09:02     juliank But I'm talking about the normal tests
2020-11-11 22:09:09     juliank e.g. foo has always failed
2020-11-11 22:09:25     juliank Bar triggers foo gets scheduled if we haven't run foo tests in a month
2020-11-11 22:09:58     juliank This avoids having to deal with the complicated continuous baseline retesting bits and extra queues
2020-11-11 22:10:35     juliank As an intermediate solution until we have baseline retesting, at which point we don't need that :(
2020-11-11 22:10:39     juliank Um :)
2020-11-11 22:10:40     rbalint if this is easier to implement, i like that
2020-11-11 22:10:54     rbalint bileto can always skip those
2020-11-11 22:11:01     juliank  Depends on whether britney has the data for it
2020-11-11 22:11:26     rbalint juliank, imo the force-badtest data is good for that
2020-11-11 22:11:43     juliank That one is easy I guess
2020-11-11 22:12:06     rbalint do we have a plan? ;-)
2020-11-11 22:12:18     juliank But also doing that for any test that is always failed and not run in a month would be more effective.
2020-11-11 22:12:34     juliank The question here is whether britney knows when a test last ran
2020-11-11 22:12:51     juliank Or whether it only knows that the test always failed
2020-11-11 22:12:54     rbalint autopkgtest.db does not seem to have dates
2020-11-11 22:13:03     juliank I think it downloads the database from autopkgtesr
2020-11-11 22:13:09     juliank Hmm
2020-11-11 22:13:27     juliank Id expect there to be a timestamp given that we see dates in the histor
2020-11-11 22:14:22     rbalint juliank, hm, it is encoded in run_id
2020-11-11 22:14:50     rbalint so that would work :-)
2020-11-11 22:15:33     juliank Now write a merge proposal :)
2020-11-11 22:16:09     juliank Also I have to have another look at britney triggers
2020-11-11 22:16:22     rbalint i hopes someone more familiar with britney would do it ;-)
...