No way to mark a test as 'accepted regression'

[Iain Lane]

Let's say we know a package has regressed on an arch, and it's not going to get better and the release team has accepted this. They'd force-badtest the package so it migrates despite the failing test.

This hint would either have a version and have to be bumped every time, or it would have 'all' as a version.

In the 'all' case, if the test *does* get fixed, we would ignore any subsequent regressions whereas we'd probably want to at least consider it.

Proposal: a new hint type that forgets pass results before the specified version: force-alwaysfail foo/version[/arch].

[Steve Langasek (vorlon)]

Cross-linking to bug #1700668; not marking either as a duplicate until we decide which approach we want to take.

I argue in that bug that requiring setting of that hint is manual busywork for the release team, and that we should instead change our definition of a "regressed" autopkgtest.

[Iain Lane (laney)]

How do you think that the baseline reset should be effected?

[Iain Lane (laney)]

Ok, I've read that bug again and I think it's a drastic reduction in our testing commitment that needs more discussion; this proposal is for an uncontentious new feature.

[Iain Lane (laney)]

We just deployed this; a new hint "force-reset-test" is now available