Provide a way to disable updating results.cache

[Robert Bruce Park]

Many packages have autopkgtests that fail for various reasons, and one thing that can sometimes happen is during development, the code can change in a way that results in a false positive pass, eg, autopkgtest reports a success incorrectly. When the incorrect success is then corrected back to the expected failure, the silo build treats this as a regression, even though the autopkgtests never succeeded in distro.

This is incorrect behavior, "regression" should only be reported for tests that regress against distro, not for tests that regress within a silo.

Please tweak results.cache so that it never contains silo results, only distro results.

[Martin Pitt (pitti)]

So I think what you want (and we talked about that on IRC) is some way to disable updating results.cache. Then every run will re-download the latest result from the PPA and compare it to the status quo in Ubuntu. Then you could also share results.cache between all silo instances.

We could make this implicit by not failing if results.cache is not writable (it might be a symlink to a read-only file, or a file owned by a different user), or explicit with a new ADT_RESULTS_UPDATE=no option. Do you have any preference?

[Robert Bruce Park (robru)]

I think explicit would be better, but what do you think of having a key
like SHARED_RESULTS_CACHE_PATH that provides a path to the shared cache
file? If blank use the current behavior, if set, switch to non-updating
mode.

This would be consistent with all the other path-specifying options and
would save me having to do symlink trickery.

Thanks.
On Jan 26, 2016 5:45 AM, "Martin Pitt" <email address hidden> wrote:

> So I think what you want (and we talked about that on IRC) is some way
> to disable updating results.cache. Then every run will re-download the
> latest result from the PPA and compare it to the status quo in Ubuntu.
> Then you could also share results.cache between all silo instances.
>
> We could make this implicit by not failing if results.cache is not
> writable (it might be a symlink to a read-only file, or a file owned by
> a different user), or explicit with a new ADT_RESULTS_UPDATE=no option.
> Do you have any preference?
>
> ** Changed in: britney
> Status: New => Incomplete
>
> ** Summary changed:
>
> - Silos should not store any autopkgtest history.
> + Provide a way to diable updating results.cache
>
> ** Summary changed:
>
> - Provide a way to diable updating results.cache
> + Provide a way to disable updating results.cache
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1537868
>
> Title:
> Provide a way to disable updating results.cache
>
> Status in Bileto:
> New
> Status in britney:
> Incomplete
>
> Bug description:
> Many packages have autopkgtests that fail for various reasons, and one
> thing that can sometimes happen is during development, the code can
> change in a way that results in a false positive pass, eg, autopkgtest
> reports a success incorrectly. When the incorrect success is then
> corrected back to the expected failure, the silo build treats this as
> a regression, even though the autopkgtests never succeeded in distro.
>
> This is incorrect behavior, "regression" should only be reported for
> tests that regress against distro, not for tests that regress within a
> silo.
>
> Please tweak results.cache so that it never contains silo results,
> only distro results.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/bileto/+bug/1537868/+subscriptions
>

[Martin Pitt (pitti)]

Roger, let's do that then.

[Martin Pitt (pitti)]

Committed in http://bazaar.launchpad.net/~ubuntu-release/britney/britney2-ubuntu/revision/567 .

The new option is ADT_SHARED_RESULTS_CACHE, added to britney.conf with a comment. Point that to the downloaded results.cache and it won't be touched.