Please add filtering on test results
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| ppa-dev-tools |
Triaged
|
High
|
Bryce Harrington | ||
Bug Description
./lp-test-ppa had - admittedly a maybe overloaded fashion of - filter functions.
It did have:
--showpass Show passing subtests (overall status is always shown)
--showskip Show skipped subtests (overall status is always shown)
-l, --last Show only the last test result per test
In PPAs with many builds or packages with many subtests the output can be too long and unreadable.
I do not insist on the semantic of --show... but some way to filter certain result types one is not interested in would be nice.
Even more so I think --last was important.
Common case as an example, all good but one was flaky so you re-ran it.
Sometimes this take multiple approaches...
Now quite likely you are not interested in the first 5 rounds of this, so you want only the latest result. --last provided that and AFAICS is missing in the newer tool set.
Could you consider bringing it back or is there a conceptual blocker against that functionality?
| Bryce Harrington (bryce) wrote | #1 |
| Changed in ppa-dev-tools: | |
| assignee: | nobody → Bryce Harrington (bryce) |
| importance: | Undecided → High |
| status: | New → Triaged |
Indeed, this is on the todo list, and in fact these args are all included in the spec.
The blocker for my progress here is indeed what you've mentioned - the semantics of the --show* options feel not quite right to me, and even though they're in the spec I have wanted to put some thought into if that could be done better. Unfortunately good ideas have eluded me so far.
I have a few other random tools where I've implemented nifty output filtering strategies, that I am thinking I'd like to steal design ideas from. Specifically, `migration-filter` from Ubuntu Helpers has filter_*() routines that can be nested together into a pipeline, which I really like from a code perspective but it's an internal implementation rather than exposed to the user to control. `parse-build-log` (also in UH) has an ignorable() filter that takes regular expressions, but isn't very sophisticated beyond that. `pkg-mp` (part of pinot/pdbq used for listing MP reviews), has a filter system similar to migration-filter but that is better exposed to the user .
At least internally I plan to implement this filter pipeline approach. But I'm unsure yet if I should expose that to the user via something like "--filter last", "--filter showpass,showskip" that would let them customize what's shown, or to make the filters be internal details and just have --showpass, --showskip, --last-only, etc. as the interface.
An argument in favor of doing the more exposed --filter approach is that I could use that as well for all the other ppa commands. `ppa list <ppa>` will need a way to filter what packages are listed, `ppa builds` will need filters although the names would differ from `ppa tests`.
Anyway, I'm still cogitating on this, but that sums up my current set of thoughts. I'd welcome more feedback in particular examples of what varieties of filters beyond lp-test-ppa's filters might be of utility.