canonical-certification-server lacks the ability to de-select tests in a suite

[Jeff Lane ]

the console UI seems to have only one mode: run everything selected.

We need an easy way for testers (read non-developers) to be able to run arbitrary subsets of tests from the whitelists.

Use Case:
Test Engineer at Cisco runs the Server-Self-test suite.
It's determined that Cisco needs to rerun memory/check, virtualization/kvm_check_vm and network/multi_nic

I can't tell them "go to such and such directory and edit this file and delete the following 100 lines"

I'd rather not have customer engineers create files at all, but would rather simply say "Re-run and at the menu, deselect everything but the following tests"

[Sylvain Pineau (sylvain-pineau)]

I'm open to suggestions to handle test selection, checkbox-cli used a very old-fashion selection mode based on letters and (+- to expand/collapse IIRC). The urwid interface had a nice widget to select tests but we dropped it as it's an external dependency.
To select the suites I only used python curses bindings. But making a tree of tests in curses is a bit challenging but not impossible though).

[Zygmunt Krynicki (zyga)]

I think we could make a multi-selection list widget (or use urwid for
it, irrelevant) and just use that. The tree can always be flattened
and I don't think we need a really fancy widget for such basic use
case.

What I worry about is how the user will be confused after making one
selection and seeing plainbox change that selection after dependencies
are added. Ideally we would have tri-state options, selected,
unselected, force-selected, with clearly different rendering and
per-selection changes so that selecting one item will immediately
select other items it depends on.

Thanks
ZK

On Fri, Dec 13, 2013 at 10:26 AM, Sylvain Pineau
<email address hidden> wrote:
> I'm open to suggestions to handle test selection, checkbox-cli used a very old-fashion selection mode based on letters and (+- to expand/collapse IIRC). The urwid interface had a nice widget to select tests but we dropped it as it's an external dependency.
> To select the suites I only used python curses bindings. But making a tree of tests in curses is a bit challenging but not impossible though).
>
>
> ** Changed in: checkbox
> Importance: Undecided => High
>
> --
> You received this bug notification because you are a member of Checkbox
> Bug Wranglers, which is subscribed to checkbox.
> https://bugs.launchpad.net/bugs/1260523
>
> Title:
> canonical-certification-server lacks the ability to de-select tests in
> a suite
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/checkbox/+bug/1260523/+subscriptions