Tests with invalid category attributes should be more visible

Bug #881617 reported by Charlie Poole
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
NUnit V2
Fix Released
Medium
Charlie Poole
nunit-console
Won't Fix
High
Unassigned

Bug Description

When a test is marked with an invalid category attribute, like aaa-bbb, that test is marked as not-runnable and shows up as an error in both the gui and console if we try to run it. However, if we make the mistake of asking nunit to run category aaa-bbb, then the test in question is not run (because aaa-bbb is a valid category expression) and no error appears. From the point of view of the user, the test seems to have just disappeared.

The following changes are needed, at a minimum, to make it clearer to the user just what is happening:

1. Not runnable tests should show as red in the gui as soon as the tests are loaded, without waiting for a test run. They should remain red even after a test run that does not include them.

2. Invalid categories should not be listed for selection by the gui at all.

3. The console runner output should use wording that indicates aaa-bbb is a category expression rather than a simple category.

See bug #881357 for more info on the motivation for this change.

Tags: console gui

Related branches

Changed in nunitv2:
importance: Undecided → Medium
assignee: nobody → Charlie Poole (charlie.poole)
status: New → In Progress
milestone: none → 2.6.0
Revision history for this message
Charlie Poole (charlie.poole) wrote :

Fixed this by implementing the three points above. Also made ignored tests show up as yellow in the gui before they are run.

Changed in nunitv2:
status: In Progress → Fix Committed
Revision history for this message
Charlie Poole (charlie.poole) wrote :

Re-opening this bug to deal with the following problem:

The yellow/red coloring of the test is lost when selecting and running a set of tests that does not include the test in question.

Changed in nunitv2:
status: Fix Committed → In Progress
Changed in nunitv2:
status: In Progress → Fix Committed
Changed in nunitv2:
status: Fix Committed → Fix Released
Changed in nunit-3.0:
status: New → Opinion
status: Opinion → Triaged
importance: Undecided → High
assignee: nobody → Charlie Poole (charlie.poole)
milestone: none → 2.9.6
Revision history for this message
Charlie Poole (charlie.poole) wrote :

Removing this from the 2.9.6 milestone because it's not yet clear that we will continue to support the same syntax for category expressions. If we do, then we will need to implement some sort of parsing either in the engine or in the console runner as well as a ToString() that can be used to clearly display any expression found.

Changed in nunit-3.0:
milestone: 2.9.6 → none
assignee: Charlie Poole (charlie.poole) → nobody
affects: nunit-3.0 → nunit-console
Revision history for this message
Charlie Poole (charlie.poole) wrote :

This bug is now reflected in the following bugs on GitHub:
  https://github.com/nunit/nunit-console/issues/4
  https://github.com/nunit/nunit-gui/issues/17
  https://github.com/nunit/nunit-gui/issues/18
and is being closed here as "wontfix"

Changed in nunit-console:
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.