Enable specifying the execution order of Action Attributes

[dasjoen]

In the Action Attribute documentation, it says, "Note that the order in which attributes are applied is indeterminate, although it will generally be stable for a single release of .NET."

What about creating an interface like this:
public interface IOrderedTestAction : ITestAction
{
    int Order { get; }
}

This way, one could specify the order of execution of the different attributes on a test.

For instance, we might have an attribute setting up dependency injection, and another one depending on it, which always should be run after it.

NUnit could then give attributes that only implement ITestAction a default order, and then execute them in the specified order.

Some of this could be solved by using base classes, as suggested here: https://groups.google.com/d/msg/nunit-discuss/ACA1FXYcygM/I56mkEpAZAUJ

However, that seems to defeat the purpose of Action Attributes, namely the possibility of composing them freely on individual tests or fixtures, without resorting to base classes (see "The Problem of Composability" here: http://www.nunit.org/index.php?p=actionAttributes&r=2.6.2 )

[Dan Lyons (dan-lyons)]

When the actions are independent of one another, the test code using them turns out quite elegant. However, I have run into a few scenarios where order matters, and the absence of some way to order the attributes forces inheritance or copy-pasted code. Much welcome refactoring could be done on some of my project's test files if I could define an order of execution.

[Charlie Poole (charlie.poole)]

To help motivate this change, please provide an example.

One workaround in some situations is to specify one attribute at the
fixture level and another at the test case level. Attributes
targetting test cases, but specified on the fixture, should execute
before those specified on the test cases.

We'll do some serious thinking about order of execution of setups,
teardowns and action attributes for NUnit 3.0.

Charlie

On Thu, Dec 26, 2013 at 1:19 PM, Dan Lyons <email address hidden> wrote:
> When the actions are independent of one another, the test code using
> them turns out quite elegant. However, I have run into a few scenarios
> where order matters, and the absence of some way to order the attributes
> forces inheritance or copy-pasted code. Much welcome refactoring could
> be done on some of my project's test files if I could define an order of
> execution.
>
> --
> You received this bug notification because you are subscribed to NUnit
> Framework.
> https://bugs.launchpad.net/bugs/1137814
>
> Title:
> Enable specifying the execution order of Action Attributes
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1137814/+subscriptions

[Dan Lyons (dan-lyons)]

The most recent examples have been tests written for UI components (done with the MVP pattern). In general, with UI presenters, most of my use-case scenarios I test share common setup steps, and at least a few of the steps have to occur in a particular order.

As a hypothetical example, let's say I have a dialog which allows a user to manage a list of work items in a queue. Let's say I want to test a scenario where a user attempts to delete an item.

The setup required for the presenter's test must include the following:
*set up the mock queue repository to return the current list of work items
*set up the mock view to raise an event for item selection
*set up the mock view to raise an event for a button/menu click to delete the item

Those events have to occur in that particular order in any stateful implementation.

Sure, if both events pass the selected object or objects in their entirety, then order doesn't matter. In fact, the test really only needs the last event. However, depending on the objects involved and the expected use cases of the dialog, it may be prohibitively expensive to pass the full selection list for every item-centric operation.

There may be many different item-centric operations available in the dialog - delete, move up/down in the queue, viewing additional details, etc. The first two steps are going to be required for any of these operations, so doing the set-up with the action attributes would be quite useful, e.g.:

[PerformSearch(order: 1)]
[SelectItem(order: 2, index: 1)]
[ClickDelete(order: 3)]
[Test]
public void it_should_remove_the_item_when_delete_is_clicked()
{
   _repo.Received().DeleteItem(_expectedItem);
}

[PerformSearch(order: 1)]
[SelectItem(order: 2, index: 1)]
[ClickDetails(order: 3)]
[Test]
public void it_should_display_details_when_details_is_clicked()
{
   _detailView.Receved().Show(_expectedItem);
}

There could even be more complex activities thrown in which require order-specific setup. For example, the dialog may have user preferences which must be loaded before the initial search, because there is a preference for default search criteria. As such, the delete test now needs to look as follows:

[GetPreferences(order: 1)]
[PerformSearch(order: 2)]
[SelectItem(order: 3, index: 1)]
[ClickDelete(order: 4)]
[Test]
public void it_should_remove_the_item_when_delete_is_clicked()
{
   _repo.Received().DeleteItem(_expectedItem);
}