Feature Request: Refactor SetUpFixtureAttribute and add AssemblyTearDown Attribute

[Rodney Foley]

Updated:

See the discussion for details on what will be implemented.

Original Request:

It would be great to have an attribute that can decorate a static method called AssemblyTearDown. It would mark the method that should be called after all the tests within the assembly have run to allow resources obtained by the assembly to be freed.

- It needs to be applied to a static method
- It would be called after all the other TearDown's have been processed.
- It should not need to be in a class that has test methods.
- NUnit should only allow one method to be decorated with this attribute within an assembly.

[Charlie Poole (charlie.poole)]

We already have such an attribute, but it has a different syntax.

Decorate a class outside of any namespace with [SetUpFixture].
Decorate a method of that class with [TearDown]. If you like, decorate
another with [SetUp].

Differences from what you are asking for:

1. The name makes it a little unobvious in this usage.

2. NUnit allows any number of these and calls them all, without any
guarantee of ordering. This is by design.

3. It can be applied to a static or instance method. If it's an
instance method, the class must have a default constructor and it will
be created with a lifespan of the entire test run. This is also by
design.

I'm open to suggestions for how to change this in NUnit 3.0, but it
shouldn't involve searching all methods of all classes to find the
proper methods. NUnit processes an assembly by looking first at each
class and then deciding what to use it for. We could simply continue
with the existing syntax or supplement it with an AssemblySetUpFixture
(or some other name) that applies to the whole assembly. Suggestions
welcome here.

Meanwhile, I'm making this a wishlist item for NUnit 3.0.

Charlie

PS: It's also possible to accomplish the same thing by defining an
ActionAttribute (see the docs).

On Fri, Jan 11, 2013 at 4:03 PM, Rodney Foley <email address hidden> wrote:
> Public bug reported:
>
> It would be great to have an attribute that can decorate a static method
> called AssemblyTearDown. It would mark the method that should be called
> after all the tests within the assembly have run to allow resources
> obtained by the assembly to be freed.
>
> - It needs to be applied to a static method
> - It would be called after all the other TearDown's have been processed.
> - It should not need to be in a class that has test methods.
> - NUnit should only allow one method to be decorated with this attribute within an assembly.
>
> ** Affects: nunitv2
> Importance: Undecided
> Status: New
>
>
> ** Tags: feature
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1098766
>
> Title:
> Feature Request: AssemblyTearDown Attribute
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1098766/+subscriptions

[Rodney Foley (rsfoley)]

I think you first point would be a great reason to provide an attribute call AssemblyTearDown and AssemblySetup.

I am not sure why you are saying it shouldn't involve searching all methods of all classes to find the proper methods. I didn't suggest such an awful thing. Not sure why it would, that is the purpose of the attributes I would think, and it can require that a class has a TestFixture attribute to make it simpler.

Basically what I am asking for is for how AssemblyCleanup works in MSTest. All the rules/restricts I mention are how it works in MSTest and it seems a safe approach and made sense.

I will look at how you suggest as it will work as it would be better than the hack I had to do to port this from an MSTest project to NUnit that had used AssemblyCleanup.

Thanks for the HOW TO and I hope you make this functionality a little more obvious and intuitive in the future.

[Charlie Poole (charlie.poole)]

If the attribute is on a method, then we have to search all methods
to find it. Requiring TestFixtureAttribute isn't a solution for two
reasons:

1) It's already optional, so requiring it in this single case would be
confusing.

2) It already means "This class contains tests" and that's not
what we're talking about here.

NUnit actually already has more functionality in this area than MSTest,
although possibly not so easily discoverable. We allow static or
instance methods and we can initialize and clean up for the whole
assembly or for a particular namespace. Rather complex inheritance
hierarchies can be set up, with the setup and teardown methods
a each level applied in a well-defined order. Many people use
this stuff.

In addition, using the parallel facility of ActionAttributes,
you can define common bits of initialization and cleanup and
apply them to an assembly, suite or test method just by
adding the attribute.

OTOH, having more functionality can make things confusing for
the user and I can see the value of having something with the
word "Assembly" in it applied to a class containing these
SetUp and TearDown methods.

One possibility for an inheritance hierarchy:

SetupFixtureAttribute
   NamespaceSetupAttribute
   AssemblySetupAttribute

Initially, NamespaceSetupAttribute would work just the same
as SetupFixtureAttribute. We would deprecate the latter and
eventually make it an abstract class.

AssemblySetupAttribute would apply at the assembly level
without regard to where the class lives in the namespace
hierarchy.

Within each class, we would consistently use [SetUp] and
[TearDown] to mark the methods to be called.

How does that sound?

Charlie

On Fri, Jan 11, 2013 at 8:55 PM, Rodney Foley <email address hidden> wrote:
> I think you first point would be a great reason to provide an attribute
> call AssemblyTearDown and AssemblySetup.
>
> I am not sure why you are saying it shouldn't involve searching all
> methods of all classes to find the proper methods. I didn't suggest such
> an awful thing. Not sure why it would, that is the purpose of the
> attributes I would think, and it can require that a class has a
> TestFixture attribute to make it simpler.
>
> Basically what I am asking for is for how AssemblyCleanup works in
> MSTest. All the rules/restricts I mention are how it works in MSTest
> and it seems a safe approach and made sense.
>
> I will look at how you suggest as it will work as it would be better
> than the hack I had to do to port this from an MSTest project to NUnit
> that had used AssemblyCleanup.
>
> Thanks for the HOW TO and I hope you make this functionality a little
> more obvious and intuitive in the future.
>
> --
> You received this bug notification because you are subscribed to NUnit
> Framework.
> https://bugs.launchpad.net/bugs/1098766
>
> Title:
> Feature Request: AssemblyTearDown Attribute
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1098766/+subscriptions

[Rodney Foley (rsfoley)]

That sounds a lot better overall. Sold I will take one. :)

FYI ... I was googling around and couldn't find anything on how to do this until your response to this request. A the time I created this request I was not able to get to nunit.org for some reason.

However when I was able to get to it later I am not sure I would have found SetupTestFixture, as I would have only been looking for "TearDown" stuff. There is no TearDownFixture (obvious now) so I may have not figured this out without help. I do agree that if I would have found SetupFixture that it would bee more clear as to how to do this as it is marking a class as a Namespace "Fixture" for setup and teardown. However nothing in the setup and teardown really gives any hit of be able to do this as it just says before and after each test in the class.

I would recommend at least maybe updating the documentation to help lead people to all the overloaded uses of an attribute like those of Setup and Teardown. I mean there is not even a "See Also" for SetupTestFixture on either of them.

Anyway, I am fully educated and very happy about how to do this now. Thanks.

[Charlie Poole (charlie.poole)]

The current documentation at nunit.org has been changed to provide the cross-referencing you suggest. In particular, see the first paragraph for SetUpAttribute and TearDownAttribute.

As for naming, perhaps the attribute should be called SetUpTearDownFixtureAttribute!

I'm moving this bug to NUnit 3.0 and editing the description to cover our discussion.

[Charlie Poole (charlie.poole)]

I guess it was already for NUnit 3.0.

[Rodney Foley (rsfoley)]

I love the change to the docs, and thank you very much for your help.