Comment 3 for bug 1071819

Hi Charlie, Simone,

I find it misleading as the "test-step" XML element, by all appearances, maps to the "Test" attribute in the class, yet the time elapsed includes "SetUp" and "TearDown". Similarly, "Maxtime" is an attribute against the "Test", not "SetUp" and "TearDown", yet they can affect those results as well. Take a look at the documentation for Maxtime. The language used indicates the timing is against the "Test" method and there are no notes stating otherwise.

I find it problematic as the only way I can see to hide or forgive the performance of "SetUp" and "TearDown" would be to structure the test such that those activities are placed in additional test steps in order to get a timing for the actual stuff I want to time (i.e. what's in "Test"). This results in superfluous test steps in the report which really aren't important in the overall scheme of the test procedure.

If there is some way of suppressing the reporting of test steps, that may alleviate that problem. Otherwise, the time should be a little more specific:

1) In addition to "time", there could be "test-time", "setup-time", and "teardown-time" attributes in the XML report.
2) Attributes such as "Maxtime" could be further parameterized to give direction as to which part's timing is critical, since it applies to more than the test step.

Thanks,
Dave

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Charlie Poole
Sent: Friday, October 26, 2012 2:20 PM
To: Kipfer, David
Subject: Re: [Nunit-core] [Bug 1071819] Re: Test time includes execution timeof Setup and Teardown methods

Hi Dave,

This has been the historical behavior of NUnit for a long time. SetUp and TearDown are considered an integral part of the test and we have generally felt that refactoring by moving some setup from SetUp to the test itself - or vice versa - should not change the time reported.

As Simone suggests, can you give us some example of why this presents a problem for you?

Charlie

On Fri, Oct 26, 2012 at 10:44 AM, Simone Busoli <email address hidden> wrote:
> Hi Dave,
>
> it would be useful to know the reasons why you think that this
> behavior is misleading and problematic.
>
> --
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
> https://bugs.launchpad.net/bugs/1071819
>
> Title:
> Test time includes execution time of Setup and Teardown methods
>
> Status in NUnit V2 Test Framework:
> New
>
> Bug description:
> The calculated time of a test includes its SetUp and TearDown methods.
>
> This includes:
>
> The "time" attribute in the "test-case" element in an XML report is
> not limited to the execution time of the decorated test method.
>
> The "MaxTime" attribute fails on this same value (time of Setup + time
> of teardown + time of test). This is first of all misleading, and
> secondly problematic.
>
> A work around would appear to be to avoid Setup and Teardown methods
> (as has been suggested for other reasons by some) and do
> setup/teardown in the scope of the fixture or the test class
> constructor/destructor. In many cases, inlining setup and teardown
> within the test case itself is not a feasible workaround.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nunit-core
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~nunit-core
> More help : https://help.launchpad.net/ListHelp

--
You received this bug notification because you are subscribed to the bug report.
https://bugs.launchpad.net/bugs/1071819

Title:
  Test time includes execution time of Setup and Teardown methods

Status in NUnit V2 Test Framework:
  New

Bug description:
  The calculated time of a test includes its SetUp and TearDown methods.

  This includes:

  The "time" attribute in the "test-case" element in an XML report is
  not limited to the execution time of the decorated test method.

  The "MaxTime" attribute fails on this same value (time of Setup + time
  of teardown + time of test). This is first of all misleading, and
  secondly problematic.

  A work around would appear to be to avoid Setup and Teardown methods
  (as has been suggested for other reasons by some) and do
  setup/teardown in the scope of the fixture or the test class
  constructor/destructor. In many cases, inlining setup and teardown
  within the test case itself is not a feasible workaround.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions

The information contained in this e-mail message is PRIVATE. It may contain confidential information and may be legally privileged. It is intended for the exclusive use of the addressee(s). If you are not the intended recipient, you are hereby notified that any dissemination, distribution or reproduction of this communication is strictly prohibited. If the intended recipient(s) cannot be reached or if a transmission problem has occurred, please notify the sender immediately by return e-mail and destroy all copies of this message.
Thank you.