I find it misleading as the "test-step" XML element, by all appearances, maps to the "Test" attribute in the class, yet the time elapsed includes "SetUp" and "TearDown". Similarly, "Maxtime" is an attribute against the "Test", not "SetUp" and "TearDown", yet they can affect those results as well. Take a look at the documentation for Maxtime. The language used indicates the timing is against the "Test" method and there are no notes stating otherwise.
I find it problematic as the only way I can see to hide or forgive the performance of "SetUp" and "TearDown" would be to structure the test such that those activities are placed in additional test steps in order to get a timing for the actual stuff I want to time (i.e. what's in "Test"). This results in superfluous test steps in the report which really aren't important in the overall scheme of the test procedure.
If there is some way of suppressing the reporting of test steps, that may alleviate that problem. Otherwise, the time should be a little more specific:
1) In addition to "time", there could be "test-time", "setup-time", and "teardown-time" attributes in the XML report.
2) Attributes such as "Maxtime" could be further parameterized to give direction as to which part's timing is critical, since it applies to more than the test step.
Thanks,
Dave
-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Charlie Poole
Sent: Friday, October 26, 2012 2:20 PM
To: Kipfer, David
Subject: Re: [Nunit-core] [Bug 1071819] Re: Test time includes execution timeof Setup and Teardown methods
Hi Dave,
This has been the historical behavior of NUnit for a long time. SetUp and TearDown are considered an integral part of the test and we have generally felt that refactoring by moving some setup from SetUp to the test itself - or vice versa - should not change the time reported.
As Simone suggests, can you give us some example of why this presents a problem for you?
Charlie
On Fri, Oct 26, 2012 at 10:44 AM, Simone Busoli <email address hidden> wrote:
> Hi Dave,
>
> it would be useful to know the reasons why you think that this
> behavior is misleading and problematic.
>
> --
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
> https://bugs.launchpad.net/bugs/1071819
>
> Title:
> Test time includes execution time of Setup and Teardown methods
>
> Status in NUnit V2 Test Framework:
> New
>
> Bug description:
> The calculated time of a test includes its SetUp and TearDown methods.
>
> This includes:
>
> The "time" attribute in the "test-case" element in an XML report is
> not limited to the execution time of the decorated test method.
>
> The "MaxTime" attribute fails on this same value (time of Setup + time
> of teardown + time of test). This is first of all misleading, and
> secondly problematic.
>
> A work around would appear to be to avoid Setup and Teardown methods
> (as has been suggested for other reasons by some) and do
> setup/teardown in the scope of the fixture or the test class
> constructor/destructor. In many cases, inlining setup and teardown
> within the test case itself is not a feasible workaround.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nunit-core
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~nunit-core
> More help : https://help.launchpad.net/ListHelp
Title:
Test time includes execution time of Setup and Teardown methods
Status in NUnit V2 Test Framework:
New
Bug description:
The calculated time of a test includes its SetUp and TearDown methods.
This includes:
The "time" attribute in the "test-case" element in an XML report is
not limited to the execution time of the decorated test method.
The "MaxTime" attribute fails on this same value (time of Setup + time
of teardown + time of test). This is first of all misleading, and
secondly problematic.
A work around would appear to be to avoid Setup and Teardown methods
(as has been suggested for other reasons by some) and do
setup/teardown in the scope of the fixture or the test class
constructor/destructor. In many cases, inlining setup and teardown
within the test case itself is not a feasible workaround.
The information contained in this e-mail message is PRIVATE. It may contain confidential information and may be legally privileged. It is intended for the exclusive use of the addressee(s). If you are not the intended recipient, you are hereby notified that any dissemination, distribution or reproduction of this communication is strictly prohibited. If the intended recipient(s) cannot be reached or if a transmission problem has occurred, please notify the sender immediately by return e-mail and destroy all copies of this message.
Thank you.
Hi Charlie, Simone,
I find it misleading as the "test-step" XML element, by all appearances, maps to the "Test" attribute in the class, yet the time elapsed includes "SetUp" and "TearDown". Similarly, "Maxtime" is an attribute against the "Test", not "SetUp" and "TearDown", yet they can affect those results as well. Take a look at the documentation for Maxtime. The language used indicates the timing is against the "Test" method and there are no notes stating otherwise.
I find it problematic as the only way I can see to hide or forgive the performance of "SetUp" and "TearDown" would be to structure the test such that those activities are placed in additional test steps in order to get a timing for the actual stuff I want to time (i.e. what's in "Test"). This results in superfluous test steps in the report which really aren't important in the overall scheme of the test procedure.
If there is some way of suppressing the reporting of test steps, that may alleviate that problem. Otherwise, the time should be a little more specific:
1) In addition to "time", there could be "test-time", "setup-time", and "teardown-time" attributes in the XML report.
2) Attributes such as "Maxtime" could be further parameterized to give direction as to which part's timing is critical, since it applies to more than the test step.
Thanks,
Dave
-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Charlie Poole
Sent: Friday, October 26, 2012 2:20 PM
To: Kipfer, David
Subject: Re: [Nunit-core] [Bug 1071819] Re: Test time includes execution timeof Setup and Teardown methods
Hi Dave,
This has been the historical behavior of NUnit for a long time. SetUp and TearDown are considered an integral part of the test and we have generally felt that refactoring by moving some setup from SetUp to the test itself - or vice versa - should not change the time reported.
As Simone suggests, can you give us some example of why this presents a problem for you?
Charlie
On Fri, Oct 26, 2012 at 10:44 AM, Simone Busoli <email address hidden> wrote: /bugs.launchpad .net/bugs/ 1071819 destructor. In many cases, inlining setup and teardown /bugs.launchpad .net/nunitv2/ +bug/1071819/ +subscriptions _______ _______ _______ _______ _______ _____ /launchpad. net/~nunit- core /launchpad. net/~nunit- core /help.launchpad .net/ListHelp
> Hi Dave,
>
> it would be useful to know the reasons why you think that this
> behavior is misleading and problematic.
>
> --
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
> https:/
>
> Title:
> Test time includes execution time of Setup and Teardown methods
>
> Status in NUnit V2 Test Framework:
> New
>
> Bug description:
> The calculated time of a test includes its SetUp and TearDown methods.
>
> This includes:
>
> The "time" attribute in the "test-case" element in an XML report is
> not limited to the execution time of the decorated test method.
>
> The "MaxTime" attribute fails on this same value (time of Setup + time
> of teardown + time of test). This is first of all misleading, and
> secondly problematic.
>
> A work around would appear to be to avoid Setup and Teardown methods
> (as has been suggested for other reasons by some) and do
> setup/teardown in the scope of the fixture or the test class
> constructor/
> within the test case itself is not a feasible workaround.
>
> To manage notifications about this bug go to:
> https:/
>
> _______
> Mailing list: https:/
> Post to : <email address hidden>
> Unsubscribe : https:/
> More help : https:/
-- /bugs.launchpad .net/bugs/ 1071819
You received this bug notification because you are subscribed to the bug report.
https:/
Title:
Test time includes execution time of Setup and Teardown methods
Status in NUnit V2 Test Framework:
New
Bug description:
The calculated time of a test includes its SetUp and TearDown methods.
This includes:
The "time" attribute in the "test-case" element in an XML report is
not limited to the execution time of the decorated test method.
The "MaxTime" attribute fails on this same value (time of Setup + time
of teardown + time of test). This is first of all misleading, and
secondly problematic.
A work around would appear to be to avoid Setup and Teardown methods destructor. In many cases, inlining setup and teardown
(as has been suggested for other reasons by some) and do
setup/teardown in the scope of the fixture or the test class
constructor/
within the test case itself is not a feasible workaround.
To manage notifications about this bug go to: /bugs.launchpad .net/nunitv2/ +bug/1071819/ +subscriptions
https:/
The information contained in this e-mail message is PRIVATE. It may contain confidential information and may be legally privileged. It is intended for the exclusive use of the addressee(s). If you are not the intended recipient, you are hereby notified that any dissemination, distribution or reproduction of this communication is strictly prohibited. If the intended recipient(s) cannot be reached or if a transmission problem has occurred, please notify the sender immediately by return e-mail and destroy all copies of this message.
Thank you.