NUnit Framework

Bug #730891
Comment #8

Comment 8 for bug 730891

Revision history for this message

Nicolas Rivera (nick4christ) wrote on 2011-07-26: Re: GUI runner option to stop upon failure

Hi Charlie,

Thanks for your prompt replies!

No, I am not asking for stopping tests through Sleep. I recognize what I did was a hack -- but it is a hack that gives me more control than my previous hack (I can now stop the unit tests without executing teardown, and I can select which unit tests I want to stop upon failure).

What I was wondering is if it would make more sense to be able to stop the a batch of unit tests programmatically through some new mechanism added to NUnit, instead of my original request of a GUI setting of "Stop upon failure". Stopping a batch of unit tests from inside TearDown() programmatically is just far more flexible for the unit test writer -- and it would not depend on the client (i.e., it could be used while running through the console also).

Let me give you a user scenario for this feature. In my case, I am running subsystem-level automated tests. Furthermore, because I write unmanaged code to interface with cellular devices, the unit tests are actually excercising code that communicates with hardware (and networks). There are some errors that are not easily reproducible. The one I am working on now is such an example. Sometimes it can be reproduced quite often in spurts, but othertimes I can run all the unit tests for hours without any error. Being able to stop the unit test programmatically depending on some condition (in this case a failure), and being able to skip the TearDown() code upon that condition (so that the current state stays in memory), allows me to make a memory dump with windbg that shows pretty much what is going on in the system at the time of the failure.

I realize my scenario is not very common. But I do suspect that any type of subsystem/integration testing would benefit from being able to do that. Oftentimes integration testing deals with multiple threads and environmental conditions that make the tests appear somewhat undeterministic. Of course, if all you are doing is strict TDD at the method level, then there is no need to do that. But I thought one of the reasons for NUnit 3.0 was to allow more flexibility for other types of testing (e.g., integration testing).

Thanks,

Nicolas

Hi Charlie,

Thanks for your prompt replies!

No, I am not asking for stopping tests through Sleep.  I recognize what I did was a hack -- but it is a hack that gives me more control than my previous hack (I can now stop the unit tests without executing teardown, and I can select which unit tests I want to stop upon failure).

What I was wondering is if it would make more sense to be able to stop the a batch of unit tests programmatically through some new mechanism added to NUnit, instead of my original request of a GUI setting of "Stop upon failure".  Stopping a batch of unit tests from inside TearDown() programmatically is just far more flexible for the unit test writer -- and it would not depend on the client (i.e., it could be used while running through the console also).

Let me give you a user scenario for this feature.  In my case, I am running subsystem-level automated tests.  Furthermore, because I write unmanaged code to interface with cellular devices, the unit tests are actually excercising code that communicates with hardware (and networks).  There are some errors that are not easily reproducible.  The one I am working on now is such an example.  Sometimes it can be reproduced quite often in spurts, but othertimes I can run all the unit tests for hours without any error.  Being able to stop the unit test programmatically depending on some condition (in this case a failure), and being able to skip the TearDown() code upon that condition (so that the current state stays in memory), allows me to make a memory dump with windbg that shows pretty much what is going on in the system at the time of the failure.

I realize my scenario is not very common.  But I do suspect that any type of subsystem/integration testing would benefit from being able to do that.  Oftentimes integration testing deals with multiple threads and environmental conditions that make the tests appear somewhat undeterministic.  Of course, if all you are doing is strict TDD at the method level, then there is no need to do that.  But I thought one of the reasons for NUnit 3.0 was to allow more flexibility for other types of testing (e.g., integration testing).

Thanks,

Nicolas