Error in Test Discovery

[Sławomir Kwasiborski]

Tests implemented in files that are not in projects root folder do not apear in Test Explorer window and are not runned during "run all" nor durng "run tests" from context menu of the file containing tests.

[Charlie Poole (charlie.poole)]

Version please?

On Thu, Nov 1, 2012 at 1:42 PM, Sławomir Kwasiborski
<email address hidden> wrote:
> Public bug reported:
>
> Tests implemented in files that are not in projects root folder do not
> apear in Test Explorer window and are not runned during "run all" nor
> durng "run tests" from context menu of the file containing tests.
>
> ** Affects: nunit-vs-adapter
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1074130
>
> Title:
> Tests in nested folders are not run
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-vs-adapter/+bug/1074130/+subscriptions

[Sławomir Kwasiborski (s-kwasiborski)]

Sorry for not including that info:

NUnit Test Adapter (Beta 3) - version: 0.93.1
NUnit - version 2.6.1 and 2.6.2 (same problem for both).

I encountered that problem when working with FakeItEasy project. You
cen reproduce this by checking out
https://github.com/skwasiborski/FakeItEasy repo.

regards

Sławek

2012/11/2 Charlie Poole <email address hidden>:
> Version please?
>
> On Thu, Nov 1, 2012 at 1:42 PM, Sławomir Kwasiborski
> <email address hidden> wrote:
>> Public bug reported:
>>
>> Tests implemented in files that are not in projects root folder do not
>> apear in Test Explorer window and are not runned during "run all" nor
>> durng "run tests" from context menu of the file containing tests.
>>
>> ** Affects: nunit-vs-adapter
>> Importance: Undecided
>> Status: New
>>
>> --
>> You received this bug notification because you are subscribed to NUnit
>> Extended Testing Platform.
>> https://bugs.launchpad.net/bugs/1074130
>>
>> Title:
>> Tests in nested folders are not run
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/nunit-vs-adapter/+bug/1074130/+subscriptions
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1074130
>
> Title:
> Tests in nested folders are not run
>
> Status in NUnit Test Adapter for Visual Studio:
> New
>
> Bug description:
> Tests implemented in files that are not in projects root folder do not
> apear in Test Explorer window and are not runned during "run all" nor
> durng "run tests" from context menu of the file containing tests.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-vs-adapter/+bug/1074130/+subscriptions

[Charlie Poole (charlie.poole)]

I looked at your project and loaded the FakeItEasy-Dev solution into VS2012. (The larger main solution shows ambiguous calll errors in the SL3 project on my machine)

I observe that there's a problem loading all the tests, but it doesn't appear to be correlated with the location of the source in any way. That's as expected, since NUnit knows nothing at all about the source.

It's a good idea to look at the discovery output when we don't seem to be finding all your tests. In this case, it indicates that 985 tests were found but the ide only lists 106 tests. There is no obvious reason why some are included and some are not. The output also shows that two test names are truncated to 449 characters by Visual Studio. This is a known problem. There is also a message "Exception has been thrown by the target of an invocation." I'll try to track that down.

Charlie

[Charlie Poole (charlie.poole)]

The above exception is caused by the test DefaultExceptionThrowerTests.Should_throw_correct_exception_when_resolved_constructors_are_used.

If you comment out that test, no problem arises. Discovery shows 982 test cases and the IDE displays them all.

Editing the expected message for the second and third cases so that the test name does not get truncated also eliminates the problem. Discovery then shows all 985 cases. Of course the edited cases then fail.

My hypothesis is that the truncated test cases are causing some failure in Visual Studio, but I have no way to verify that. I'm told that the truncation is by design. We have a bug for NUnit to try to shorten long names in a more useful way before they go to Visual Studio.

By the way, your tests run fine in the NUnit Gui.

I don't see any more that we can do about this problem, but I'll wait for you to comment before closing the bug. Your best recourse is to either shorten the names of the tests yourself or provide an alternate name using TestCaseData or in your own code that emulates it. I spent a bit of time on this because I thought there might be some way that NUnit could highlight the source of the problem, but it appears to be out of our hands.

Charlie

[Charlie Poole (charlie.poole)]

Removing from 0.94, since it depends on a fix in NUnit itself.

[Charlie Poole (charlie.poole)]

Moved to https://github.com/nunit/nunit-vs-adapter/issues/1 and marked Wont Fix here