NUnit XML log files - assembly name with base class in inherited methods from base class

[Jiri Pik]

Imagine this object hierarchy of Test Fixtures:

BaseClass with test BaseClassTest. This is extended by Level01 with test Level01Test. This is extended with Level02 class with Level02Test etc.

When you run this with NUnit (e.g. on Teamcity), and save an XML log

<test-case name="NUnitInheritanceTests.Level05.BaseClass.BaseClassTest" executed="True" result="Success" success="True" time="0.000" asserts="1" />
 <test-case name="NUnitInheritanceTests.Level05.Level01.Level01Test" executed="True" result="Success" success="True" time="0.000" asserts="1" />
 <test-case name="NUnitInheritanceTests.Level05.Level02.Level02Test" executed="True" result="Success" success="True" time="0.000" asserts="1" />
 <test-case name="NUnitInheritanceTests.Level05.Level03.Level03Test" executed="True" result="Success" success="True" time="0.000" asserts="1" />
 <test-case name="NUnitInheritanceTests.Level05.Level04.Level04Test" executed="True" result="Success" success="True" time="0.000" asserts="1" />
 <test-case name="NUnitInheritanceTests.Level05.Level05Test" executed="True" result="Success" success="True" time="0.000" asserts="1" />

 As you can see, the NUnit puts into the test case name apart from the actual class also the base class of the test.

That is NUnitInheritanceTests.Level05.BaseClass.BaseClassTest actually refers to NUnitInheritanceTests.Level05.BaseClassTest (i.e. BaseClass omitted).

As this reference makes it rather difficult to match the class with the results from assembly traversal using reflection and as clarified with
Charlie Poole that there is no longer any need for the inclusion of the base class padding, i suggest to remove that Base class references,
i.e. in this case it would be NUnitInheritanceTests.Level05.BaseClassTest

[Jiri Pik (jiri-j)]

The fix is simple

=== modified file 'NUnitFramework/src/framework/Internal/Tests/TestMethod.cs'
--- NUnitFramework/src/framework/Internal/Tests/TestMethod.cs 2012-09-07 04:08:58 +0000
+++ NUnitFramework/src/framework/Internal/Tests/TestMethod.cs 2012-11-05 05:16:26 +0000
@@ -77,11 +77,6 @@
             this.Name = method.Name;
             this.FullName += "." + this.Name;

- // Disambiguate call to base class methods
- // TODO: This should not be here - it's a presentation issue
- if( method.DeclaringType != method.ReflectedType)
- this.Name = method.DeclaringType.Name + "." + method.Name;
-
             // Needed to give proper fullname to test in a parameterized fixture.
             // Without this, the arguments to the fixture are not included.
             string prefix = method.ReflectedType.FullName;

[Jiri Pik (jiri-j)]

There are a few more open issues

FIRST OPEN ISSUE:
----------------------

The case is even more complex. The fix for the above is ready but there is one more problem - the confusion about the names of attributes. Look at Name and FullName in the example below. FullName makes sense but the Name doesn’t.

Is this intentional?

I suggest to introduce new attributes BaseName (or so) and Name to distinguish between the real name and the name of the base class.

If I modify the below two files according to our use case - i.e. base class test which uses a virtual method for some modification of data, with the virtual method being overriden in the descendants, the NUnitLiteTests reports

1) TestInheritedShortMethodName (NUnit.Framework.Internal.TestMethodSignatureTes
ts.TestInheritedShortMethodName)
  Expected string length 43 but was 41. Strings differ at index 23.
  Expected: "InheritanceTestCaseNameLevel2Class.BaseTest"
  But was: "InheritanceTestCaseNameBaseClass.BaseTest"
  ----------------------------------^ ----------------------------------^

but this is not correct. I think we should have instead of

NUnitFramework/src/testdata/TestMethodSignatureFixture.cs
Last modified: 05/11/2012 05:49:04, Status: modified, Kind: file

        [TestFixture]
        public abstract class InheritanceTestCaseNameBaseClass
        {
            public abstract string GetString();

            [Test]
            public void BaseTest()
            {
                Assert.AreEqual("test", this.GetString());
            }
        }

        [TestFixture]
        public class InheritanceTestCaseNameLevel1Class : InheritanceTestCaseNameBaseClass
        {
            public override string GetString()
            {
                return "FailedTest";
            }

            [Test]
            public void Level1Test()
            {

            }
        }

        [TestFixture]
        public class InheritanceTestCaseNameLevel2Class : InheritanceTestCaseNameLevel1Class
        {

            public override string GetString()
            {
                return "test";
            }

            [Test]
            public void Level2Test()
            {

            }
        }
                }
}

NUnitFramework/src/tests/Internal/TestMethodSignatureTests.cs
Last modified: 05/11/2012 06:04:50, Status: modified, Kind: file

        [Test]
        public void TestInheritedFullMethodName()
        {
            ITestResult result = TestBuilder.RunTestCase(typeof(TestMethodSignatureFixture.InheritanceTestCaseNameLevel2Class), "BaseTest");

            Assert.That(result.FullName, Is.EqualTo("NUnit.TestData.TestMethodSignatureFixture.TestMethodSignatureFixture+InheritanceTestCaseNameLevel2Class.BaseTest"));
        }

        [Test]
        public void TestInheritedShortMethodName()
        {
            ITestResult result = TestBuilder.RunTestCase(typeof(TestMethodSignatureFixture.InheritanceTestCaseNameLevel2Class), "BaseTest");

            ResultSummary summary = new ResultSummary(result);

            Assert.That(summary.Name, Is.EqualTo("InheritanceTestCaseNameLevel2Class.BaseTest"));
        }

    }
}

SECOND OPEN ISSUE:
=======================...