Subclasses of ParameterDataAttribute producing nondeterministic output are marked as not run

[Grzegorz Galezowski]

[This bug is now tracked at https://github.com/nunit/nunit-vs-adapter/issues/15]

The following code works fine with NUnit test adapter (VS 2012, nunit 2.6.2):

    [AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false, Inherited = false)]
    public class AnyAttribute : ParameterDataAttribute
    {
      public override IEnumerable GetData(ParameterInfo parameter)
      {
          if (parameter.ParameterType == typeof(int))
          {
            return new[] { 122 };
          }
          else
          {
            return new[] { "a" };
          }
      }
    }

    public class Class1234
    {
      [Test]
      public void ShouldBEHAVIOR([Any] int a, [Any] string b)
      {
        Assert.Fail("FAILED!");
      }
    }

But the following does not (although it works just fine in gui standalone nunit runner) and VS Test Runner marks the test as not run:

    [AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false, Inherited = false)]
    public class AnyAttribute : ParameterDataAttribute
    {
      public override IEnumerable GetData(ParameterInfo parameter)
      {
          if (parameter.ParameterType == typeof(int))
          {
            return new[] { 122 };
          }
          else
          {
            return new[] { "a" + new Random().Next() }; //______ONLY CHANGE________!
          }
      }
    }

    public class Class1234
    {
      [Test]
      public void ShouldBEHAVIOR([Any] int a, [Any] string b)
      {
        Assert.Fail("FAILED!");
      }
    }

Best regards,
grzesiek

[Grzegorz Galezowski (spectral)]

This includes also tests having arguments marked with [Random] attribute.

[Charlie Poole (charlie.poole)]

I've confirmed this. Any use of Random in generating the test cases causes the tests to be not run without explanation under the TestExplorer. The same tests work fine in NUnit. This can be easily duplicated by use of ValueSource with a value-generating method that uses Random or - as stated - using RandomAttribute.

[Charlie Poole (charlie.poole)]

The project info here on Launchpad explains that we have moved to GitHub and you should not file bugs here.

Since I already made a comment on it, I'll enter an issue on GitHub pointing back to this one. Kindly read the instructions next time.

[Grzegorz Galezowski (spectral)]

Charlie, thanks for taking care of this and pointing me to github! I promise to be a good citizen next time :-).