The project file is intended to be something permanent, specifying one
or more assemblies
as well as settings that are *necessary* for running the tests in
those assemblies. To
understand what I mean by necessary, consider an assembly with dependencies that
are in a subdirectory. If we don't set up a privatebinpath that
includes that subdirectory,
the assemblies won't be found, so it's not possible to run the tests
without that setting.

On the other hand, selecting particular test cases is not a permanent
thing related to
the test assemblies but is more transient. That is, today I want to
run a certain test
fixture, but tomorrow it might be different.

More specific comments inline...

> Ok, so i've tried to understand the NUnit project file, it looks like it
> is most useful for the GUI runner, isn't it? What i didn't understand is
> that there is a reference to a configfile, huh!!??

It's possible to have multiple projects for the same assembly, using
different configs.

> The test-assemblies can be listed only once, there is no more redundant information on a set for debug/release configuration

This could be seen as an improvement on the existing format, but it
has a limitation:
each config must always contain the same assemblies. The ability to
list different
assemblies in each config has always been a feature of the format.
Much of what you
suggest below only works if the configs all contain the same assemblies.

> A set of tests can be configured independant of the configurations.

Of course, this only works if the same assemblies appear in each config.

> More than one set of tests can be configured independant of the configurations, classified by a testset name

Or one could simply add more configurations, no?

> A autorun-property specifies if the Config runs automatically when the ProjectFile is passed as argument to the console-runner.exe

This is the existing defaultConfig setting. It's not shown in the
example file because it
uses a setting that exists for NUnit tests only. Since NUnit is
self-testing,t the autoConfig
setting is used to run Debug when we are running under a Debug build of NUnit,
Release otherwise.

> Each testcase e. g. testassembly will produce it's own xml resultfile

Actually, we go to a lot of trouble to produce a single xml resultfile for each
project. In fact, that's one of the main reasons for having the project file.
What's the advantage of separate result files that you would then have
to re-combine for analysis?

> About the GUI-runner: I have seen that using the checkboxes within the
> treeview, each test can be checked or unchecked. But i have not seen
> that this is persistet within the NUnit ProjectFile. Do i miss something
> here?

That's because this is considered something transient, not permanent. The
info is, however, persisted in the VisualState.xml file.

***

I'll summarize using the points you listed at the start of your comment.

> Provide basic configuration stuff.
> Provide the test-dll names

This is what the current format does

> Provide path/name of output files e. g. report files
> Provide include/exclude of categories
> Provide filters/selectors to choose certain testmethods by namespace

This is currently done on the command line, because I envision them as changing
from run to run. For that reason, one run of the console only allows you to
specify one set of these parameters.

To move to the next step, it would be helpful to understand

1) Why the last three points are permanent settings for the test runs
in your environment.
That is, why might one *always* want to run an assembly with
/exclude:Experimental.

(2) Why we might need to have different result files per assembly,
which is a need I
haven't run into before.

Charlie

>
> ** Attachment added: "Alternative NUnit Project File"
>    https://bugs.launchpad.net/nunit-3.0/+bug/1231877/+attachment/3855339/+files/NUnitTestprojectPeter.xml
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1231877
>
> Title:
>   Passing parameters to console runner by file
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1231877/+subscriptions