test suite doesn't discriminate between unattended and attended tests

[Robert Collins]

Some of the tests will run completely happily unattended, others need user interaction. This makes it hard to run the tests automatically (see for instance bug 439757), and this makes it tricky to build automated QA

[Robert Collins (lifeless)]

I suggest controlling this either with separate make targets, or environment variables. I haven't looked deeply enough into the package to tell yet.

A desirable behaviour for packages is for
./configure
make check

to 'just work' (given that the relevant dependencies are present).

[Mirco Müller (macslow)]

What do you suggest? "make check-attended" and "make check-unattended"? What about "make distcheck"? Should that implicitly include attended and unattended tests? Or not?

[Robert Collins (lifeless)]

On Thu, 2009-10-01 at 13:32 +0000, Mirco Müller wrote:
> What do you suggest? "make check-attended" and "make check-unattended"?
> What about "make distcheck"? Should that implicitly include attended and
> unattended tests? Or not?

So, I think 'make check' should be something simple, safe and robust -
its what CI tools, debbuild etc will all [reasonably] expect to be able
to run - and they should run the unattended, xvfb based etc tests.

distcheck will naturally run that as well.

I think either
DO_ATTENDED_TESTS=1 make check
or
make check-attended

would be a good way to spell 'run the tests that need a human'.

-Rob

[Robert Collins (lifeless)]

This is now blocking automated test builds; such builds are running without 'make check' at the moment as a result.