TestCommand: please add the possibility of making test failures fatal

[Michał Górny]

Currently, the 'test' command just runs the test suite and relies on user reading the report. This is really inconvenient for automated testing.

For that reason, I'd like to ask you to add an option to make the test command actually fatal; that is, aborting setup.py with failure exit status. With that feature, it will be possible to run tests and automatically determine if they failed.

[Jonathan Lange (jml)]

Hi Michał,

I'm not 100% clear what you want here:

 1. `python setup.py test` to return with a non-zero exit code if the tests fail
 2. `python setup.py test` to optionally abort the test run immediately on the first failure
 3. Both 1 & 2

1. is a bug, 2. is a feature request.

Thanks,
jml

[Michał Górny (mgorny)]

I want `python setup.py test` to return with a non-zero exit code. Otherwise, I have to grep the output to know whether the tests succeeded which is fragile.