Ok, so you want to add another status code to the stream.
unexpectedsuccess is a combination of:
- behaviour (the test did not raise an assertion)
- expectation (the test was expected to raise an assertion)
We have the following status codes:
success (no exception, none expected)
failure ('assertion' exception, none expected)
error (other exception, none expected)
skip (not executed, no exception expected)
xfail (any exception, exception expected)
so adding a new status code does round that out. I remain dissatisfied with the modelling though; I would kind of like to separate the observed behaviour and the expected behaviour; a bit like jelmers patch to coerce xfail to success.
Something like:
status code becomes pure behavior:
completed
raised exception
not executed
expected behaviour becomes:
not supplied
completes
raises exception
not executed
unexpected success then is:
(outcome=completed, expected_outcome=raises exception)
xfail is:
(outcome=raises exception, expected_outcome=raises exception)
One problem with this is that xfail may be quite specific (raises /this/ exception) but subunit doesn't know enough to process that. A subject for TIP I think.
Ok, so you want to add another status code to the stream.
unexpectedsuccess is a combination of:
- behaviour (the test did not raise an assertion)
- expectation (the test was expected to raise an assertion)
We have the following status codes:
success (no exception, none expected)
failure ('assertion' exception, none expected)
error (other exception, none expected)
skip (not executed, no exception expected)
xfail (any exception, exception expected)
so adding a new status code does round that out. I remain dissatisfied with the modelling though; I would kind of like to separate the observed behaviour and the expected behaviour; a bit like jelmers patch to coerce xfail to success.
Something like:
status code becomes pure behavior:
completed
raised exception
not executed
expected behaviour becomes:
not supplied
completes
raises exception
not executed
unexpected success then is: outcome= raises exception) outcome= raises exception)
(outcome=completed, expected_
xfail is:
(outcome=raises exception, expected_
One problem with this is that xfail may be quite specific (raises /this/ exception) but subunit doesn't know enough to process that. A subject for TIP I think.