testr run --failing when a test runner fails to load tests reports PASSED

[Robert Collins]

Appears to be tied into the optimisation to not list-tests explicitly.

[Robert Collins (lifeless)]

Note that it doesn't lose track of what is failing, which is good, but this is certainly confusing at best.

$ testr run --failing
running=tools/start-container $INSTANCE_PROFILE 1
running=tools/run-container $INSTANCE_PROFILE 2 /tmp/tmptxYCKM -- ${PYTHON:-python} -m subunit.run discover --load-list /tmp/tmptxYCKM
Ran tests in 0.211s (+0.180s)
PASSED (id=6013)
(scratch)robertc@lifeless-z140:~/personal/testrepository$ testr run --failing
running=tools/start-container $INSTANCE_PROFILE 1
running=tools/run-container $INSTANCE_PROFILE 2 /tmp/tmpBIzxPf -- ${PYTHON:-python} -m subunit.run discover --load-list /tmp/tmpBIzxPf
======================================================================
FAIL: testrepository.tests.test_testcommand.TestTestCommand.test_get_run_command_default_and_list_expands
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "testrepository/tests/test_testcommand.py", line 233, in test_get_run_command_default_and_list_expands
    ('popen', (expected_cmd,),
NameError: global name 'expected_cmd' is not defined
Ran 1 (+1) tests in 0.782s
FAILED (id=6014, failures=1 (+1))