testr run doesn't complain when filtering results in no selected tests

Bug #1264274 reported by Thomas Herve on 2013-12-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Testrepository
Wishlist
Unassigned

Bug Description

If I do:

$ testr run mypackage.mydule_typo

I don't get any error from testr run, and also a success error code. It seems to somewhat be by design, because testr lists all the tests available, and then filter with what I pass on the command line. In this case though I pass directly a module name, so it'd be nice if 1) It just tried to import it and 2) it told me that it failed to do so.

Jaspreet (walia-jaspreet) wrote :

I agree with Thomas Herve suggestion. I ran the below non existing test case and got the success message, however I didn't get test count(since there was no test run) .

It would be nice if it could inform user about non-existent tests.

I ran two set of tests. With non existing tests I got following message
PASSED (id=5)

With existing tests I got
Ran 8 tests in 7.346s (+2.018s)
PASSED (id=4)

Please find below tests I ran
-----------------------------------------------------------------
# 1 . testr run tempest.api.compute.flavors.test_flavors_negative1
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list
-------------
PASSED (id=5)
-----------

Ran the following test , run ran successfully along with number of tests it ran .
# testr run tempest.api.compute.flavors.test_flavors_negative

running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --load-list /tmp/tmpKZMyg_
Ran 8 tests in 7.346s (+2.018s)
PASSED (id=4)

Robert Collins (lifeless) wrote :

I'm not sure what to do about this - its certainly a valid complaint, but I worry about the difference between None and 0 here - basically filtering down to nothing to do is a legitimate thing. Note that the arguments are regexes, so a) you can be a lot more pithy with your selectors and b) we don't know (and currently can't) whether something is a module to import or whatnot - testr is multi-language.

Changed in testrepository:
status: New → Triaged
summary: - testr run doesn't complain when it doesn't run tests
+ testr run doesn't complain when filtering results in no selected tests
Changed in testrepository:
importance: Undecided → Wishlist
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers