| 2017-02-21 16:16:31 |
Masayuki Igawa |
bug |
|
|
added bug |
| 2017-02-21 16:24:45 |
Masayuki Igawa |
description |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex ." option, it just doesn't work, it means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_sub [0.160894s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_testtools [0.006166s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover [0.007800s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover_path [0.005592s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb [0.007164s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb_path [0.005918s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --serial: not allowed with argument --parallel
{0} os_testr.tests.test_ostestr.TestGetParser.test_parallel [0.009608s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --pretty/-p: not allowed with argument --no-pretty
{0} os_testr.tests.test_ostestr.TestGetParser.test_pretty [0.007776s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --slowest: not allowed with argument --no-slowest
{0} os_testr.tests.test_ostestr.TestGetParser.test_slowest [0.010752s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005553s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006085s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.007783s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.005995s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004675s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.003263s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004938s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004601s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.696434s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.006149s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004827s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.003709s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails [2.269518s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails_no_slowest [1.785471s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing [1.969635s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing_no_slowest [1.885153s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_fails [1.927992s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_passing [2.734155s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_list [2.534997s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_fails [3.258661s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
----------- |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex" option, it works, it means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_sub [0.160894s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_testtools [0.006166s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover [0.007800s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover_path [0.005592s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb [0.007164s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb_path [0.005918s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --serial: not allowed with argument --parallel
{0} os_testr.tests.test_ostestr.TestGetParser.test_parallel [0.009608s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --pretty/-p: not allowed with argument --no-pretty
{0} os_testr.tests.test_ostestr.TestGetParser.test_pretty [0.007776s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --slowest: not allowed with argument --no-slowest
{0} os_testr.tests.test_ostestr.TestGetParser.test_slowest [0.010752s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005553s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006085s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.007783s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.005995s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004675s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.003263s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004938s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004601s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.696434s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.006149s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004827s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.003709s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails [2.269518s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails_no_slowest [1.785471s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing [1.969635s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing_no_slowest [1.885153s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_fails [1.927992s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_passing [2.734155s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_list [2.534997s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_fails [3.258661s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
-----------
And with "--until-failure --regex regex", it works.
--------------
$ tox -e py27 -- --until-failure --regex regex
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='437009315'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex regex
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005498s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.004846s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004682s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.004495s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004165s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005095s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004879s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004493s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.872275s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.003902s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004028s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.005301s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.007411s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005171s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006087s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004581s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.003456s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.007177s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005229s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.002719s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004716s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.844681s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.002802s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004415s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.004982s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.006891s] ... ok |
|
| 2017-02-21 17:29:58 |
Masayuki Igawa |
os-testr: assignee |
|
Masayuki Igawa (igawa) |
|
| 2017-02-21 20:23:36 |
Masayuki Igawa |
description |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex" option, it works, it means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_sub [0.160894s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_testtools [0.006166s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover [0.007800s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover_path [0.005592s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb [0.007164s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb_path [0.005918s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --serial: not allowed with argument --parallel
{0} os_testr.tests.test_ostestr.TestGetParser.test_parallel [0.009608s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --pretty/-p: not allowed with argument --no-pretty
{0} os_testr.tests.test_ostestr.TestGetParser.test_pretty [0.007776s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --slowest: not allowed with argument --no-slowest
{0} os_testr.tests.test_ostestr.TestGetParser.test_slowest [0.010752s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005553s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006085s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.007783s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.005995s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004675s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.003263s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004938s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004601s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.696434s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.006149s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004827s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.003709s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails [2.269518s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails_no_slowest [1.785471s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing [1.969635s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing_no_slowest [1.885153s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_fails [1.927992s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_passing [2.734155s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_list [2.534997s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_fails [3.258661s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
-----------
And with "--until-failure --regex regex", it works.
--------------
$ tox -e py27 -- --until-failure --regex regex
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='437009315'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex regex
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005498s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.004846s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004682s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.004495s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004165s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005095s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004879s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004493s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.872275s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.003902s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004028s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.005301s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.007411s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005171s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006087s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004581s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.003456s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.007177s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005229s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.002719s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004716s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.844681s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.002802s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004415s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.004982s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.006891s] ... ok |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex" option, it works which means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_sub [0.160894s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_testtools [0.006166s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover [0.007800s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover_path [0.005592s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb [0.007164s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb_path [0.005918s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --serial: not allowed with argument --parallel
{0} os_testr.tests.test_ostestr.TestGetParser.test_parallel [0.009608s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --pretty/-p: not allowed with argument --no-pretty
{0} os_testr.tests.test_ostestr.TestGetParser.test_pretty [0.007776s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --slowest: not allowed with argument --no-slowest
{0} os_testr.tests.test_ostestr.TestGetParser.test_slowest [0.010752s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005553s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006085s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.007783s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.005995s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004675s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.003263s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004938s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004601s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.696434s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.006149s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004827s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.003709s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails [2.269518s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails_no_slowest [1.785471s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing [1.969635s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing_no_slowest [1.885153s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_fails [1.927992s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_passing [2.734155s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_list [2.534997s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_fails [3.258661s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
-----------
And with "--until-failure --regex regex", it works.
--------------
$ tox -e py27 -- --until-failure --regex regex
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='437009315'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex regex
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005498s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.004846s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004682s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.004495s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004165s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005095s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004879s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004493s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.872275s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.003902s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004028s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.005301s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.007411s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005171s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006087s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004581s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.003456s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.007177s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005229s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.002719s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004716s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.844681s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.002802s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004415s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.004982s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.006891s] ... ok |
|
| 2017-02-21 20:29:09 |
Masayuki Igawa |
description |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex" option, it works which means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_sub [0.160894s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_testtools [0.006166s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover [0.007800s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_no_discover_path [0.005592s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb [0.007164s] ... ok
{0} os_testr.tests.test_ostestr.TestCallers.test_pdb_path [0.005918s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --serial: not allowed with argument --parallel
{0} os_testr.tests.test_ostestr.TestGetParser.test_parallel [0.009608s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --pretty/-p: not allowed with argument --no-pretty
{0} os_testr.tests.test_ostestr.TestGetParser.test_pretty [0.007776s] ... ok
usage: run.py [-h] [--version] [--blacklist-file BLACKLIST_FILE]
[--whitelist-file WHITELIST_FILE]
[--regex REGEX | --path FILE_OR_DIRECTORY | --no-discover TEST_ID]
[--black-regex BLACK_REGEX] [--pretty | --no-pretty] [--subunit]
[--list] [--color] [--slowest | --no-slowest] [--pdb TEST_ID]
[--parallel | --serial] [--concurrency WORKERS]
[--until-failure] [--print-exclude]
run.py: error: argument --slowest: not allowed with argument --no-slowest
{0} os_testr.tests.test_ostestr.TestGetParser.test_slowest [0.010752s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005553s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006085s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.007783s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.005995s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004675s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.003263s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004938s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004601s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.696434s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.006149s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004827s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.003709s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails [2.269518s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_fails_no_slowest [1.785471s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing [1.969635s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_passing_no_slowest [1.885153s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_fails [1.927992s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_default_serial_passing [2.734155s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_list [2.534997s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_fails [3.258661s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
-----------
And with "--until-failure --regex regex", it works.
--------------
$ tox -e py27 -- --until-failure --regex regex
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='437009315'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex regex
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005498s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.004846s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004682s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.004495s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.004165s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005095s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.004879s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004493s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.872275s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.003902s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004028s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.005301s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.007411s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005171s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments [0.006087s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comments_and_regex [0.004581s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comment_print_skips [0.003456s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments [0.007177s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_without_comments_and_regex [0.005229s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_regex_passthrough [0.002719s] ... ok
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_whitelist_regex_with_comments [0.004716s] ... ok
{0} os_testr.tests.test_regex_builder.TestGetTestList.test__get_test_list [0.844681s] ... ok
{0} os_testr.tests.test_regex_builder.TestPathToRegex.test_file_name [0.002802s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_read_whitelist_file [0.004415s] ... ok
{0} os_testr.tests.test_regex_builder.TestWhitelistFile.test_whitelist_regex_without_comments_and_regex [0.004982s] ... ok
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.006891s] ... ok |
An error occurs with --until-failure option.
-------------------
$ tox -e py27 -- --until-failure
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='2382833288'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure
Traceback (most recent call last):
File ".tox/py27/bin/ostestr", line 10, in <module>
sys.exit(main())
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 312, in main
exit(ostestr(sys.argv[1:]))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 308, in ostestr
return (_select_and_call_runner(opts, regex, others))
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 239, in _select_and_call_runner
opts.until_failure, opts.color, None, others)
File "/Users/igawa/git/openstack/os-testr/os_testr/ostestr.py", line 144, in call_testr
test_list = rb._get_test_list(regex, env)
File "/Users/igawa/git/openstack/os-testr/os_testr/regex_builder.py", line 23, in _get_test_list
stdout=subprocess.PIPE, universal_newlines=True)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: coercing to Unicode: need string or buffer, NoneType found
ERROR: InvocationError: '/Users/igawa/git/openstack/os-testr/.tox/py27/bin/ostestr --until-failure'
__________ summary ____________
ERROR: py27: commands failed
------------------------------------------------
However, with "--regex" option, it works which means it finish after the test running even if it succeed.
-----------
$ tox -e py27 -- --until-failure --regex .
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='664289420'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex .
{0} os_testr.tests.test_ostestr.TestCallers.test_call_subunit_run_pretty [0.037627s] ... ok
<SNIP>
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_no_pretty_passing [2.733892s] ... ok
{0} os_testr.tests.test_return_codes.TestReturnCodes.test_testr_subunit_fails ... SKIPPED: Skipped because of testrepository lp bug #1411804
No tests were successful during the run
Ran 31 tests without failure
___________________________________________________________________________________ summary ____________________________________________________________________________________
py27: commands succeeded
congratulations :)
-----------
And with "--until-failure --regex regex", it works.
--------------
$ tox -e py27 -- --until-failure --regex regex
py27 develop-inst-nodeps: /Users/igawa/git/openstack/os-testr
py27 installed: alabaster==0.7.9,appdirs==1.4.0,Babel==2.3.4,coverage==4.2,ddt==1.1.1,debtcollector==1.9.0,docutils==0.12,extras==1.0.0,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,keystoneauth1==2.15.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,mox3==0.18.0,openstack-requirements==1.0.0,os-client-config==1.24.0,-e git+https://review.openstack.org/p/openstack/os-testr@f0e3a1a01473e86351b99f8b3a73ee699b4d1374#egg=os_testr,oslosphinx==4.8.0,oslotest==2.11.0,packaging==16.8,Parsley==1.3,pbr==1.10.0,pep8==1.5.7,positional==1.1.1,pyflakes==0.8.1,Pygments==2.1.3,pyparsing==2.1.10,python-mimeparse==1.6.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,requests==2.12.5,requestsexceptions==1.1.3,six==1.10.0,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unittest2==1.1.0,wrapt==1.10.8
py27 runtests: PYTHONHASHSEED='437009315'
py27 runtests: commands[0] | find . -type f -name *.pyc -delete
py27 runtests: commands[1] | ostestr --until-failure --regex regex
{0} os_testr.tests.test_regex_builder.TestConstructRegex.test_blacklist_regex_with_comment_print_skips [0.005498s] ... ok
<SNIP>
{0} os_testr.tests.testlist_builder.TestConstructList.test_overlapping_black_regex [0.006891s] ... ok |
|
| 2017-02-21 21:42:54 |
OpenStack Infra |
os-testr: status |
New |
In Progress |
|
| 2017-03-03 00:04:44 |
OpenStack Infra |
os-testr: status |
In Progress |
Fix Released |
|
