Looks like the 'time' filed of task result for tempest test is string rather than floating and thus failed the type validation.
Part of the output:
======
Totals
======
Ran: 1 tests in 27.0000 sec.
- Passed: 1
- Skipped: 0
- Expected Fail: 0
- Unexpected Success: 0
- Failed: 0
Sum of execute time for each test: 0.0913 sec.
==============
Worker Balance
==============
- Worker 0 (1 tests) => 0:00:00.091320
2016-04-06 04:01:12.274 26534 INFO rally.task.runner [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | ITER: 0 END: OK
(1L, <class 'rally.plugins.openstack.scenarios.tempest.tempest.TempestScenario'>, 'single_test', {'task': <rally.common.objects.task.Task object at 0x3c1d310>, 'admin': {'credential': <rally.common.objects.credential.Credential object at 0x3c150d0>}, 'verifier': <rally.verification.tempest.tempest.Tempest object at 0x3c15510>, 'tmp_results_dir': u'/tmp/5907bddb-3252-4934-aedd-cfc726c487c0-results', 'user': {'credential': <rally.common.objects.credential.Credential object at 0x3c15710>, 'id': u'4479c28d9b6a4274970f979247571c15', 'tenant_id': u'b358465bf671484f9480ecaf385bc08b'}, 'scenario_name': 'TempestScenario.single_test', 'config': {'tempest': {}, 'users': {'project_domain': 'default', 'users_per_tenant': 1, 'tenants': 1, 'resource_management_workers': 20, 'user_domain': 'default'}}, 'tenant': {'id': u'b358465bf671484f9480ecaf385bc08b', 'name': u'c_rally_5907bddb_UnIap1WN'}}, {'tempest_conf': '/etc/tempest/tempest.conf', 'test_name': 'tempest.api.image.v1.test_images.ListImagesTest.test_index_no_params'})
2016-04-06 04:01:12.279 26534 INFO rally.task.runner [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | ITER: 1 START
2016-04-06 04:01:12.280 26534 INFO rally.verification.tempest.tempest [-] Tempest config file: /etc/tempest/tempest.conf
2016-04-06 04:01:12.290 26293 INFO rally.plugins.openstack.context.not_for_production.tempest [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | Starting: Exit context: `tempest`
2016-04-06 04:01:12.291 26293 INFO rally.plugins.openstack.context.not_for_production.tempest [-] Built-in stress cleanup from Tempest looks like can help to shot yourself in the foot. Sorry, but even Rally can not clean up after Tempest. Deal with it.
2016-04-06 04:01:12.291 26293 INFO rally.plugins.openstack.context.not_for_production.tempest [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | Completed: Exit context: `tempest`
2016-04-06 04:01:12.291 26293 INFO rally.plugins.openstack.context.keystone.users [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | Starting: Exit context: `users`
tee: /tmp/5907bddb-3252-4934-aedd-cfc726c487c0-results/tmpAKrrE0: 没有那个文件或目录
2016-04-06 04:01:15.230 26293 INFO rally.plugins.openstack.context.keystone.users [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | Completed: Exit context: `users`
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine [-] '12.942' is not of type 'number', 'null'
Failed validating 'type' in schema['properties']['atomic_actions']['patternProperties']['.*']:
{'type': ['number', 'null']}
On instance['atomic_actions']['test_execution']:
'12.942'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine Traceback (most recent call last):
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/task/engine.py", line 340, in run
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine workload.args)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/task/runner.py", line 196, in run
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._run_scenario(cls, method_name, context, args)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/plugins/common/runners/constant.py", line 210, in _run_scenario
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._join_processes(process_pool, result_queue)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/task/runner.py", line 243, in _join_processes
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._send_result(result_queue.get())
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/task/runner.py", line 262, in _send_result
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine r = ScenarioRunnerResult(result)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/openstack_repo/openstack/rally/rally/task/runner.py", line 131, in __init__
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine jsonschema.validate(result_list, self.RESULT_SCHEMA)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/python/virtualenv/SENLINENV/lib/python2.7/site-packages/jsonschema/validators.py", line 478, in validate
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine cls(schema, *args, **kwargs).validate(instance)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/stack/huyanyan/python/virtualenv/SENLINENV/lib/python2.7/site-packages/jsonschema/validators.py", line 123, in validate
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine raise error
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine ValidationError: '12.942' is not of type 'number', 'null'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine Failed validating 'type' in schema['properties']['atomic_actions']['patternProperties']['.*']:
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine {'type': ['number', 'null']}
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine On instance['atomic_actions']['test_execution']:
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine '12.942'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.627 26293 INFO rally.task.engine [-] Task 5907bddb-3252-4934-aedd-cfc726c487c0 | Completed: Benchmarking.
When I tried to run the following sample tempest test, a type validation error happened:
http:// git.openstack. org/cgit/ openstack/ rally/tree/ samples/ tasks/scenarios /tempest- do-not- run-against- production/ single_ test.yaml
Looks like the 'time' filed of task result for tempest test is string rather than floating and thus failed the type validation.
Part of the output:
======
Totals
======
Ran: 1 tests in 27.0000 sec.
- Passed: 1
- Skipped: 0
- Expected Fail: 0
- Unexpected Success: 0
- Failed: 0
Sum of execute time for each test: 0.0913 sec.
============== 3252-4934- aedd-cfc726c487 c0 | ITER: 0 END: OK plugins. openstack. scenarios. tempest. tempest. TempestScenario '>, 'single_test', {'task': <rally. common. objects. task.Task object at 0x3c1d310>, 'admin': {'credential': <rally. common. objects. credential. Credential object at 0x3c150d0>}, 'verifier': <rally. verification. tempest. tempest. Tempest object at 0x3c15510>, 'tmp_results_dir': u'/tmp/ 5907bddb- 3252-4934- aedd-cfc726c487 c0-results' , 'user': {'credential': <rally. common. objects. credential. Credential object at 0x3c15710>, 'id': u'4479c28d9b6a4 274970f97924757 1c15', 'tenant_id': u'b358465bf6714 84f9480ecaf385b c08b'}, 'scenario_name': 'TempestScenari o.single_ test', 'config': {'tempest': {}, 'users': {'project_domain': 'default', 'users_per_tenant': 1, 'tenants': 1, 'resource_ management_ workers' : 20, 'user_domain': 'default'}}, 'tenant': {'id': u'b358465bf6714 84f9480ecaf385b c08b', 'name': u'c_rally_ 5907bddb_ UnIap1WN' }}, {'tempest_conf': '/etc/tempest/ tempest. conf', 'test_name': 'tempest. api.image. v1.test_ images. ListImagesTest. test_index_ no_params' }) 3252-4934- aedd-cfc726c487 c0 | ITER: 1 START ion.tempest. tempest [-] Tempest config file: /etc/tempest/ tempest. conf openstack. context. not_for_ production. tempest [-] Task 5907bddb- 3252-4934- aedd-cfc726c487 c0 | Starting: Exit context: `tempest` openstack. context. not_for_ production. tempest [-] Built-in stress cleanup from Tempest looks like can help to shot yourself in the foot. Sorry, but even Rally can not clean up after Tempest. Deal with it. openstack. context. not_for_ production. tempest [-] Task 5907bddb- 3252-4934- aedd-cfc726c487 c0 | Completed: Exit context: `tempest` openstack. context. keystone. users [-] Task 5907bddb- 3252-4934- aedd-cfc726c487 c0 | Starting: Exit context: `users` 3252-4934- aedd-cfc726c487 c0-results/ tmpAKrrE0: 没有那个文件或目录 openstack. context. keystone. users [-] Task 5907bddb- 3252-4934- aedd-cfc726c487 c0 | Completed: Exit context: `users`
Worker Balance
==============
- Worker 0 (1 tests) => 0:00:00.091320
2016-04-06 04:01:12.274 26534 INFO rally.task.runner [-] Task 5907bddb-
(1L, <class 'rally.
2016-04-06 04:01:12.279 26534 INFO rally.task.runner [-] Task 5907bddb-
2016-04-06 04:01:12.280 26534 INFO rally.verificat
2016-04-06 04:01:12.290 26293 INFO rally.plugins.
2016-04-06 04:01:12.291 26293 INFO rally.plugins.
2016-04-06 04:01:12.291 26293 INFO rally.plugins.
2016-04-06 04:01:12.291 26293 INFO rally.plugins.
tee: /tmp/5907bddb-
2016-04-06 04:01:15.230 26293 INFO rally.plugins.
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine [-] '12.942' is not of type 'number', 'null'
Failed validating 'type' in schema[ 'properties' ]['atomic_ actions' ]['patternPrope rties'] ['.*']:
{'type': ['number', 'null']}
On instance[ 'atomic_ actions' ]['test_ execution' ]: stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ task/engine. py", line 340, in run stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ task/runner. py", line 196, in run scenario( cls, method_name, context, args) stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ plugins/ common/ runners/ constant. py", line 210, in _run_scenario processes( process_ pool, result_queue) stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ task/runner. py", line 243, in _join_processes result( result_ queue.get( )) stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ task/runner. py", line 262, in _send_result esult(result) stack/huyanyan/ openstack_ repo/openstack/ rally/rally/ task/runner. py", line 131, in __init__ validate( result_ list, self.RESULT_SCHEMA) stack/huyanyan/ python/ virtualenv/ SENLINENV/ lib/python2. 7/site- packages/ jsonschema/ validators. py", line 478, in validate .validate( instance) stack/huyanyan/ python/ virtualenv/ SENLINENV/ lib/python2. 7/site- packages/ jsonschema/ validators. py", line 123, in validate 'properties' ]['atomic_ actions' ]['patternPrope rties'] ['.*']: 'atomic_ actions' ]['test_ execution' ]: 3252-4934- aedd-cfc726c487 c0 | Completed: Benchmarking.
'12.942'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine Traceback (most recent call last):
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine workload.args)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._run_
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._join_
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine self._send_
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine r = ScenarioRunnerR
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine jsonschema.
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine cls(schema, *args, **kwargs)
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine File "/home/
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine raise error
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine ValidationError: '12.942' is not of type 'number', 'null'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine Failed validating 'type' in schema[
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine {'type': ['number', 'null']}
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine On instance[
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine '12.942'
2016-04-06 04:01:16.368 26293 ERROR rally.task.engine
2016-04-06 04:01:16.627 26293 INFO rally.task.engine [-] Task 5907bddb-