feature request: better validation of lava-test-case name in YAML files

[Andrew McDermott]

I was auto generating a YAML file based on the Java package names and each entry looks like:

    - 'lava-test-case java/lang/Integer --shell ./jtreg-lava java/lang/Integer'
    - 'lava-test-case java/lang/String --shell ./jtreg-lava java/lang/String'

What I hadn't realized was that the error I was getting back:

<LAVA_TEST_RUNNER>: exiting
linaro-test [rc=0]# echo LAVA_ACK<LAVA_DISPATCHER>2013-08-16 03:18:18 PM INFO: lava_test_shell seems to have completed
<LAVA_DISPATCHER>2013-08-16 03:18:18 PM WARNING: [ACTION-E] lava_test_shell is finished with error (ValidationError: Object lacks property 'result' object_expr='object.test_runs[0].test_results[2]', schema_expr='schema.properties.test_runs.items.properties.test_results.items.properties.result.optional')).
Lava failed at action lava_test_shell with error:ValidationError: Object lacks property 'result' object_expr='object.test_runs[0].test_results[2]', schema_expr='schema.properties.test_runs.items.properties.test_results.items.properties.result.optional')
Traceback (most recent call last):
[stacktrace omitted as not useful]

is actually related to the lava-test-case name. Which is now obvious in hindsight as I'm guessing the test case name becomes a filename.

If I change this to:

    - 'lava-test-case java.lang.Integer --shell ./jtreg-lava java/lang/Integer'
    - 'lava-test-case java.lang.String --shell ./jtreg-lava java/lang/String'

then this runs to completion.

It would be helpful if either the name could be validated and an error thrown (and the test not run) or perhaps the test name could be normalized under the hood to remove path separators. I was confused for a long time because I had some other tests preceding those entries (as listed above) and I thought it was actually something wrong with the parser... Ho hum!