rally 0.9.1 task junit report generator does not count errors

[jiri]

running the following:

rally deployment create --fromenv --name=rally

rally task start /opt/rally/samples/tasks/scenarios/nova/boot-and-live-migrate.json
rally task start /opt/rally/samples/tasks/scenarios/nova/boot-from-volume-and-delete.json
rally task start /opt/rally/samples/tasks/scenarios/cinder/create-and-attach-volume.json
rally task start /opt/rally/samples/tasks/scenarios/neutron/create-and-delete-networks.json
rally task start /opt/rally/samples/tasks/scenarios/neutron/create-and-delete-routers.json
rally task start /opt/rally/samples/tasks/scenarios/keystone/create-user-update-password.json

tasks=`rally task list | awk '{print $2}' | grep -v uuid`
rally task report --tasks $tasks --junit --out report.xml

RESULTS:
even though errors happen during tasks run report does not count them. As seen below.

root@ctl01:~# cat report.xml
<testsuite errors="0" failures="0" name="Rally test suite" tests="7" time="116.14"><testcase classname="NovaServers" name="boot_and_live_migrate_server" time="19.72" /><testcase classname="NovaServers" name="boot_server_from_volume_and_delete" time="10.75" /><testcase cla
ssname="CinderVolumes" name="create_and_attach_volume" time="19.06" /><testcase classname="CinderVolumes" name="create_and_attach_volume" time="22.33" /><testcase classname="NeutronNetworks" name="create_and_delete_networks" time="6.01" /><testcase classname="NeutronNetwo
rks" name="create_and_delete_routers" time="30.82" /><testcase classname="KeystoneBasic" name="create_user_update_password" time="7.46" /></testsuite>

[chenhb (chen-haibing1)]

What was wrong? sorry, i did not see the problem. please get me more details.

[jiri (broulik)]

Hey, the problem is here: <testsuite errors="0"

as the html generates the output correctly but junit does not count errors

2017-08-17 9:54 GMT+02:00 chenhb <email address hidden>:

> What was wrong? sorry, i did not see the problem. please get me more
> details.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1711082
>
> Title:
> rally 0.9.1 task junit report generator does not count errors
>
> Status in Rally:
> Confirmed
>
> Bug description:
> running the following:
>
> rally deployment create --fromenv --name=rally
>
> rally task start /opt/rally/samples/tasks/scenarios/nova/boot-and-live-
> migrate.json
> rally task start /opt/rally/samples/tasks/scenarios/nova/boot-from-
> volume-and-delete.json
> rally task start /opt/rally/samples/tasks/scenarios/cinder/create-and-
> attach-volume.json
> rally task start /opt/rally/samples/tasks/scenarios/neutron/create-and-
> delete-networks.json
> rally task start /opt/rally/samples/tasks/scenarios/neutron/create-and-
> delete-routers.json
> rally task start /opt/rally/samples/tasks/scenarios/keystone/create-
> user-update-password.json
>
> tasks=`rally task list | awk '{print $2}' | grep -v uuid`
> rally task report --tasks $tasks --junit --out report.xml
>
> RESULTS:
> even though errors happen during tasks run report does not count them.
> As seen below.
>
> root@ctl01:~# cat report.xml
> <testsuite errors="0" failures="0" name="Rally test suite" tests="7"
> time="116.14"><testcase classname="NovaServers" name="boot_and_live_migrate_server"
> time="19.72" /><testcase classname="NovaServers"
> name="boot_server_from_volume_and_delete" time="10.75" /><testcase cla
> ssname="CinderVolumes" name="create_and_attach_volume" time="19.06"
> /><testcase classname="CinderVolumes" name="create_and_attach_volume"
> time="22.33" /><testcase classname="NeutronNetworks"
> name="create_and_delete_networks" time="6.01" /><testcase
> classname="NeutronNetwo
> rks" name="create_and_delete_routers" time="30.82" /><testcase
> classname="KeystoneBasic" name="create_user_update_password" time="7.46"
> /></testsuite>
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/rally/+bug/1711082/+subscriptions
>

--

----------------------------------
S přáním krásného dne
Ing. Jiří Broulík

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/513744
Committed: https://git.openstack.org/cgit/openstack/rally/commit/?id=e8ed18e11bb7f3923df5a847f50477cb0b0777f4
Submitter: Zuul
Branch: master

commit e8ed18e11bb7f3923df5a847f50477cb0b0777f4
Author: Andrey Kurilin <email address hidden>
Date: Fri Oct 20 17:38:18 2017 +0300

    [task] Rework junit-xml exporter

    * show different tasks results separately
    * fix calculations

    PS: module rally.common.io.junit will be removed as soon as we clean CLI
    layer from old formats.

    Change-Id: Id950615e4e3f873d7928678278c030211e7cc998
    Closes-Bug: #1711082