'run_in_parallel' config option does not work

[Mike Boruta]

From my testing and after looking at the source code it does not seem that the 'run_in_parallel' task configuration option (as described here: https://github.com/openstack/rally/blob/master/doc/specs/implemented/new_rally_input_task_format.rst, and defined here: https://github.com/openstack/rally/blob/3.1.0/rally/common/objects/task.py#L57) is anything but a NOP at this point.

The configuration option does not seem to be read anywhere, and the comment in db/models.py indicates that the value is always false: https://github.com/openstack/rally/blob/3.1.0/rally/common/db/models.py#L177. Is my understanding correct? If so, shouldn't references to it be removed from the source code?

Note: Closely related to: https://answers.launchpad.net/rally/+question/693863.

[Andriy Kurilin (andreykurilin)]

Hi!

You are right, this feature is not implemented yet (would you like to work on it?) and this parameter should be removed from jsonschema .

PS: https://github.com/openstack/rally/blob/master/doc/specs/implemented/new_rally_input_task_format.rst was about input format, but not about feature itself.

[Mike Boruta (mboruta1)]

Hello Andrey,

Ok, understood. If I cannot achieve my goal otherwise I will strongly consider implementing a first draft of this feature.

Thank you for your help!

[Mike Boruta (mboruta1)]

Hello Andrey, how much effort do you think it would be to implement the `run_in_parallel` feature? Where would you suggest one start?

[Andriy Kurilin (andreykurilin)]

Sorry for the long reply. New Year deadlines and holidays :)

If you are still interested in working on this task:

1) run_in_parallel option is incorrect and should be replaced with something like 'runner' section like we have at workloads.

2) There is special class that covers the logic of running tasks - rally.task.engine

3) https://github.com/openstack/rally/blob/3.2.0/rally/task/engine.py#L500 takes care of running the workloads.

4) https://github.com/openstack/rally/blob/3.2.0/rally/task/engine.py#L466 is taking care of running subtasks. This code covers 'serial' execution

5) The new entity SubtaskRunner should be introduced. The logic of finding the proper plugin and further execution can be copy-pasted (with adoption) from workloads (see #3 point).

6) The current code of _run_subtasks (see #4) should go to SerialSubtaskRunner.

7) The feature of running workloads in parallel should be implemented in the separate plugin of SubtaskRunner. It can be similar to https://github.com/openstack/rally/blob/master/rally/plugins/task/runners/constant.py#L154 .

PS: Feel free to ping me on Gitter or email, it should be a quicker way to reach me.

[Mike Boruta (mboruta1)]

Thanks a lot for your detailed response Andrey.

I have decided to go for lower hanging fruit first, but plan on revisiting this at a later date. Will be sure to keep in touch (via email and/or Gitter) when I do.

Thanks again! :)