Task serialization when YAQL used takes too much time

[Stanislaw Bogatkin]

Fuel 10.0, current master

When we use tasks with YAQL conditions on environment with many nodes (200+), it takes really long time to get serialization done.
On machine with Intel(R) Xeon(R) CPU E5-2620 (12 cores) we have next serialization time:

5 nodes:
2016-04-15 14:15:49.834 tasks serialization is started.
2016-04-15 14:16:05.277 tasks serialization is finished.
diff = 16s

50 nodes:
2016-04-15 14:30:29.376 tasks serialization is started.
2016-04-15 14:33:25.227 tasks serialization is finished.
diff: 176s (2:56)

200 nodes:
VSIZE: 1.2G, RSIZE: 488.3M
2016-04-15 14:39:25.297 tasks serialization is started.
2016-04-15 15:09:05.396 tasks serialization is finished.
diff: 1780s (29:40)

500 nodes:
VSIZE:2.2G, RSIZE: 1.6G
2016-04-15 15:33:56.381 tasks serialization is started.
2016-04-15 18:08:02.706 tasks serialization is finished
diff: 9246s (2:34:06)

It's really much, we need to find a way to improve this.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/307740

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/318206

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/318206
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=9a806f1d12ed45d7fa623dfb0dec664a8bcb2328
Submitter: Jenkins
Branch: master

commit 9a806f1d12ed45d7fa623dfb0dec664a8bcb2328
Author: Stanislaw Bogatkin <email address hidden>
Date: Wed May 18 19:28:36 2016 +0300

    Use C vairant of YAML dumper at tasks serialization

    To speed up seiralization time, use faster implementation of
    YAML dumping when it is possible.

    Change-Id: I992410328c07d14a1f5903aa91d456b040175a69
    Partial-Bug: #1572103
    Co-Authored-With: V. Kuklin <email address hidden>
    Co-Authored-With: B. Gaifullin <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/322217

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/322217
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=6dea7664c2365bc5fd6ac72e80943d3136a5880f
Submitter: Jenkins
Branch: stable/mitaka

commit 6dea7664c2365bc5fd6ac72e80943d3136a5880f
Author: Stanislaw Bogatkin <email address hidden>
Date: Wed May 18 19:28:36 2016 +0300

    Use C vairant of YAML dumper at tasks serialization

    To speed up seiralization time, use faster implementation of
    YAML dumping when it is possible.

    Change-Id: I992410328c07d14a1f5903aa91d456b040175a69
    Partial-Bug: #1572103
    Co-Authored-With: V. Kuklin <email address hidden>
    Co-Authored-With: B. Gaifullin <email address hidden>
    (cherry picked from commit 9a806f1d12ed45d7fa623dfb0dec664a8bcb2328)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/322960

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/307740
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=02c4283fe7ac850c0d51e03ee9c05cc2ee3a2f60
Submitter: Jenkins
Branch: master

commit 02c4283fe7ac850c0d51e03ee9c05cc2ee3a2f60
Author: Stanislaw Bogatkin <email address hidden>
Date: Tue Apr 19 14:55:57 2016 +0300

    Multiprocess tasks serialization

    Tasks serialization process take long time in case when environment
    contains many nodes. To reduce time of serialization, make
    serialization process work in parallel (reduces time of
    serialization mostly linear way with increasing workers pool)

    DocImpact
    Change-Id: Id3753dbc6983256d410e69c98ab02b61ab6bfb7f
    Partial-Bug: #1572103
    Co-Authored-With: V. Kuklin <email address hidden>
    Co-Authored-With: B. Gaifullin <email address hidden>

[Bulat Gaifullin (bulat.gaifullin)]

ETA: 1 day

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/322960
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=4efe81f5894299f361fede9836c65f4ec4ba7853
Submitter: Jenkins
Branch: stable/mitaka

commit 4efe81f5894299f361fede9836c65f4ec4ba7853
Author: Stanislaw Bogatkin <email address hidden>
Date: Tue Apr 19 14:55:57 2016 +0300

    Multiprocess tasks serialization

    Tasks serialization process take long time in case when environment
    contains many nodes. To reduce time of serialization, make
    serialization process work in parallel (reduces time of
    serialization mostly linear way with increasing workers pool)

    DocImpact
    Change-Id: Id3753dbc6983256d410e69c98ab02b61ab6bfb7f
    Partial-Bug: #1572103
    Co-Authored-With: V. Kuklin <email address hidden>
    Co-Authored-With: B. Gaifullin <email address hidden>
    (cherry picked from commit 02c4283fe7ac850c0d51e03ee9c05cc2ee3a2f60)

[Kyrylo Galanov (kgalanov)]

Verified on ISO #465 RC1