tripleoclient needs a way to receive multiple messages from a workflow

[Dougal Matthews]

At the moment, tripleoclient expects to only receive one message from the zaqar queue. It waits on a specific queue and finishes when a message comes with the execution ID for the workflow it started. It also ignores all messages from sub-workflows (even if they use the same queue, because it checks for a execution ID match.

This should be fairly easy to to, unless we plan on sending multiple messages from the same queue. Then we may need to use some convention to tell tripleoclient when to stop listening. We could use a convention like this:

1. Wait for messages on a queue.
2. consume and forward all messages from all workflows
3. If a message is from the execution ID it started (the parent workflow) and it contains "workflow_finished: true". Then stop listening.

[Julie Pichon (jpichon)]

https://review.openstack.org/#/c/406178/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/443053

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446049

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446054

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446081

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446337

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446338

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446339

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446432

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446433

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446434

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446435

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446436

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446437

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446438

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/446439

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/406178
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=07477487434d57ecf329143c07f050564ab30edf
Submitter: Jenkins
Branch: master

commit 07477487434d57ecf329143c07f050564ab30edf
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 8 09:24:13 2017 +0000

    Handle multiple messages from workflow executions

    Currently, tripleoclient will ignore messages from workflows if they
    don't match the execution ID of the worflow it starts. This can happen
    if the workflow starts sub-workflows, which run under their own
    execution ID. tripleoclient will also stop listening for messages after
    it recieves the first message from the workflow it started. Meaning, it
    assumes the workflow to send one, and only one, message.

    This change tackles the first limitation, by turning the message
    consumption into a generator which yields each message one at a time.
    It then starts a convention of assuming the workflow has finished if it
    recieves a message with the top level execution ID and it contains a
    status that isn't "RUNNING". For example: {"status": "SUCCESS"}.

    Partial-Bug: #1646887
    Change-Id: Ida07718b4fab41e0a6088f2db8f9b42b6fb09f72

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446432
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=3b9c2a454b59fff526b209bdceb950c5fa9e2be7
Submitter: Jenkins
Branch: master

commit 3b9c2a454b59fff526b209bdceb950c5fa9e2be7
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 15 15:39:09 2017 +0000

    Support multiple messages from register_or_update

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: Ib99f4871ded5a531f9408e72813977eaf2dbd272
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/443053

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/446433

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/443053

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.openstack.org/446433

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446433
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=fd8817d7f4969c2e03670dc0e30c6f3b571ec795
Submitter: Jenkins
Branch: master

commit fd8817d7f4969c2e03670dc0e30c6f3b571ec795
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 08:17:42 2017 +0000

    Support multiple messages from create_default_plan

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I602e434f425838aec320b68a405f1014fd856d0f
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/443053
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=b93b7e9ce1e49cb29fffa2f19ce0f7350251b0d6
Submitter: Jenkins
Branch: master

commit b93b7e9ce1e49cb29fffa2f19ce0f7350251b0d6
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 09:35:21 2017 +0000

    Support multiple messages from get_passwords

    This updates the workflow wrapper to use the multiple message interface.

    This patch also adds test coverage for this code. Previously it was
    mocked out in all places that touched it.

    Change-Id: Ie2619238592563d0ca3442e0f26967b12d9c6eb1
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446438
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=fcf8c4d4cbb99b2eb64bd2bfa833601ca9f372f9
Submitter: Jenkins
Branch: master

commit fcf8c4d4cbb99b2eb64bd2bfa833601ca9f372f9
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 09:55:07 2017 +0000

    Support multiple messages from delete_node

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I1b4ace2cef45f4aa979e26f4ba88aea6a17bd03e
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446049
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=17f20bdaa68e38e802cf614d0ff684a7fac3e2aa
Submitter: Jenkins
Branch: master

commit 17f20bdaa68e38e802cf614d0ff684a7fac3e2aa
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 15 15:39:09 2017 +0000

    Support multiple messages from provide workflow

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I8ec4c3db9a2bd84d8cb577febef177cf6b663a86
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446054
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=f71fc36a3c723d8ed19b187733e67c43156b5228
Submitter: Jenkins
Branch: master

commit f71fc36a3c723d8ed19b187733e67c43156b5228
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 15 16:25:20 2017 +0000

    Support multiple messages from introspect workflow

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I065652693a1c15e8cad9a06f715a0d5306ff828f
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446065
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=363ff169000abca67e7246c842c8f10f00a5047f
Submitter: Jenkins
Branch: master

commit 363ff169000abca67e7246c842c8f10f00a5047f
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 15 16:52:07 2017 +0000

    Support multiple messages from introspect manageable

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: If29cb791acb5a01629bf2a97767fbcb2299b154c
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446081
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=165c0a0c04dda9d930e868b7dd43478a106d209e
Submitter: Jenkins
Branch: master

commit 165c0a0c04dda9d930e868b7dd43478a106d209e
Author: Dougal Matthews <email address hidden>
Date: Wed Mar 15 17:08:14 2017 +0000

    Support multiple messages from provide_manageable_nodes

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I05f303649664b19ff932b6928b93a64c7636db71
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446337
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=6d39eb5e943b566ad0b0604755fabcc09e3957b5
Submitter: Jenkins
Branch: master

commit 6d39eb5e943b566ad0b0604755fabcc09e3957b5
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 07:48:27 2017 +0000

    Support multiple messages from configure workflow

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I4b807b0e8d13f75193e33cffab77e4df74e05b5b
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446338
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=67b4beb66c56b3f2d02f952b602bfa5b0a45af07
Submitter: Jenkins
Branch: master

commit 67b4beb66c56b3f2d02f952b602bfa5b0a45af07
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 08:00:04 2017 +0000

    Support multiple messages from configure_manageable_nodes

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I208b14fef721d0fcdb2e340cb5237d590e832a98
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446339
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=9bbc6c6c2719b7d4d422b384628cf17763733c5e
Submitter: Jenkins
Branch: master

commit 9bbc6c6c2719b7d4d422b384628cf17763733c5e
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 08:02:35 2017 +0000

    Support multiple messages from create_raid_configuration

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I5aea1f924c91675301e35f007634ae01fa6d39de
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446436
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=0f2b860d86ad89276c54d914c877d809602d1b20
Submitter: Jenkins
Branch: master

commit 0f2b860d86ad89276c54d914c877d809602d1b20
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 09:09:27 2017 +0000

    Support multiple messages from delete_stack

    This updates the workflow wrapper to use the multiple message interface.

    NOTE: the test for this patch use the fake websocket that was updated in
    a previous patch, so while this code is tested, no test updates were
    needed for this change.

    Change-Id: I6f6b09589646b6eaf5811fdf23558c949fbbc2b8
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446435
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=7378a389289dd7ff9ad7d870d01ec104c1664c82
Submitter: Jenkins
Branch: master

commit 7378a389289dd7ff9ad7d870d01ec104c1664c82
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 08:43:16 2017 +0000

    Support multiple messages from package_update

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I5a04fb057979ca43c725b99900a1607dafe86a81
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446434
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=ad0aae9cff1e40fb8a7001e2b6fd199ce101fbf1
Submitter: Jenkins
Branch: master

commit ad0aae9cff1e40fb8a7001e2b6fd199ce101fbf1
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 08:25:33 2017 +0000

    Support multiple messages from create/update_deployment_plan

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I972a4f5baebf4e719243b7519eb3189cd2036dde
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446437
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=aa593917874a14a9755b216758fa64f223ad35d5
Submitter: Jenkins
Branch: master

commit aa593917874a14a9755b216758fa64f223ad35d5
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 09:44:31 2017 +0000

    Support multiple messages from deploy_plan

    This updates the workflow wrapper to use the multiple message interface.

    Change-Id: I09b4cbfb32fca59c4b055a61f20c33f2c893c96b
    Partial-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446439
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=70998137f6bc8ada0fa5e6fecd8c4f655028e17c
Submitter: Jenkins
Branch: master

commit 70998137f6bc8ada0fa5e6fecd8c4f655028e17c
Author: Dougal Matthews <email address hidden>
Date: Thu Mar 16 09:59:09 2017 +0000

    Remove the single-message interface to websockets

    This is longer used and the multiple-message interface should be used.

    Change-Id: I84a2d593ae3bad2559de530de8678925a2109547
    Closes-Bug: #1646887

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 7.0.0 release.