Zaqar error when Mistral message size exceeded during Pike minor update.

[Marios Andreou]

As discussed in https://bugzilla.redhat.com/show_bug.cgi?id=1498916
for the minor update we need to limit the size of the mistral update task result. This result is sent to zaqar, and we set a limit of 1mbyte at [0]. In some cases, such as running the minor update against all nodes in the deployment, the size of the task result exceeds this limit and zaqar returns a 400 error, even though the task itself is successful.

[0] https://github.com/openstack/instack-undercloud/blob/d60cd845022679f70c382f355bd59f5007210e17/elements/puppet-stack-config/puppet-stack-config.yaml.template#L681

[Bogdan Dobrelya (bogdando)]

I think this is a dup of https://bugs.launchpad.net/tripleo/+bug/1727406

[Marios Andreou (marios-b)]

why do you think so Bogdan? I had a quick look there but not sure, I should have included the error trace from that BZ perhaps to make it clearer (it is literally a 400 response from zaqar)

fig', u'type': u'direct'}}, u'name': u'update_nodes', u'tags': [u'tripleo-common-managed'], u'version': u'2.0', u'input': [{u'node_user': u'heat-admin'}, u'nodes', u'playbook', u'inventory_file', {u'queue_name': u'tripleo'}], u'description': u'Take a container and perform an update nodes by nodes'}}}}}}']
 ZaqarAction.queue_post failed: Error response from Zaqar. Code: 400. Title: Invalid API request. Description: Message collection size is too large. Max size 1048576.
] (execution_id=0b455e17-611d-4cb3-91a2-714f13a3a30e)
2017-10-05 10:05:53.812 17734 DEBUG mistral.services.triggers [req-eac2a476-cb08-4d63-8ddb-c2dc788d3f6d 81c58aff164344a9811bddd511c739ea 9ed039ecceba4438b980339efe25a93a - default default] No JSON object could be decoded on_workflow_complete /usr/lib/python2.7/site-packages/mistral/services/triggers.py:239

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/520571

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/518328
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=82db05923a322826cb58c84b6713571e675b7c90
Submitter: Zuul
Branch: master

commit 82db05923a322826cb58c84b6713571e675b7c90
Author: marios <email address hidden>
Date: Tue Nov 7 15:02:26 2017 +0200

    Truncate the zaqar message to 512 kbytes

    We need to limit this otherwise zaqar will return 400 error as we
    are setting a maximum of 1mbyte at [0]. We limit at 256 kbytes
    for each of stderr and stdout.

    Closes-Bug: 1730663
    [0] https://github.com/openstack/instack-undercloud/blob/d60cd845022679f70c382f355bd59f5007210e17/elements/puppet-stack-config/puppet-stack-config.yaml.template#L681
    Change-Id: I1e9a962d7c3e4448163a05c30c8facfe346a4252

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/520571
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=ae239264c9edc56976b235e42d34d3efb91b8140
Submitter: Zuul
Branch: stable/pike

commit ae239264c9edc56976b235e42d34d3efb91b8140
Author: marios <email address hidden>
Date: Tue Nov 7 15:02:26 2017 +0200

    Truncate the zaqar message to 512 kbytes

    We need to limit this otherwise zaqar will return 400 error as we
    are setting a maximum of 1mbyte at [0]. We limit at 256 kbytes
    for each of stderr and stdout.

    Closes-Bug: 1730663
    (cherry picked from commit 82db05923a322826cb58c84b6713571e675b7c90)
    [0] https://github.com/openstack/instack-undercloud/blob/d60cd845022679f70c382f355bd59f5007210e17/elements/puppet-stack-config/puppet-stack-config.yaml.template#L681
    Change-Id: I1e9a962d7c3e4448163a05c30c8facfe346a4252

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 7.6.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 8.2.0 release.