JSON size violation gives a bad error with nested templates

Bug #1499379 reported by Amit Ugol on 2015-09-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
High
Rico Lin
tempest
Undecided
Rico Lin

Bug Description

There is a 1048576 size limit to a template. In TripleO it might grow to be very big and might reach that limit but even a little bit below that limit I constantly his this error (the number 524288 seems constant no matter what size I use to test):

heatclient.exc.HTTPBadRequest: ERROR: Request limit exceeded: Template exceeds maximum allowed size (524288 bytes)

To create large templates I used this bash script (thanks Zane for the idea, sh is faster though):

#!/bin/sh

STRING=$(yes A | head -n $1 | tr -d "\n")
echo "heat_template_version: '2013-05-23'
resources:
  huge:
    type: OS::Heat::SoftwareConfig
    properties: {config: $STRING}" > huge.yaml

Save to file and run it ./test.sh N (where N is the number of characters you add to a SoftwareConfig parameter).
To reach the limit of 1048576 characters you need to add 1048334 characters to config ^

parent.yaml:
heat_template_version: '2013-05-23'
resources:
  huge:
    type: Test::Test

env.yaml:
resource_registry:
  Test::Test: file:///home/stack/huge.yaml

If we do this:
./test.sh 1048335 (one more from what we are allowed to use) and try to create a stack directly from huge.yaml:
ERROR: Request limit exceeded: JSON body size (1048578 bytes) exceeds maximum allowed size (1048576 bytes).
If we do the same with 1048334 (exactly 1048576 bytes) the stack is created normally.

To test the same with a nested template and to reach the size limit +1, set the number to 1048126 and run like so:
heat stack-create test-012 -f parent.yaml -e env.yaml
This gives the same error, but if I am exactly on limit or a little below that limit, I see this error (no matter the size I use):

heatclient.exc.HTTPBadRequest: ERROR: Request limit exceeded: Template exceeds maximum allowed size (524288 bytes)

tested on heat version 0.6.0

Changed in heat:
milestone: none → mitaka-1
importance: Undecided → High
status: New → Triaged
Rico Lin (rico-lin) on 2015-12-01
Changed in heat:
assignee: nobody → Rico Lin (rico-lin)

Fix proposed to branch: master
Review: https://review.openstack.org/252201

Changed in heat:
status: Triaged → In Progress
Rico Lin (rico-lin) on 2015-12-02
Changed in tempest:
assignee: nobody → Rico Lin (rico-lin)

Fix proposed to branch: master
Review: https://review.openstack.org/252234

Changed in tempest:
status: New → In Progress

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/252234

Changed in heat:
milestone: mitaka-1 → mitaka-2
Rico Lin (rico-lin) on 2015-12-10
no longer affects: tempest
Rico Lin (rico-lin) on 2015-12-29
Changed in tempest:
assignee: nobody → Rico Lin (rico-lin)
status: New → In Progress
Sergey Kraynev (skraynev) wrote :

Looks like dependent patch is not still merged, so move it to m-3

Changed in heat:
milestone: mitaka-2 → mitaka-3
Rico Lin (rico-lin) wrote :

This patch [1] is under a deadlock situation

Rico Lin (rico-lin) wrote :

[1] https://review.openstack.org/#/c/252201

we wait for depending tempest [2] to get merged, but since this will broke them until [1] merged, they're waiting for us to have at less one +2 in [1]. And now we waiting for each other.

[2] https://review.openstack.org/#/c/252234/

Reviewed: https://review.openstack.org/252234
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=539fd7816ed61ad3ddc98e86c7776249a8d60977
Submitter: Jenkins
Branch: master

commit 539fd7816ed61ad3ddc98e86c7776249a8d60977
Author: ricolin <email address hidden>
Date: Wed Dec 2 15:40:51 2015 +0800

    Change validate message for test_exceed_max_template_size_fails

    The error message changed for Template size validation.
    This patch change the validation message to match new parttern.

    Change-Id: I981d477c96345e68baa5a3d96c6671ad242dbb63
    Depends-On: Iddce0260436042e37eebe2aecb810707e53a4473
    Partial-Bug: #1499379

Reviewed: https://review.openstack.org/252201
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=fc1c513d35e39a14fa98308b62ade5e26c8dd37d
Submitter: Jenkins
Branch: master

commit fc1c513d35e39a14fa98308b62ade5e26c8dd37d
Author: ricolin <email address hidden>
Date: Tue Dec 29 15:32:05 2015 +0800

    JSON size violation gives a bad error with nested templates

    This patch propose a normalized error message for both template
    validation and JSON validation.

    Also Add template validation in api template parser to prevent huge template
    (besides nested stacks) pass to engine.

    Change-Id: If62621d7a6f44100a158974f0fc9c3ddf58d8d99
    Depends-On: I981d477c96345e68baa5a3d96c6671ad242dbb63
    Closes-Bug: 1499379

Changed in heat:
status: In Progress → Fix Released

This issue was fixed in the openstack/heat 6.0.0.0b3 development milestone.

Ken'ichi Ohmichi (oomichi) wrote :

This bug seems to be fixed already on both sides.

Changed in tempest:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers