JSON size violation gives a bad error with nested templates

Bug #1499379 reported by Removed by request
6
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
High
Rico Lin
tempest
Fix Released
Undecided
Rico Lin

Bug Description

There is a 1048576 size limit to a template. In TripleO it might grow to be very big and might reach that limit but even a little bit below that limit I constantly his this error (the number 524288 seems constant no matter what size I use to test):

heatclient.exc.HTTPBadRequest: ERROR: Request limit exceeded: Template exceeds maximum allowed size (524288 bytes)

To create large templates I used this bash script (thanks Zane for the idea, sh is faster though):

#!/bin/sh

STRING=$(yes A | head -n $1 | tr -d "\n")
echo "heat_template_version: '2013-05-23'
resources:
  huge:
    type: OS::Heat::SoftwareConfig
    properties: {config: $STRING}" > huge.yaml

Save to file and run it ./test.sh N (where N is the number of characters you add to a SoftwareConfig parameter).
To reach the limit of 1048576 characters you need to add 1048334 characters to config ^

parent.yaml:
heat_template_version: '2013-05-23'
resources:
  huge:
    type: Test::Test

env.yaml:
resource_registry:
  Test::Test: file:///home/stack/huge.yaml

If we do this:
./test.sh 1048335 (one more from what we are allowed to use) and try to create a stack directly from huge.yaml:
ERROR: Request limit exceeded: JSON body size (1048578 bytes) exceeds maximum allowed size (1048576 bytes).
If we do the same with 1048334 (exactly 1048576 bytes) the stack is created normally.

To test the same with a nested template and to reach the size limit +1, set the number to 1048126 and run like so:
heat stack-create test-012 -f parent.yaml -e env.yaml
This gives the same error, but if I am exactly on limit or a little below that limit, I see this error (no matter the size I use):

heatclient.exc.HTTPBadRequest: ERROR: Request limit exceeded: Template exceeds maximum allowed size (524288 bytes)

tested on heat version 0.6.0

Changed in heat:
milestone: none → mitaka-1
importance: Undecided → High
status: New → Triaged
Rico Lin (rico-lin)
Changed in heat:
assignee: nobody → Rico Lin (rico-lin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/252201

Changed in heat:
status: Triaged → In Progress
Rico Lin (rico-lin)
Changed in tempest:
assignee: nobody → Rico Lin (rico-lin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/252234

Changed in tempest:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tempest (master)

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/252234

Changed in heat:
milestone: mitaka-1 → mitaka-2
Rico Lin (rico-lin)
no longer affects: tempest
Rico Lin (rico-lin)
Changed in tempest:
assignee: nobody → Rico Lin (rico-lin)
status: New → In Progress
Revision history for this message
Sergey Kraynev (skraynev) wrote :

Looks like dependent patch is not still merged, so move it to m-3

Changed in heat:
milestone: mitaka-2 → mitaka-3
Revision history for this message
Rico Lin (rico-lin) wrote :

This patch [1] is under a deadlock situation

Revision history for this message
Rico Lin (rico-lin) wrote :

[1] https://review.openstack.org/#/c/252201

we wait for depending tempest [2] to get merged, but since this will broke them until [1] merged, they're waiting for us to have at less one +2 in [1]. And now we waiting for each other.

[2] https://review.openstack.org/#/c/252234/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.openstack.org/252234
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=539fd7816ed61ad3ddc98e86c7776249a8d60977
Submitter: Jenkins
Branch: master

commit 539fd7816ed61ad3ddc98e86c7776249a8d60977
Author: ricolin <email address hidden>
Date: Wed Dec 2 15:40:51 2015 +0800

    Change validate message for test_exceed_max_template_size_fails

    The error message changed for Template size validation.
    This patch change the validation message to match new parttern.

    Change-Id: I981d477c96345e68baa5a3d96c6671ad242dbb63
    Depends-On: Iddce0260436042e37eebe2aecb810707e53a4473
    Partial-Bug: #1499379

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/252201
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=fc1c513d35e39a14fa98308b62ade5e26c8dd37d
Submitter: Jenkins
Branch: master

commit fc1c513d35e39a14fa98308b62ade5e26c8dd37d
Author: ricolin <email address hidden>
Date: Tue Dec 29 15:32:05 2015 +0800

    JSON size violation gives a bad error with nested templates

    This patch propose a normalized error message for both template
    validation and JSON validation.

    Also Add template validation in api template parser to prevent huge template
    (besides nested stacks) pass to engine.

    Change-Id: If62621d7a6f44100a158974f0fc9c3ddf58d8d99
    Depends-On: I981d477c96345e68baa5a3d96c6671ad242dbb63
    Closes-Bug: 1499379

Changed in heat:
status: In Progress → Fix Released
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/heat 6.0.0.0b3

This issue was fixed in the openstack/heat 6.0.0.0b3 development milestone.

Revision history for this message
Ken'ichi Ohmichi (oomichi) wrote :

This bug seems to be fixed already on both sides.

Changed in tempest:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.