Nested stacks are still loaded into memory too often

Bug #1731349 reported by Zane Bitter on 2017-11-09
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
High
Zane Bitter

Bug Description

Loading nested stacks into memory at the same time as the parent stack concentrates memory use into a single engine, and tends to result in a high-water-mark effect. Since Kilo, when we split operations on nested stacks so that they happen over RPC in different engine worker processes, we've been endeavouring to eliminate loading of nested stacks using the StackResource.nested() call, most notably in these patches:

https://review.openstack.org/#/c/384718/
https://review.openstack.org/#/c/383839/

which together reduced TripleO memory usage in the gate by >20%.[1]

However, there remain other places where nested() is still called - many hidden by the monstrosity that is grouputils. The following changes could further improve memory usage:

* StackResource._validate_nested_resources() should use the current template or something - anything - to ascertain the number of resources currently in the stack in preference to loading the nested stack to count them. AFAICT this will result in every nested stack being loaded into memory simultaneously for validation purposes whenever we do a stack update, unless the max_resources_per_stack config option is set to -1. (TripleO *does* set it to -1, but most installations do not.)

* InstanceGroup._replace() and ResourceGroup._replace() (including ResourceGroup._count_black_listed()) should use some other method of determining the current size when doing a rolling update.

* AutoscalingGroup, InstanceGroup, ResourceGroup, and ResourceChain should use outputs from the nested stack (added in Pike by https://review.openstack.org/#/c/475931/) to get attribute values from nested resources, and only fall back on the grouputils functions when the outputs don't exist (which would happen if the stack has not been updated since before Pike).

* ResourceChain should add outputs to the nested stack to get reference IDs for nested resources, and only fall back on the grouputils functions when the outputs don't exist.

* grouputils.get_members() should be reimplemented or replaced with something that does a resource list over RPC to get the list of non-failed child resources. (And get_size() should be reimplemented to just return len(get_members(...)).)

[1] http://lists.openstack.org/pipermail/openstack-dev/2017-January/109748.html

Fix proposed to branch: master
Review: https://review.openstack.org/529715

Changed in heat:
assignee: nobody → Zane Bitter (zaneb)
status: Triaged → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/530968

Fix proposed to branch: master
Review: https://review.openstack.org/530972

Fix proposed to branch: master
Review: https://review.openstack.org/531925

Fix proposed to branch: master
Review: https://review.openstack.org/531926

Fix proposed to branch: master
Review: https://review.openstack.org/531929

Fix proposed to branch: master
Review: https://review.openstack.org/531930

Fix proposed to branch: master
Review: https://review.openstack.org/531931

Zane Bitter (zaneb) wrote :

Looks like a 14% reduction in TripleO memory use to 1.12GiB, with convergence enabled (see https://review.openstack.org/#/c/531931/ for details).

Zane Bitter (zaneb) on 2018-01-09
Changed in heat:
importance: Medium → High

Fix proposed to branch: master
Review: https://review.openstack.org/532301

Fix proposed to branch: master
Review: https://review.openstack.org/532302

Fix proposed to branch: master
Review: https://review.openstack.org/532341

Fix proposed to branch: master
Review: https://review.openstack.org/532571

Zane Bitter (zaneb) on 2018-01-10
Changed in heat:
milestone: none → queens-3

Reviewed: https://review.openstack.org/529715
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e5707618f3a5cf22e2f4260ed1aa64b598d3a941
Submitter: Zuul
Branch: master

commit e5707618f3a5cf22e2f4260ed1aa64b598d3a941
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Avoid always loading nested stack on update

    Previously, when calling StackResource._validate_nested_resources() (which
    we do whenever we create or update a nested stack), we would load the
    nested stack into memory to validate the number of resources in the nested
    stack, unless the max_resources_per_stack config option was set to -1. This
    meant we would load the nested stack into memory in the same engine as the
    parent on every update.

    To reduce the memory high-water mark, fetch the information we need over
    RPC from another engine instead.

    To ensure this is only called once, move the call into the validate code.
    (Previously it was called again in the create/update itself.)

    Change-Id: I78d12ecc8240c697e26893ae2d7172b60883fb93
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/530968
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=ce120bfda848e55ba06b606681e7c1b8ae3de923
Submitter: Zuul
Branch: master

commit ce120bfda848e55ba06b606681e7c1b8ae3de923
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Avoid loading nested stack in some grouputils functions

    We want to avoid loading a nested stack in memory wherever possible, since
    that is known to cause memory high-water-mark issues. The grouputils
    functions are among the worst offenders at doing this. Some of the data
    that they return is easily obtained from an RPC call to
    list_stack_resources, so swap out the implementations using nested().

    Rather than simply add more utility functions, a GroupInspector class is
    created that can cache the data returned. In future this will allow groups
    that need to access multiple functions from the grouputils to do so without
    making multiple RPC calls. (Previously, the data was cached in the group's
    nested Stack.)

    Change-Id: Icd35d91bce30ee36d9592b0516767ef273a9f34d
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/530972
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=b023fa88d1ed7194faf0e54330814f520c7a010e
Submitter: Zuul
Branch: master

commit b023fa88d1ed7194faf0e54330814f520c7a010e
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Don't load nested stack in batched ResourceGroup

    Previously whenever we did a batched operation on a ResourceGroup, we
    loaded the nested stack into memory in the local engine in order to
    determine the current capacity. Change to using the GroupInspector class to
    calculate the capacity using only RPC.

    Change-Id: Ie4c6791bf70df01a66e49cb8ef104e8155c90443
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/531925
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=3ec13d9a8fe1ccf03913ed0a02c720312c13332b
Submitter: Zuul
Branch: master

commit 3ec13d9a8fe1ccf03913ed0a02c720312c13332b
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Fix attribute caching in SoftwareDeploymentGroup

    When the parent template references the 'deploy_stdouts', 'deploy_stderrs',
    or 'deploy_status_codes' (all plural) attributes of
    SoftwareDeploymentGroup, ensure that outputs are generated that reference
    the corresponding 'deploy_stdout', 'deploy_stderr', or 'deploy_status_code'
    (all singular) attributes of the child template's SoftwareDeployment
    resources.

    Change-Id: Iff7101dbaa57602be68e3c7447971dacdd747880
    Closes-Bug: #1741552
    Partial-Bug: #1731349
    Related-Bug: #1660831

Rico Lin (rico-lin) on 2018-01-29
Changed in heat:
milestone: queens-3 → queens-rc1

Reviewed: https://review.openstack.org/530975
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=4a3a1ad15bdcfd67e0548481c70545e41a9476d9
Submitter: Zuul
Branch: master

commit 4a3a1ad15bdcfd67e0548481c70545e41a9476d9
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Fix attribute caching in Heat AutoScalingGroup

    Attributes for an individual resource in an OS::Heat::AutoScalingGroup can
    be retrieved using the attribute name 'resource.<index>'. Because we were
    treating the index as a resource name when generating the template, we
    would never generate an output in the nested template that referenced the
    resource's attribute, so it wouldn't get cached. Since we can't know the
    indices ahead of time, the only way to cache these is to do it for all
    resources in the group - which is the same data cached by the equivalent
    'outputs' attribute.

    We also need to take into account both possible forms of syntax:

        [<asg_name>, resource.<index>.<attr_name>, ...]

    or:

        [<asg_name>, resource.<index>, <attr_name>, ...]

    Change-Id: I1d2898cdd4759b1bb9de1896a40056685e728f44
    Closes-Bug: #1737047
    Partial-Bug: #1731349
    Related-Bug: #1660831

Reviewed: https://review.openstack.org/532301
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=77112137bb368304be2437de09f0224e8e38ab58
Submitter: Zuul
Branch: master

commit 77112137bb368304be2437de09f0224e8e38ab58
Author: Zane Bitter <email address hidden>
Date: Tue Jan 9 11:29:25 2018 -0500

    Fix resource.<n> attribute caching in ResourceGroup/Chain

    When generating the nested template output, we need to take into account
    both forms of syntax:

       get_attr: [<group_name>, resource.<index>.<attr_name>, ...]

    or

       get_attr: [<group_name>, resource.<index>, <attr_name>, ...]

    Previously we would generate an output for the former case but not the
    latter, so the attribute wouldn't get cached when that syntax was used.

    Change-Id: I2a059d5cc42d794ca71caaa1661c32eb76c733fc
    Closes-Bug: #1742185
    Partial-Bug: #1731349
    Related-Bug: #1660831

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/539661

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/539662

Reviewed: https://review.openstack.org/531926
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=7a046e6ffd8daaa21202eeb691d12f2dbf9abfc1
Submitter: Zuul
Branch: master

commit 7a046e6ffd8daaa21202eeb691d12f2dbf9abfc1
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Include outputs in rolling update of ResourceGroup

    The template generated by ResourceGroup should include outputs that
    reference any needed resource attributes. However, previously it only did
    so when rolling updates were not enabled by an update_policy.

    Change-Id: Ice7a92fe8d2b14e2470d9089a8e191c8385994f1
    Closes-Bug: #1741981
    Partial-Bug: #1731349
    Related-Bug: #1660831

Reviewed: https://review.openstack.org/530977
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=adcb72b17bd136d6ce15af8ad3f3fd19f623e013
Submitter: Zuul
Branch: master

commit adcb72b17bd136d6ce15af8ad3f3fd19f623e013
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Supply outputs for reference IDs in StackResources

    Previously, in 1fff9516398e40f126a84a5fd6d2ad261838129b, we added outputs
    to the generated templates of various nested stack resources that
    referenced any attributes required to calculate the parent resource's
    attributes. That was necessary because attribute values are not cached
    unless they are referenced somewhere. It isn't necessary for reference IDs
    (i.e. the results of get_resource) because those are always available.
    However, providing reference IDs as outputs as well will allow us to get
    all attributes of the parent resource from outputs of the nested stack.

    Unlike ResourceGroup/Chain, in InstanceGroup and AutoscalingGroup the order
    and content of the member in list-type outputs cannot be determined at
    template generation time (failed resources are excluded, and the sort order
    depends on creation time). Therefore, always use dicts instead of lists as
    outputs, both for the new reference ID outputs in AutoscalingGroup and
    existing attribute outputs. For ResourceGroup/Chain, create both dicts and
    lists as appropriate, so that each attribute value can be obtained directly
    from an output.

    Change-Id: I3139246a5acb17df9018e7c7a2cf88f740b3f8be
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/531929
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=ac87bc7c7932b9a0245d12a9a46b6fbad42be9b5
Submitter: Zuul
Branch: master

commit ac87bc7c7932b9a0245d12a9a46b6fbad42be9b5
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Get ResourceGroup/Chain attributes from nested stack outputs

    Since Pike we've generated the templates for various nested stack resource
    types with outputs that produce the attribute values that are referenced in
    the parent stack, and a previous patch did the same for resource IDs. Use
    these output values when available to calculate the attribute values. This
    is efficient, because the all of the outputs are fetched together and
    cached, and avoids using the grouputils functions that cause the nested
    stack to get loaded into memory in the same process.

    Fall back to the existing implementation (using grouputils) if the required
    output isn't available (generally this would be due to the nested stack
    being created on an earlier version of Heat that did not include the
    outputs in the generated template, and not updated since).

    Change-Id: I9146e1d99f494213daef7d61ae75c92a4ef981a9
    Partial-Bug: #1731349

Rico Lin (rico-lin) on 2018-02-09
Changed in heat:
milestone: queens-rc1 → rocky-1

Reviewed: https://review.openstack.org/531930
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=9f9605d927d89b1b9041fee6b1fad7a73481eec6
Submitter: Zuul
Branch: master

commit 9f9605d927d89b1b9041fee6b1fad7a73481eec6
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Get Instance/AutoScalingGroup attributes from nested stack outputs

    Since Pike we've generated the templates for various nested stack resource
    types with outputs that produce the attribute values that are referenced in
    the parent stack, and a previous patch did the same for resource IDs. Use
    these output values when available to calculate the attribute values. This
    is efficient, because the all of the outputs are fetched together and
    cached, and avoids using the grouputils functions that cause the nested
    stack to get loaded into memory in the same process.

    Fall back to the existing implementation (using grouputils) if the required
    output isn't available (generally this would be due to the nested stack
    being created on an earlier version of Heat that did not include the
    outputs in the generated template, and not updated since).

    The InstanceGroup and AutoScalingGroup implementations are more complex
    than those of ResourceGroup and ResourceChain because they exclude failed
    resources at runtime, and also must have a stable sort order for list-type
    outputs that cannot be determined at template generation time. Therefore
    the underlying output value is always a dict, and the attribute value is
    calculated from it.

    Change-Id: Ie4ad85068fdc16b8038a09df709ed7eb86434cfa
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/531931
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=32ec5141de1a8017ccfddc5dccf114deff8271ba
Submitter: Zuul
Branch: master

commit 32ec5141de1a8017ccfddc5dccf114deff8271ba
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Don't load nested stack to get ResourceGroup blacklist

    In ResourceGroup, we (unwisely in retrospect) allow blacklisting of
    resources by reference ID (i.e. physical resource ID in most cases). In
    order to both determine whether a removal policy entry was an existing
    resource name and (if not) to find the resource by physical ID, we would
    always load the nested stack into memory.

    To avoid this, unconditionally create an output in the nested stack that
    returns a mapping of all the resource reference IDs and use it for finding
    a resource with a matching reference ID. Only if the output is not
    available (because the nested stack hasn't been updated since prior to the
    output being defined), fall back to the old way of doing it.

    Avoid looking up IDs for names that already appear in the blacklist
    (because they were blacklisted in a previous update and have been removed
    from the stack.)

    Since this is more expensive in time (though hopefully cheaper in space),
    update the blacklist only once on resource creation and whenever it
    actually changes during an update. This is accomplished by moving the
    update to a separate function that is called explicitly, rather than making
    it a side-effect of getting the current blacklist (which occurs up to 4
    times in each create/update).

    Change-Id: Ic32d7d99bad06f92b3d2919d58cd1f9128bcfe83
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/545360
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=cbcc15b4046d456032441105941a4a4ea93b48e6
Submitter: Zuul
Branch: stable/queens

commit cbcc15b4046d456032441105941a4a4ea93b48e6
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Don't load nested stack to get ResourceGroup blacklist

    In ResourceGroup, we (unwisely in retrospect) allow blacklisting of
    resources by reference ID (i.e. physical resource ID in most cases). In
    order to both determine whether a removal policy entry was an existing
    resource name and (if not) to find the resource by physical ID, we would
    always load the nested stack into memory.

    To avoid this, unconditionally create an output in the nested stack that
    returns a mapping of all the resource reference IDs and use it for finding
    a resource with a matching reference ID. Only if the output is not
    available (because the nested stack hasn't been updated since prior to the
    output being defined), fall back to the old way of doing it.

    Avoid looking up IDs for names that already appear in the blacklist
    (because they were blacklisted in a previous update and have been removed
    from the stack.)

    Since this is more expensive in time (though hopefully cheaper in space),
    update the blacklist only once on resource creation and whenever it
    actually changes during an update. This is accomplished by moving the
    update to a separate function that is called explicitly, rather than making
    it a side-effect of getting the current blacklist (which occurs up to 4
    times in each create/update).

    This also avoids updating the blacklist during a preview, which the
    previous code erroneously did.

    Change-Id: Ic32d7d99bad06f92b3d2919d58cd1f9128bcfe83
    Partial-Bug: #1731349
    Closes-Bug: #1749426
    (cherry picked from commit 32ec5141de1a8017ccfddc5dccf114deff8271ba)

tags: added: in-stable-queens

Reviewed: https://review.openstack.org/539659
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=cabb3b965c3358a0051facf1372aef6a679d7367
Submitter: Zuul
Branch: stable/pike

commit cabb3b965c3358a0051facf1372aef6a679d7367
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Fix attribute caching in SoftwareDeploymentGroup

    When the parent template references the 'deploy_stdouts', 'deploy_stderrs',
    or 'deploy_status_codes' (all plural) attributes of
    SoftwareDeploymentGroup, ensure that outputs are generated that reference
    the corresponding 'deploy_stdout', 'deploy_stderr', or 'deploy_status_code'
    (all singular) attributes of the child template's SoftwareDeployment
    resources.

    Change-Id: Iff7101dbaa57602be68e3c7447971dacdd747880
    Closes-Bug: #1741552
    Partial-Bug: #1731349
    Related-Bug: #1660831
    (cherry picked from commit 3ec13d9a8fe1ccf03913ed0a02c720312c13332b)

tags: added: in-stable-pike

Reviewed: https://review.openstack.org/539661
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=07382d99d1b4dcf5697c88e43b8f81f647965d3c
Submitter: Zuul
Branch: stable/pike

commit 07382d99d1b4dcf5697c88e43b8f81f647965d3c
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Fix attribute caching in Heat AutoScalingGroup

    Attributes for an individual resource in an OS::Heat::AutoScalingGroup can
    be retrieved using the attribute name 'resource.<index>'. Because we were
    treating the index as a resource name when generating the template, we
    would never generate an output in the nested template that referenced the
    resource's attribute, so it wouldn't get cached. Since we can't know the
    indices ahead of time, the only way to cache these is to do it for all
    resources in the group - which is the same data cached by the equivalent
    'outputs' attribute.

    We also need to take into account both possible forms of syntax:

        [<asg_name>, resource.<index>.<attr_name>, ...]

    or:

        [<asg_name>, resource.<index>, <attr_name>, ...]

    Change-Id: I1d2898cdd4759b1bb9de1896a40056685e728f44
    Closes-Bug: #1737047
    Partial-Bug: #1731349
    Related-Bug: #1660831
    (cherry picked from commit 4a3a1ad15bdcfd67e0548481c70545e41a9476d9)

Reviewed: https://review.openstack.org/539662
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=e94e90b4f2f787cc4830ff67897df8464b994f92
Submitter: Zuul
Branch: stable/pike

commit e94e90b4f2f787cc4830ff67897df8464b994f92
Author: Zane Bitter <email address hidden>
Date: Tue Jan 9 11:29:25 2018 -0500

    Fix resource.<n> attribute caching in ResourceGroup/Chain

    When generating the nested template output, we need to take into account
    both forms of syntax:

       get_attr: [<group_name>, resource.<index>.<attr_name>, ...]

    or

       get_attr: [<group_name>, resource.<index>, <attr_name>, ...]

    Previously we would generate an output for the former case but not the
    latter, so the attribute wouldn't get cached when that syntax was used.

    Change-Id: I2a059d5cc42d794ca71caaa1661c32eb76c733fc
    Closes-Bug: #1742185
    Partial-Bug: #1731349
    Related-Bug: #1660831
    (cherry picked from commit 77112137bb368304be2437de09f0224e8e38ab58)

Reviewed: https://review.openstack.org/541763
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=1cb985536c6b208b2a9616f6ac9083570bb786d8
Submitter: Zuul
Branch: stable/pike

commit 1cb985536c6b208b2a9616f6ac9083570bb786d8
Author: Zane Bitter <email address hidden>
Date: Mon Jan 8 17:23:12 2018 -0500

    Include outputs in rolling update of ResourceGroup

    The template generated by ResourceGroup should include outputs that
    reference any needed resource attributes. However, previously it only did
    so when rolling updates were not enabled by an update_policy.

    The tests are trivially different than the patch on master, only because
    ce120bfda848e55ba06b606681e7c1b8ae3de923 is not present in Pike.

    Change-Id: Ice7a92fe8d2b14e2470d9089a8e191c8385994f1
    Closes-Bug: #1741981
    Partial-Bug: #1731349
    Related-Bug: #1660831
    (cherry picked from commit 7a046e6ffd8daaa21202eeb691d12f2dbf9abfc1)

Reviewed: https://review.openstack.org/532302
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=42b386d8a69a60259e4803755810b945ef81b0d5
Submitter: Zuul
Branch: master

commit 42b386d8a69a60259e4803755810b945ef81b0d5
Author: Zane Bitter <email address hidden>
Date: Tue Dec 19 16:35:46 2017 -0500

    Eliminate nested stack loading in InstanceGroup/ASG scaling

    Obtain the current group capacity, and the reference IDs for updating load
    balancers (if any) over RPC from a GroupInspector and the nested stack
    outputs. Fall back to grouputils (which loads the nested stack in memory)
    only if the output containing the reference IDs is not available.

    Change-Id: Ic6909e42edf709ba1cd99ba71f2d2f06570a615f
    Closes-Bug: #1731349

Changed in heat:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/532341
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=94b62e591077b22738e911f9010cbbdbfebb3d22
Submitter: Zuul
Branch: master

commit 94b62e591077b22738e911f9010cbbdbfebb3d22
Author: Zane Bitter <email address hidden>
Date: Tue Jan 9 16:45:46 2018 -0500

    Avoid loading nested stack in CloudFormation Stack resource

    We can obtain the reference ID of an AWS::CloudFormation::Stack resource
    much more cheaply by not calling nested().

    Change-Id: I212aaae5c68b90b72c7d693f658c83693cc07bf4
    Partial-Bug: #1731349

Reviewed: https://review.openstack.org/532571
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=c6b0d58888b8f451bb8197871bbec8119f1f2632
Submitter: Zuul
Branch: master

commit c6b0d58888b8f451bb8197871bbec8119f1f2632
Author: Zane Bitter <email address hidden>
Date: Wed Jan 10 11:22:49 2018 -0500

    Don't load nested stack to get TemplateResource template

    In the worst-case scenario if we can't find the template for an existing
    TemplateResource, we fetch it from the extant nested stack. Rather than do
    this by loading the nested stack in-memory, fetch the template via RPC.

    Change-Id: Ide96ef861ef4cbb6afd9ab09e7b51b69e03e5520
    Partial-Bug: #1731349

This issue was fixed in the openstack/heat 11.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers