Some intrinsic functions fail when referencing get_attr

Bug #1559807 reported by Dan Prince on 2016-03-21
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Triaged
Medium
Rico Lin

Bug Description

The following sample stack fails:

heat_template_version: 2016-04-08

parameters:
  ControllerServices:
    default:
      - OS::TripleO::Services::Keystone
    type: comma_delimited_list

resources:

  GoldChain:
    type: OS::Heat::ResourceChain
    properties:
      resources: {get_param: ControllerServices}
      resource_properties:
        Foo: bar

  Controller:
    type: OS::Heat::ResourceGroup
    depends_on: GoldChain
    properties:
      count: 1
      resource_def:
        type: foo.yaml
        properties:
          Foo: {map_merge: {get_attr: [GoldChain, role_data, config_settings]}}

----

The error messages shows up as: ERROR: Property error: : resources.Controller.properties.resource_def: : Incorrect arguments to "map_merge" should be: "map_merge" : [ { "key1": "val1" }, { "key2": "val2" } ]

----

I've tried a similar stack and skipping the resource group and instead doing the same merge_map in an output and it seems to work fine.

Steven Hardy (shardy) on 2016-03-21
Changed in heat:
assignee: nobody → Steven Hardy (shardy)
tags: added: tripleo

Looks like this is a bug in the map_merge implementation, we need to handle referenced to functions at the validate stage, both referencing a list and each item in the list referencing a function.

summary: - merge_map fails to merge resource chains in a resource_group definition
+ map_merge fails to merge resource chains in a resource_group definition
Changed in heat:
status: New → Confirmed
importance: Undecided → Medium
Steven Hardy (shardy) wrote :

Looks like this is a bug in the map_merge implementation, we need to handle referenced to functions at the validate stage, both referencing a list and each item in the list referencing a function, working on a patch.

Steven Hardy (shardy) wrote :

So, after further investigation, this is actually a much more general problem.

Basically get_attr returns None in the case where a resource is not yet created:

https://github.com/openstack/heat/blob/master/heat/engine/hot/functions.py#L163

Which then breaks the value validation when we validate properties that indirectly reference get_attr via an intrinsic function that validates the data it's passed (because we actually call the function "result" function, not only "validate" when we validate, because of this call:

https://github.com/openstack/heat/blob/master/heat/engine/resource.py#L1333

Essentially the strict_validate (which turns on value validation for properties, which triggers the call to get the function result), is enabled for all non-nested stacks, so we're trying to resolve the functions too early in those cases.

I'm not sure of the cleanest solution - there are several functions where "None" isn't a valid input, so perhaps we should never validate values at the resource.validate_template stage.

Steven Hardy (shardy) wrote :

Another example which shows breakage unrelated to map_merge or ResourceGroup:

heat_template_version: 2016-04-08
resources:
  foo:
    type: foo_list.yaml

  server:
    type: OS::Nova::Server
    properties:
      image: cirros-0.3.4-x86_64-uec
      flavor: m1.small
      metadata:
        # Neither of these work, fail with validation errors
        map_merge: {get_attr: [foo, foo]}
        foo: {list_join: {get_attr: [foo, foo]}}

foo_list.yaml is:

heat_template_version: 2015-04-30
outputs:
  foo:
    value: [{"foo": 123}]

summary: - map_merge fails to merge resource chains in a resource_group definition
+ Some intrinsic functions fail when referencing get_attr
Steven Hardy (shardy) wrote :

I think that preview updates are broken for the same reason, we compare the bfore/after properties and the previewed stack attributes are all None.

Steven Hardy (shardy) wrote :

ML thread started looking for additional input on this:

http://lists.openstack.org/pipermail/openstack-dev/2016-March/090241.html

Rico Lin (rico-lin) wrote :

As discussed in newton summit https://etherpad.openstack.org/p/heat-newton-validation-improvements
We possibly go with place holder for further solving this issue.
I'm now digging on this and test on some poc develop for this isse.

Rico Lin (rico-lin) on 2016-10-05
tags: added: heat-validate

Fix proposed to branch: master
Review: https://review.openstack.org/387463

Changed in heat:
assignee: Steven Hardy (shardy) → Rico Lin (rico-lin)
status: Confirmed → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/387464

Fix proposed to branch: master
Review: https://review.openstack.org/387465

Fix proposed to branch: master
Review: https://review.openstack.org/387466

Fix proposed to branch: master
Review: https://review.openstack.org/387467

Fix proposed to branch: master
Review: https://review.openstack.org/387468

Fix proposed to branch: master
Review: https://review.openstack.org/387469

Fix proposed to branch: master
Review: https://review.openstack.org/387470

Fix proposed to branch: master
Review: https://review.openstack.org/387471

Fix proposed to branch: master
Review: https://review.openstack.org/387472

Fix proposed to branch: master
Review: https://review.openstack.org/387473

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387465
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387463
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387467
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387469
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387470
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387471
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387473
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387466
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387468
Reason: Just a test patch, not going to be final solution

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387472

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387464

Rico Lin (rico-lin) on 2017-09-11
Changed in heat:
milestone: none → queens-1
Rico Lin (rico-lin) on 2017-10-25
Changed in heat:
milestone: queens-1 → queens-2
Zane Bitter (zaneb) on 2017-11-29
Changed in heat:
status: In Progress → Triaged
Rico Lin (rico-lin) on 2017-12-06
Changed in heat:
milestone: queens-2 → queens-3
Rico Lin (rico-lin) on 2018-01-29
Changed in heat:
milestone: queens-3 → rocky-1
Rico Lin (rico-lin) on 2018-04-20
Changed in heat:
milestone: rocky-1 → rocky-2
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers