Some intrinsic functions fail when referencing get_attr

[Dan Prince]

The following sample stack fails:

heat_template_version: 2016-04-08

parameters:
  ControllerServices:
    default:
      - OS::TripleO::Services::Keystone
    type: comma_delimited_list

resources:

  GoldChain:
    type: OS::Heat::ResourceChain
    properties:
      resources: {get_param: ControllerServices}
      resource_properties:
        Foo: bar

  Controller:
    type: OS::Heat::ResourceGroup
    depends_on: GoldChain
    properties:
      count: 1
      resource_def:
        type: foo.yaml
        properties:
          Foo: {map_merge: {get_attr: [GoldChain, role_data, config_settings]}}

----

The error messages shows up as: ERROR: Property error: : resources.Controller.properties.resource_def: : Incorrect arguments to "map_merge" should be: "map_merge" : [ { "key1": "val1" }, { "key2": "val2" } ]

----

I've tried a similar stack and skipping the resource group and instead doing the same merge_map in an output and it seems to work fine.

[Steven Hardy (shardy)]

Looks like this is a bug in the map_merge implementation, we need to handle referenced to functions at the validate stage, both referencing a list and each item in the list referencing a function.

[Steven Hardy (shardy)]

Looks like this is a bug in the map_merge implementation, we need to handle referenced to functions at the validate stage, both referencing a list and each item in the list referencing a function, working on a patch.

[Steven Hardy (shardy)]

So, after further investigation, this is actually a much more general problem.

Basically get_attr returns None in the case where a resource is not yet created:

https://github.com/openstack/heat/blob/master/heat/engine/hot/functions.py#L163

Which then breaks the value validation when we validate properties that indirectly reference get_attr via an intrinsic function that validates the data it's passed (because we actually call the function "result" function, not only "validate" when we validate, because of this call:

https://github.com/openstack/heat/blob/master/heat/engine/resource.py#L1333

Essentially the strict_validate (which turns on value validation for properties, which triggers the call to get the function result), is enabled for all non-nested stacks, so we're trying to resolve the functions too early in those cases.

I'm not sure of the cleanest solution - there are several functions where "None" isn't a valid input, so perhaps we should never validate values at the resource.validate_template stage.

[Steven Hardy (shardy)]

Another example which shows breakage unrelated to map_merge or ResourceGroup:

heat_template_version: 2016-04-08
resources:
  foo:
    type: foo_list.yaml

  server:
    type: OS::Nova::Server
    properties:
      image: cirros-0.3.4-x86_64-uec
      flavor: m1.small
      metadata:
        # Neither of these work, fail with validation errors
        map_merge: {get_attr: [foo, foo]}
        foo: {list_join: {get_attr: [foo, foo]}}

foo_list.yaml is:

heat_template_version: 2015-04-30
outputs:
  foo:
    value: [{"foo": 123}]

[Steven Hardy (shardy)]

I think that preview updates are broken for the same reason, we compare the bfore/after properties and the previewed stack attributes are all None.

[Steven Hardy (shardy)]

ML thread started looking for additional input on this:

http://lists.openstack.org/pipermail/openstack-dev/2016-March/090241.html

[Rico Lin (rico-lin)]

As discussed in newton summit https://etherpad.openstack.org/p/heat-newton-validation-improvements
We possibly go with place holder for further solving this issue.
I'm now digging on this and test on some poc develop for this isse.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387463

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387464

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387465

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387466

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387467

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387468

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387469

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387470

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387471

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387472

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387473

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387465
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387463
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387467
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387469
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387470
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387471
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387473
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387466
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387468
Reason: Just a test patch, not going to be final solution

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387472

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rico Lin (<email address hidden>) on branch: master
Review: https://review.openstack.org/387464