yaql function breaks due to get_attr returning null

[Steven Hardy]

I know this is part of a more general problem, but I'm trying to figure out the cleanest workaround for this situation:

(undercloud) [stack@undercloud ~]$ cat test.yaml
heat_template_version: pike

resources:

  ServiceChain:
    type: OS::Heat::ResourceChain
    properties:
      resources:
        - Rsrc::One
        - Rsrc::Two
        - Rsrc::Disabled
        - Rsrc::Three
      concurrent: true

# This fails like
# ERROR: Property error: : resources.PuppetStepConfig.properties.value: : No method "zip" for receiver None matches supplied arguments
# Because the get_attr function returns None during validation
  PuppetStepConfig:
    type: OS::Heat::Value
    properties:
      type: string
      value:
        yaql:
          expression:
            # select 'step_config' only from services that do not have a docker_config
            $.data.service_names.zip($.data.step_config, $.data.docker_config).where($[2] = null).where($[1] != null).select($[1]).join("\n")
          data:
            service_names: {get_attr: [ServiceChain, role_data, service_name]}
            step_config: {get_attr: [ServiceChain, role_data, step_config]}
            docker_config: {get_attr: [ServiceChain, role_data, docker_config]}
outputs:
  role_data:
    value: {get_attr: [ServiceChain, role_data]}
  # The same expression here works fine
  debug:
    value:
      yaql:
        expression:
          # select 'step_config' only from services that do not have a docker_config
          $.data.service_names.zip($.data.step_config, $.data.docker_config).where($[2] = null).where($[1] != null).select($[1]).join("\n")
        data:
          service_names: {get_attr: [ServiceChain, role_data, service_name]}
          step_config: {get_attr: [ServiceChain, role_data, step_config]}
          docker_config: {get_attr: [ServiceChain, role_data, docker_config]}

(undercloud) [stack@undercloud ~]$ cat env.yaml
resource_registry:
  Rsrc::One: one.yaml
  Rsrc::Two: two.yaml
  Rsrc::Disabled: OS::Heat::None
  Rsrc::Three: three.yaml
(undercloud) [stack@undercloud ~]$ cat one.yaml
heat_template_version: pike

outputs:
  role_data:
    value:
      service_name: service_one
      step_config: dontuseme
      docker_config: usethis
(undercloud) [stack@undercloud ~]$ cat two.yaml
heat_template_version: pike

outputs:
  role_data:
    value:
      service_name: service_two
      step_config: usethispuppet
(undercloud) [stack@undercloud ~]$ cat three.yaml
heat_template_version: pike

outputs:
  role_data:
    value:
      service_name: service_three
      step_config: usethispuppet2

The issue is the nested get_attr in the data for yaql evaluates to None, and it's not only happening when the function.validate is called, e.g I added this debug:

2017-07-19 08:11:05.079 23710 INFO heat.engine.resource [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] Validating Value "PuppetStepConfig"
2017-07-19 08:11:05.080 23710 WARNING heat.engine.hot.functions [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] SHDEBUG in Yaql.result
2017-07-19 08:11:05.080 23710 WARNING heat.engine.hot.functions [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] SHDEBUG in Yaql.validate
2017-07-19 08:11:05.080 23710 WARNING heat.engine.hot.functions [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] SHDEBUG self._expression=$.data.service_names.zip($.data.step_config, $.data.docker_config).where($[2] = null).where($[1] != null).select($[1]).join("\n")
2017-07-19 08:11:05.080 23710 WARNING heat.engine.hot.functions [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] SHDEBUG self._data={u'service_names': <heat.engine.hot.functions.GetAttAllAttributes {get_attr: [u'ServiceChain', u'role_data', u'service_name']} -> None>, u'step_config': <heat.engine.hot.functions.GetAttAllAttributes {get_attr: [u'ServiceChain', u'role_data', u'step_config']} -> None>, u'docker_config': <heat.engine.hot.functions.GetAttAllAttributes {get_attr: [u'ServiceChain', u'role_data', u'docker_config']} -> None>}
2017-07-19 08:11:05.198 23710 WARNING heat.engine.hot.functions [req-11901086-947b-44d5-98a1-977f4cf0212d - admin - - -] SHDEBUG resolved data={u'service_names': None, u'step_config': None, u'docker_config': None}

We can see the value is validated for the resource, which causes the function evaluation, which means get_attr returns None which breaks the otherwise valid yaql query (see the output, it works fine if you comment the value resource, because we don't strictly validate outputs due to https://bugs.launchpad.net/heat/+bug/1599114)

[David N (david-noriega)]

Running into this as well. Using yaql to get the length of a list and I get the same receiver for None error. What is interesting is when I instead used yaql to check if the data is a list, it returns true.

  project_count:
    type: OS::Heat::Value
    properties:
      value:
        yaql:
          expression: isList($.data.project_list)
          data:
            project_list: { get_attr: [project_list, value] }