Heat WaitCondition URL points to internalURL Endpoint

[Bjoern]

When using the WaitCondition resource in Heat (RPC 10+) the get_attr: ['wait_handle', 'curl_cli'] } call will return a internal URL as endpoint not a public one. Since those handles are used to trigger handles from inside the instance, the internal IP in such cases is usually not usable.

At this time I recommend to set the token heat_watch_server_uri to a external URL as a default value, in any case.

"{{ heat_watch_proto }}://{{ external_lb_vip_address }}:{{ heat_watch_port }}"

Additional changes might be necessary inside the heat.conf.

I did test alternative options to set the heat watch server URI the URL above and also the [client] section inside the heat.conf to publicURL but it still was using the internal URL endpoints. So we need to investigate this a little more.

Example hot template:
heat_template_version: 2013-05-23

description: >
  HOT template to demonstrate usage of the Heat native waitcondition resources
  This is expected to work with any image containing curl and something which
  runs the raw user-data script, e.g cirros or some image containing cloud-init
parameters:
  key_name:
    type: string
    description: Name of keypair to assign to server
    default: rpc_support
  image:
    type: string
    description: Name of image to use for server
    default: "Ubuntu 14.04.1 LTS Trusty Tahr (cloudimg)"
  flavor:
    type: string
    description: Flavor to use for server
    default: m1.small
  timeout:
    type: number
    description: Timeout for WaitCondition, depends on your image and environment
    default: 120

resources:
  wait_condition:
    type: OS::Heat::WaitCondition
    properties:
      handle: {get_resource: wait_handle}
      # Note, count of 5 vs 6 is due to duplicate signal ID 5 sent below
      count: 5
      timeout: {get_param: timeout}

  wait_handle:
    type: OS::Heat::WaitConditionHandle

  instance1_port0:
    type: OS::Neutron::Port
    properties:
      admin_state_up: true
      network_id: 57942055-c5f5-48fe-b06d-b79aad7b6dd9
      security_groups:
        - rpc-support

 instance1:
    type: OS::Nova::Server
    properties:
      image: {get_param: image}
      flavor: {get_param: flavor}
      key_name: {get_param: key_name}
      user_data_format: RAW
      networks:
         - port: { get_resource: instance1_port0 }
      user_data:
        str_replace:
          template: |
            #!/bin/sh
            # Below are some examples of the various ways signals
            # can be sent to the Handle resource

            # Simple success signal
            wc_notify --data-binary '{"status": "SUCCESS"}'

          params:
            wc_notify: { get_attr: ['wait_handle', 'curl_cli'] }

In such case the call

get_attr: ['wait_handle', 'curl_cli'] }

will create the example curl

curl -i -X POST -H 'X-Auth-Token: xxxxxx' -H 'Content-Type: application/json' -H 'Accept: application/json' http://172.29.236.10:8004/v1/64c92024ae7d4c9b94040f0f99491ef8/stacks/wc-test/d0d15f25-42aa-4389-8586-759c1563cdec/resources/wait_handle/signal --data-binary '{"status": "SUCCESS"}'

[Miguel Grinberg (miguelgrinberg)]

Hey Bjoern,

For the OS::Heat::WaitCondition resource the Heat signal URL is obtained from the catalog. The [client_heat] section in heat.conf has the endpoint_type setting, which determines which of the public, internal, admin endpoints are to be used. The watch server setting is not used in this resource as far as I remember, that is for cloudformation resources.

In master and kilo we have it set as follows:

    [clients_heat]
    endpoint_type = publicURL

But in Juno we have it set to internalURL. Reference: https://github.com/stackforge/os-ansible-deployment/blob/juno/rpc_deployment/roles/heat_common/templates/heat.conf#L55.

My expectation is that you are seeing this problem only in the Juno release. Correct?

[Bjoern (bjoern-t)]

That's actually it. Thanks Miguel, I must have overseen this setting. I did set it to publicUrl and I think we should back port it to publicUrl for the juno branch as well. What does the client section actually do ? Maybe we should revisit our endpoint selection for the heat.conf including the waitserver etc.

[Miguel Grinberg (miguelgrinberg)]

The client configuration in Heat is very confusing. For all the [clients_XX] sections except heat these settings tell Heat how to invoke the other clients on behalf of the user. For these it makes perfect sense to use internalURL, since Heat is going to call the services from the inside.

The [clients_heat] has a different purpose, it tells Heat how to generate callback URLs for several resources, including the watch condition. Since these are intended to be invoked from instances, it has to be set to publicURL.

Heat.conf also has a [clients] section, which provides defaults, for any missing [clients_XX] sections. In Juno the implementation of these defaults in upstream Heat was broken, so we have all the client sections. The fix for this bug is in Kilo, so at that point we switch to a default [clients] that sets internalURL, and only [clients_heat] overrides that default.

This is for heat resources, there are also a bunch of other settings that apply to the AWS compatible resources, including the watch server URL that you mentioned, but that is not the only one. I think we have all correctly set in Kilo+, I will review all of them in Juno and make sure the right ones are set to externalURL.

Hope this helps clarify the settings a bit!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: juno
Review: https://review.openstack.org/186221

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/186221
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=0bd8ee96f4bd04030b6bd874863d194b22122e2f
Submitter: Jenkins
Branch: juno

commit 0bd8ee96f4bd04030b6bd874863d194b22122e2f
Author: Miguel Grinberg <email address hidden>
Date: Wed May 27 16:16:45 2015 -0700

    Use external URLs for heat signal endpoints

    The heat URLs that are given to instances to use as callbacks must use
    externally accessible URLs.

    Change-Id: If9993294b367df03036cdab9701e567665326888
    Closes-Bug: #1459414

[Jesse Pretorius (jesse-pretorius)]

As per the discussion, this only affects juno and the patch has been submitted and merged.