heat stack-create fails with "Resource CREATE failed: EndpointNotFound: externalURL endpoint for orchestration service not found"

[Mike Metral]

heat stack-create fails with:

"Create_Failed: Resource CREATE failed: EndpointNotFound: externalURL endpoint for orchestration service not found"

this is using OSAD commit 67d6361:

the heat client endpoint is being set here: https://github.com/stackforge/os-ansible-deployment/blob/deaf440546d3396679dc63595202a859c415b54b/playbooks/roles/os_heat/templates/heat.conf.j2#L35 as "externalURL" as defined in https://github.com/stackforge/os-ansible-deployment/blob/7fc8223c2e628567783ee7532329664666f493f2/playbooks/roles/os_heat/defaults/main.yml#L28

if you change it to internalURL instead of externalURL and restart the heat-api then heat stack-creation works.

Talked with shep and he seems to think that it should be externalURL but there is something funky happening with either the LB and/or FW configuration that is not doing the proper nat'ing.

Shep's comments:

[10:24:40 AM] <galstrom> i think the externalURL for that env is the actual Public IP that is on the front side of the LB
[10:24:49 AM] <galstrom> and the VMs all live behind the LB..
[10:25:03 AM] <galstrom> actually the pub is on the front of the FW
[10:25:15 AM] <galstrom> then NAT'd to a RC1918 ip on the front of the LB
[10:25:49 AM] <galstrom> if i were a betting man.. i would assume it is either a LB or FW config issue
[10:26:04 AM] <galstrom> say 1.2.3.4 is routed to the front of the FW
[10:26:12 AM] <galstrom> the FW NAT's that to 10.0.0.4
[10:26:16 AM] <galstrom> that is the VIP on the LB
[10:26:39 AM] <galstrom> your VMs are trying to hit 1.2.3.4 and the FW or LB is telling them no go
[10:26:49 AM] <galstrom> instead of NAT'ing that back for them

[Kevin Carter (kevin-carter)]

My question here would be wether your using a "real" load balancer or our HAProxy role and if you've seen the same behavior in Master/Kilo? I ask because this issue has been brought up before and seems similar to this issue https://bugs.launchpad.net/openstack-ansible/+bug/1432742 . that said maybe Miguel Grinberg will be able to speak to this more as he has been working on heat.

[Mike Metral (mike-metral)]

I've only strictly been working with the developer allinone stacks, but they have been the current master at the time of deployment. the bug also does currently exist as in your dev cluster setup i got to play with last week, but i'm not sure what the architecture of that looks like in terms of the load balancer being real or haproxy.

[Miguel Grinberg (miguelgrinberg)]

The configuration sets the endpoint type as "externalURL", but the correct setting is "publicURL". Fix coming up soon.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: kilo
Review: https://review.openstack.org/177982

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/177982
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=b50a0e07c931fc4dd8bc864c55b29e5232b953be
Submitter: Jenkins
Branch: kilo

commit b50a0e07c931fc4dd8bc864c55b29e5232b953be
Author: Miguel Grinberg <email address hidden>
Date: Mon Apr 27 14:44:53 2015 -0700

    Use publicURL instead of externalURL

    This change corrects an endpoint_type in the heat configuration that was
    incorrectly set to externalURL. The correct endpoint type is publicURL.

    Change-Id: I93db339fc6135adb6f53efd7fe0841d2504b22be
    Closes-Bug: #1449181

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/177981
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=838d42decb3e6ddcdc7bf15e8953020126b1cd25
Submitter: Jenkins
Branch: master

commit 838d42decb3e6ddcdc7bf15e8953020126b1cd25
Author: Miguel Grinberg <email address hidden>
Date: Mon Apr 27 14:44:53 2015 -0700

    Use publicURL instead of externalURL

    This change corrects an endpoint_type in the heat configuration that was
    incorrectly set to externalURL. The correct endpoint type is publicURL.

    Change-Id: I93db339fc6135adb6f53efd7fe0841d2504b22be
    Closes-Bug: #1449181

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.