[RFE] Unable to dynamically add endpoints into EndpointMap

[Alex Schultz]

Today EndpointMap is an input that is used to generate all the endpoints, urls, etc for the cloud. Currently we don't have any way to add additional endpointmap entries without having to provide the entire list. This can be seen in the ssl environment files that essentially change the default EndpointMap value. We have EndpointMapOverride, however this is an override for the generated EndpointMap list so when you provide a value here, it assumes it has all the expected values in it.

The use case is a 3rd party would like to provide their endpoints but not have to carry a complete list for the normal entries and all the tls varients.

[Bogdan Dobrelya (bogdando)]

Having that "apply" semantics additionally to the only "update" option that we have today would be in general very nice to have for all heat value resources tripleo relies on

[Bogdan Dobrelya (bogdando)]

This goal probably can be achieved via a custom tripleo-ansible action plugin that fetches the heat value (EndpointMap for this particular issue) then desides how to merge the data in it with the given EndpointMapOverride and provided merge strategy. Than it updates the value in Heat.

[Bogdan Dobrelya (bogdando)]

this also requires using Heat as a lib to fetch JSON values off it. Running CLI inside of action plugins to parse stdout as a json sounds not the best solution to me.

[Bogdan Dobrelya (bogdando)]

As I got that feature request, the ask is to have it working like updating some {k1: v2: {subk2: subv2} } and have that appended to existing {k1: v2: {subk1: ...}} if any?

Cuz currently you can only merge those and there is no deep_merge for such updates

[Rabi Mishra (rabi)]

I'm not sure about the usecase here, but we can override one endpoint like[1] or add an additional endpoint as we do map_merge[2].

[1]
parameter_defaults:
  EndpointMapOverride:
    AodhAdmin:
      host: 172.16.13.50
      host_nobrackets: 172.16.13.50
      port: 8042
      protocol: http
      uri: http://172.16.13.50:8042
      uri_no_suffix: http://172.16.13.50:8042

[2] https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L604-L606

[Alex Schultz (alex-schultz)]

Yea the issue is the host/uri. If we're integrating with tripleo, the vip info is generated by heat so a 3rd party won't know these values at deploy time. You can't dynamically add an additional endpoint without specifying the entire EndpointMap. So if you have two endpoints for two 3rd parties, it's on the end user to merge all the values. What we need is something similar to the firewall rules/keystone resources/etc where the endpoint map information is gathered based on the services being deployed rather than specifying all values all the time. In order to correct this we need to significantly change how we handle the EndpointMap and ServiceNetMaps

[Bogdan Dobrelya (bogdando)]

> we need to significantly change how we handle the EndpointMap and ServiceNetMaps

hm indeed. I'm not sure now if the way I described earlier would work. Since updating Heat Value runtime after having it deep merged with a new value locally in the ansible action plugin, would be possible.

If doing all that server-side, parameter_merge_strategies would do deep_merge for us. But I'm not sure that would be any different to "significantly changing how we handle the EndpointMap and ServiceNetMaps"...

[Bogdan Dobrelya (bogdando)]

Another possible approach is to merge the extra endpoints into the map extracted from Heat for config-download (ansible inventory vars)

[Rabi Mishra (rabi)]

> As I got that feature request, the ask is to have it working like updating some {k1: v2: {subk2: subv2} } and have that appended to existing {k1: v2: {subk1: ...}} if any?

deep-merge would make it pretty complex and difficult interface for the users to understand and surely would mess it up more often. It's better if we can make it simpler how to patch host/port for endpoints.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/796137

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/796137
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/a263da138fe575fef0dddb6db21782740b435b0f
Submitter: "Zuul (22348)"
Branch: master

commit a263da138fe575fef0dddb6db21782740b435b0f
Author: ramishra <email address hidden>
Date: Sat Jun 12 14:45:30 2021 +0530

    Use merge strategy for EndpointMap

    Now someone can override individual endpoints like
    below in an environment:

    parameter_merge_strategies:
      EndpointMap: merge

    parameter_defaults:
      EndpointMap:
        AodhPublic:
          protocol: http
          port: 9999
          host: 121.21.2.1

    Closes-Bug: #1897592
    Change-Id: Ibf2e0d183dd51421c4feb7467c3c01fb416d2965

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.