Doesn't properly get keystone endpoint when Keystone is configured to use templated catalog

[Sudarshan]

When using the keystone static catalog file to register endpoints (http://docs.openstack.org/developer/keystone/configuration.html#file-based-service-catalog-templated-catalog), an endpoint registered (correctly) as catalog.region.data_processing.... gets read as "data-processing" by keystone.
Thus, when Sahara looks for an endpoint, it is unable to find one for data_processing.

This causes a problem with the commandline interface and the dashboard.

Keystone seems to be converting underscores to dashes here: https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/templated.py#L47

modifying this line to not perform the replacement seems to work fine for me, but may have unintended consequences.

[Chad Roberts (croberts)]

Is this actually a sahara bug, or might it be more proper (and a more robust fix) to bring this up against keystone?

[Sergey Lukjanov (slukjanov)]

It sounds like we should move to dash instead of underline in Kilo for the endpoint name

[Michael McCune (mimccune)]

agreed with Sergey about moving to a dash.

Also, maybe we should also add a fallback mechanism to check for the underscore replacement by attempting to acquire the endpoint with a dash after failing to find the name with underscore. I realize this may have unintended consequences with there were 2 service names like "data_processing" and "data-processing", but in this case it would fail in Keystone as well.

[Andrew Lazarev (alazarev)]

Agreed about moving to a dash.

I don't think fallback mechanism is possible. Sahara itself doesn't use endpoint type (only in documentation). So, this is mostly for horizon and devstack.

[Andrew Lazarev (alazarev)]

Assigned to Telles per discussion in IRC

[Lin Hua Cheng (lin-hua-cheng)]

Horizon does not perform any translation of the endpoint from the service catalog returned by Keystone.

There is no work required to fix in Horizon, once Keystone is fixed it should work work in Horizon.

[Telles Mota Vidal Nóbrega (tellesmvn)]

Great

[Andrew Lazarev (alazarev)]

@Lin
https://github.com/openstack/horizon/blob/stable/juno/openstack_dashboard/api/sahara.py#L28

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/135454

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/135458

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/135489

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135472
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=5faa22168d47ec380ee527d966db9cc413033544
Submitter: Jenkins
Branch: master

commit 5faa22168d47ec380ee527d966db9cc413033544
Author: Telles Nobrega <email address hidden>
Date: Tue Nov 18 22:01:31 2014 -0300

    Replacing data_processing with data-processing

    Sahara is changing the endpoint name from data_processing to
    data-processing. The Sahara docs are updated here to reflect this
    change.

    Change-Id: Ide6c216f939964621bdea29c5588f83c49aad103
    Partial-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135148
Committed: https://git.openstack.org/cgit/openstack/python-saharaclient/commit/?id=7ecac90599c54654fd20d72b7ec30ec2b7d4220c
Submitter: Jenkins
Branch: master

commit 7ecac90599c54654fd20d72b7ec30ec2b7d4220c
Author: Telles Nobrega <email address hidden>
Date: Mon Nov 17 22:45:14 2014 -0300

    Replacing data_processing with data-processing

    Using data_processing causes a problem when defining the endpoint
    statically in keystone.

    Change-Id: I8fb2be802c4390bdcd49e1722cd36f50781b1feb
    Partial-bug: #1356053

[Marcos Lobo (marcos-fermin-lobo)]

Sudarshan, I'm working in a patch for solve that https://review.openstack.org/#/c/120011/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/138332

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/138332
Committed: https://git.openstack.org/cgit/openstack/python-saharaclient/commit/?id=be164236407d8bc3937292062969d8ea0ba946c0
Submitter: Jenkins
Branch: master

commit be164236407d8bc3937292062969d8ea0ba946c0
Author: Sergey Lukjanov <email address hidden>
Date: Tue Dec 2 15:06:14 2014 +0300

    Set default service_type to "data-processing" in client

    We're using the default value in many places, so, we should switch it
    to the data-processing by default. The fallback mechanism is already
    implemented, so, backward compatibility supported.

    It's needed to make CLI support both endpoints.

    Change-Id: I8818c77a96685c8a557aa6fec77002c93064118e
    Partial-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Lyle (<email address hidden>) on branch: master
Review: https://review.openstack.org/135458
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135454
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=37a06f017ba6ef38159ee65ac25bdd890ccbd102
Submitter: Jenkins
Branch: master

commit 37a06f017ba6ef38159ee65ac25bdd890ccbd102
Author: Telles Nobrega <email address hidden>
Date: Tue Nov 18 07:59:10 2014 -0300

    Add data-processing service for Sahara

    When registering endpoint with data_processing keystone transforms it
    into data-processing. This problem causes sahara to not find the
    endpoint afterwards

    We need to have two endpoints for correct working tempest and horizon
    with devstack. After resolving bug in tempest and horizon need to remove
    old data_processing endpoint

    Change-Id: I97827d23ffe8a1218abd61e76804b918b1b7cbe0
    Partial-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/161191

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/161243

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/161631

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/161243
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=6373b5178f39370613007fcfc39bcf0aba9262eb
Submitter: Jenkins
Branch: master

commit 6373b5178f39370613007fcfc39bcf0aba9262eb
Author: Sergey Reshetnyak <email address hidden>
Date: Wed Mar 4 17:47:52 2015 +0300

    Replacing data_processing with data-processing

    Using data_processing when setting keystone static catalog causes
    Sahara not to find the endpoint afterwards since keystone translates
    data_processing to data-processing

    Change-Id: I959c1bc76eebc4ddb787f872ce7319bd6ba83be8
    Closes-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/161191
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=8c4a2cd1265c86d3493b82cb9d1277a2274f63cf
Submitter: Jenkins
Branch: master

commit 8c4a2cd1265c86d3493b82cb9d1277a2274f63cf
Author: Sergey Reshetnyak <email address hidden>
Date: Wed Mar 4 15:20:38 2015 +0300

    Replacing data_processing with data-processing

    Using data_processing when setting keystone static catalog causes
    Sahara not to find the endpoint afterwards since keystone translates
    data_processing to data-processing.

    Change-Id: I5b253e4f5a0486913c59150d2bafd631f06a8cf1
    Partial-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/161631
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=94c12267f47fb3165a53e4aae8d9515e1df1cca7
Submitter: Jenkins
Branch: stable/juno

commit 94c12267f47fb3165a53e4aae8d9515e1df1cca7
Author: Telles Nobrega <email address hidden>
Date: Tue Nov 18 07:59:10 2014 -0300

    Add data-processing service for Sahara

    When registering endpoint with data_processing Keystone transforms it
    into data-processing. This problem causes sahara to not find the
    endpoint afterwards

    We need to have two endpoint for correct working tempest and horizon
    with devstack. After resolving bug in tempest and horizon need to remove
    old data_processing endpoint

    Change-Id: I97827d23ffe8a1218abd61e76804b918b1b7cbe0
    Partial-bug: #1356053
    (cherry picked from commit 77f82f38496f364b7cc6d2a5bc8b45432934a130)

[Dmitry Mescheryakov (dmitrymex)]

The corresponding bug for fuel is moved to https://bugs.launchpad.net/mos/+bug/1431983

[Ghanshyam Mann (ghanshyammann)]

For DevStack-

>https://github.com/openstack/sahara/blob/master/devstack/plugin.sh#L37

As Tempest publish the "data_processing" catalog type as config option, be careful about removing the 'data_processing' catalog_type support from devstack otherwise it going to break the user using with old default value.

Or we have to have some fallback logic for backward compatibility in Tempest.

[Luigi Toscano (ltoscano)]

If Tempest is patches to use data-processing by default, as I suspect ylobankov is going to do, I don't think it would be a problem to remove setup of the "data_processing" service from devstack. Do you have any concern?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135489
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=6189b0a69505a3a97fc559113694b3b5a6816257
Submitter: Jenkins
Branch: master

commit 6189b0a69505a3a97fc559113694b3b5a6816257
Author: Telles Nobrega <email address hidden>
Date: Tue Nov 18 23:22:17 2014 -0300

    Replacing data_processing with data-processing

    Changing data_processing with data-processing where needed to match the
    new defined endpoint of Sahara.

    Co-Authored-By: Sergey Reshetnyak <email address hidden>
    Co-Authored-By: Yaroslav Lobankov <email address hidden>

    Closes-Bug: #1356053

    Change-Id: Iba45c15ed57d43a11e6fac74d75d6b2db46f6a2f

[Sergey Reshetnyak (sreshetniak)]

affects fuel-puppets and fuel-ostf

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/266316

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/266316
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=fd9501197fd7e16dfb4ab85e919913924b4696b3
Submitter: Jenkins
Branch: master

commit fd9501197fd7e16dfb4ab85e919913924b4696b3
Author: Denis Egorenko <email address hidden>
Date: Tue Jan 12 14:55:33 2016 +0300

    Fix service_type for Sahara

    Currently we are using wrong service_type for Sahara, it was introduced
    because of bug in Tempest in upstream during Kilo cycle and now
    this bug is fixed and we should use proper service_type as well.

    Change-Id: I890cf8bb8e15b568e72f42327c22ed4d852ea6e4
    Closes-bug: #1356053

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/266905

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/266905
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=727572b8fab20da331229d1b3dc57ab5276de8ca
Submitter: Jenkins
Branch: stable/8.0

commit 727572b8fab20da331229d1b3dc57ab5276de8ca
Author: Denis Egorenko <email address hidden>
Date: Tue Jan 12 14:55:33 2016 +0300

    Fix service_type for Sahara

    Currently we are using wrong service_type for Sahara, it was introduced
    because of bug in Tempest in upstream during Kilo cycle and now
    this bug is fixed and we should use proper service_type as well.

    Change-Id: I890cf8bb8e15b568e72f42327c22ed4d852ea6e4
    Closes-bug: #1356053
    (cherry picked from commit fd9501197fd7e16dfb4ab85e919913924b4696b3)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/268958

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/268958
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=309b3115bce2c3c478e2f636a3afafa675505949
Submitter: Jenkins
Branch: master

commit 309b3115bce2c3c478e2f636a3afafa675505949
Author: Denis Egorenko <email address hidden>
Date: Mon Jan 18 12:48:36 2016 +0300

    Set proper service_type for Sahara for sahara template providers

    Change-Id: I918241635749f9a6e8d0b10808627ec963707000
    Related-bug: #1356053
    Closes-bug: #1535089

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/8.0
Review: https://review.openstack.org/269552

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/269552
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=1460127ca23ea8ee1ac1d02ead9ccf99b754edb9
Submitter: Jenkins
Branch: stable/8.0

commit 1460127ca23ea8ee1ac1d02ead9ccf99b754edb9
Author: Denis Egorenko <email address hidden>
Date: Mon Jan 18 12:48:36 2016 +0300

    Set proper service_type for Sahara for sahara template providers

    Change-Id: I918241635749f9a6e8d0b10808627ec963707000
    Related-bug: #1356053
    Closes-bug: #1535089
    (cherry picked from commit 309b3115bce2c3c478e2f636a3afafa675505949)