Stale endpoint selection logic in keystone client

[Alexander Makarov]

In V3 endpoint groups from different regions are not grouped. It results in a problem in "url_for":
it only stores for result endpoints of the last region of all with similar service type.

[Alexander Makarov (amakarov)]

Here is the case reproduced on a cloud deployed by devstack:

CLI commands:

$ export OS_USERNAME=admin
$ export OS_PASSWORD=qwe123
$ export OS_TENANT_NAME=admin
$ export OS_AUTH_URL=http://localhost:5000/v2.0

$ keystone service-create --name=nova --type=compute --description="Second Openstack Compute Service";
$ keystone endpoint-create --region RegionTwo --service-id=73be0c7f663e4823bb93600dc71a3b23 --publicurl='http://172.16.0.30:9292' --internalurl='http://192.168.0.194:9292' --adminurl='http://192.168.0.194:9292'

Python code:

import keystoneclient.v3.client as ksclient

client = ksclient.Client(
            endpoint="http://172.18.10.122:5000/v3")

client.authenticate(
    auth_url="http://172.18.10.122:5000/v3",
    username="admin",
    password="qwe123",
    tenant_name="admin")

heat_url = client.service_catalog.url_for(
    service_type='compute',
    region_name="RegionOne",
    endpoint_type='publicURL')

print heat_url

And the output:

Traceback (most recent call last):
  File "/home/alexander/PycharmProjects/python-keystoneclient/triage.py", line 17, in <module>
    endpoint_type='publicURL')
  File "/home/alexander/PycharmProjects/python-keystoneclient/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/home/alexander/PycharmProjects/python-keystoneclient/keystoneclient/service_catalog.py", line 232, in url_for
    raise exceptions.EndpointNotFound(msg)
keystoneclient.openstack.common.apiclient.exceptions.EndpointNotFound: publicURL endpoint for compute service in RegionOne region not found

[Alexander Makarov (amakarov)]

The problem is: structure returned in 'catalog' field of the response can contains several endpoints with identical type and name for each region, while tests expect single endpoint for all regions with the same type and name.

[Jamie Lennox (jamielennox)]

So yes there is the possibility within keystone server to create multiple services with the same service_type. In this case the service catalog will only check one of the entries for a matching URL.

My opinion here is that keystone should be hardened to make the service_type unique. In your example after you have let devstack run you create another compute service_type and add something new to it. There is already a compute service_type being deployed by devstack and you should add your endpoints to that.

In its current state a cloud deployed with multiple service_types will fail to work so i don't think it is a compatibility issue to shut this down on the server.

[Morgan Fainberg (mdrnstm)]

This is not something the client should care about, if anything keystone needs to enforce this.

[Morgan Fainberg (mdrnstm)]

This looks, feels, and smells like a legitimate bug. However, we will need to look into the possibility that *someone* has been using services with the same "type" and being distinguished by "name". I think it's highly unlikely, and uniqueness should be a constraint.

This is similar to https://bugs.launchpad.net/keystone/+bug/1376937

[Jamie Lennox (jamielennox)]

Alexander, I really want to apologize for marking this invalid and then essentially merging the same thing later with a new bug.

We got some pressure coming down from the infra teams to get this fixed in keystoneclient and when i went looking for this bug i couldn't find it (my mistake i though 'catalog' was mentioned somewhere).

What was merged is almost exactly the same thing you proposed.

I'm marking this a duplicate of the new bug (when it should be the other way around) as the new bug has the merged patch attached.

Again, sorry about this.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alexander Makarov (<email address hidden>) on branch: master
Review: https://review.openstack.org/125923