v3 service catalog returns services without names, but v2.0 api does not

[David J Hu]

For services without names, it appeared that v2.0 api filtered out these services (w/o names) from the service catalog. To the contrary, v3 api presents these services in the service catalog (w/o names) without filtering them out, which I think is a bug.

Here the dump...

v3
==

$ curl -s -H "Content-Type: application/json" -d '{ "auth": {"identity": {"methods": ["password"],"password": {"user": {"name": "admin","domain": { "id": "default" },"password": "changemenow"}}}}}' http://192.0.2.27:5000/v3/auth/tokens | python -mjson.tool
{
    "token": {
        "audit_ids": [
            "ZVXOhZIzQ_2TRl3FHXs7uQ"
        ],
        "catalog": [
            { <== Service without a name
                "endpoints": [],
                "id": "03759dde22f0498f91ab92fe63da7e37",
                "type": "service-test"
            },
            {
                "endpoints": [
                    {
                        "id": "89689c655c194cd1bf2894b798d4fe60",
                        "interface": "public",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:5000/v2.0"
                    },
                    {
                        "id": "927e31c648c24e75b5c28e86da249c37",
                        "interface": "internal",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:5000/v2.0"
                    },
                    {
                        "id": "d553b1bebbf84efc96507a5fc353df36",
                        "interface": "admin",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:35357/v2.0"
                    }
                ],
                "id": "1af4a779d5c6444d885521e1697b3cde",
                "name": "keystone",
                "type": "identity"
            },
            {
                "endpoints": [
                    {
                        "id": "75db190e2e3f486c86a948947f1256a9",
                        "interface": "admin",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:8080/v1"
                    },
                    {
                        "id": "826f8098256d46e6b89075b63ccb9a3b",
                        "interface": "internal",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:8080/v1/AUTH_edf891223bac4b6ea2a85a15e6ce9cd3"
                    },
                    {
                        "id": "a9bc3e3ac7c44882aa1f02b0c2940fc8",
                        "interface": "public",
                        "region": "regionOne",
                        "region_id": "regionOne",
                        "url": "http://192.0.2.27:8080/v1/AUTH_edf891223bac4b6ea2a85a15e6ce9cd3"
                    }
                ],
                "id": "301ec05f525d40aaa41be512f820d19a",
                "name": "swift",
                "type": "object-store"
            },
            { <== Another service without a name.
                "endpoints": [],
                "id": "5e11382bdd854179b5c3c19f848bf64f",
                "type": "service-test"
            },

* ommited*

v2.0
====

$ curl -s -H "Content-Type: application/json" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "changeyourpasswordnow"}}}' http://192.0.2.27:5000/v2.0/tokens | python -mjson.tool
{
    "access": {
        "metadata": {
            "is_admin": 0,
            "roles": [
                "9fe2ff9ee4384b1894a90878d3e92bab",
                "ab13600a7b1f4beab319d0b6860db24b"
            ]
        },
        "serviceCatalog": [
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8774/v2/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "id": "1f999cd0633d409cb477c9343cf747e6",
                        "internalURL": "http://192.0.2.27:8774/v2/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "publicURL": "http://192.0.2.27:8774/v2/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "nova",
                "type": "compute"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:9696/",
                        "id": "9a8c06f2ffa64e49b0ee078d4cf9a1e8",
                       "internalURL": "http://192.0.2.27:9696/",
                        "publicURL": "http://192.0.2.27:9696/",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
               "name": "neutron",
                "type": "network"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8774/v3",
                        "id": "54fd6e6b246545eba9cf9b0b9abcf91b",
                        "internalURL": "http://192.0.2.27:8774/v3",
                        "publicURL": "http://192.0.2.27:8774/v3",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "nova",
                "type": "computev3"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:9292/",
                        "id": "a9689bd7b565415da89c2148182d1eae",
                        "internalURL": "http://192.0.2.27:9292/",
                        "publicURL": "http://192.0.2.27:9292/",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "glance",
                "type": "image"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:21131/v1",
                        "id": "517e691d44fe4a89a587671cd2fb184e",
                        "internalURL": "http://192.0.2.27:21131/v1",
                        "publicURL": "http://192.0.2.27:21131/v1",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "sherpa",
                "type": "hp-catalog"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8777/",
                        "id": "056f62f042cd4848a4d0d31730999996",
                        "internalURL": "http://192.0.2.27:8777/",
                        "publicURL": "http://192.0.2.27:8777/",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "ceilometer",
                "type": "metering"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8776/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "id": "5f00c3084e1343c1ba21dbb6119ed569",
                        "internalURL": "http://192.0.2.27:8776/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "publicURL": "http://192.0.2.27:8776/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "cinder",
                "type": "volume"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8773/services/Admin",
                        "id": "781bea5f4d534c5b9a0118c3e10d0bac",
                        "internalURL": "http://192.0.2.27:8773/services/Cloud",
                        "publicURL": "http://192.0.2.27:8773/services/Cloud",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "ec2",
                "type": "ec2"
            },
            {
                "endpoints": [
                   {
                        "adminURL": "http://192.0.2.27:8004/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "id": "527eb678746c4b858a945490ed5fac19",
                        "internalURL": "http://192.0.2.27:8004/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "publicURL": "http://192.0.2.27:8004/v1/edf891223bac4b6ea2a85a15e6ce9cd3",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "heat",
                "type": "orchestration"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:8080/v1",
                        "id": "75db190e2e3f486c86a948947f1256a9",
                        "internalURL": "http://192.0.2.27:8080/v1/AUTH_edf891223bac4b6ea2a85a15e6ce9cd3",
                        "publicURL": "http://192.0.2.27:8080/v1/AUTH_edf891223bac4b6ea2a85a15e6ce9cd3",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "swift",
                "type": "object-store"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://192.0.2.27:35357/v2.0",
                        "id": "89689c655c194cd1bf2894b798d4fe60",
                        "internalURL": "http://192.0.2.27:5000/v2.0",
                        "publicURL": "http://192.0.2.27:5000/v2.0",
                        "region": "regionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "keystone",
                "type": "identity"
            }
        ],
        "token": {
            "audit_ids": [
                "gs6MPh1XRxuo9PIoxmtvjw"
            ],
            "expires": "2014-11-12T22:04:32Z",
            "id": "b96d96ad5bd74b4889c4a2cef1966441",
            "issued_at": "2014-11-12T18:04:32.715281",
            "tenant": {
                "description": null,
                "enabled": true,
                "id": "edf891223bac4b6ea2a85a15e6ce9cd3",
                "name": "admin"
            }
        },
        "user": {
            "id": "832ac2ef7983406294aa45b830e30028",
            "name": "admin",
            "roles": [
                {
                    "name": "_member_"
                },
                {
                    "name": "admin"
                }
            ],
            "roles_links": [],
            "username": "admin"
        }
    }
}

[Lance Bragstad (lbragstad)]

According to some of the examples in the API spec [1], it looks like name should be included in the v3 catalog response. It would appear that the documentation is inconsistent with the implementation.

[1] https://github.com/openstack/keystone-specs/blob/master/api/v3/identity-api-v3.rst#catalog-opt-out

[Dolph Mathews (dolph)]

Subscribed Jamie, because I thought he fixed this already? Does something need to be backported?

[Jamie Lennox (jamielennox)]

Ok, so I talked to ling. It seems the problem here is a mismatch between what our API dictates and what our server expects.

The API says that 'name' is an optional field when creating a service. The v2 implementation requires a name field when building the catalog, if it's not present it will blow up with a KeyError. The keystone CLI and openstack client both have name as a required field, so ling has had to create a service using curl or similar directly to have found this.

V3 is slightly smarter, if the service name is not available then it will exclude the name field from the catalog (not include it with an empty value).

Given that v3 works with this situation I don't think we can change the API (though i really don't think it would impact anyone).

Recommend that name should always be included in the service in the catalog, default to either None or empty string if not set by the user.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/135808

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/135808
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=ca3df19da46e330cd9ce1de77224452177f06071
Submitter: Jenkins
Branch: master

commit ca3df19da46e330cd9ce1de77224452177f06071
Author: lin-hua-cheng <email address hidden>
Date: Wed Nov 19 17:32:56 2014 -0800

    Always return the service name in the catalog

    Fix implemention of service catalog creation in v2, to handle the case
    when the service name is not available. Using service.get('name', '')
    does not work because that doesn't invoke the __getitem__() from
    sql.DictBase parent class of Service. Instead the service name have to
    be directly read from the 'extra' attribute of the service object using
    service.extra.get('name', '').
    The name attribute of the service will now always be included in the
    service catalog. The value will be defaulted to empty string if no
    service name is available.

    Change-Id: I305abd8446fee57af18aebf7873e3c67a1b83c92
    Closes-Bug: #1393518