Need to be able to opt-out of X-Service-Catalog header

[Guang Yee]

Service catalog could get very large and easily exceed the 8k header limit. Furthermore, most services do not use the X-Service-Catalog header.

We need configurable option, "nocatalog", to opt-out of service catalog.

[Dolph Mathews (dolph)]

Is the header limit actually enforced in the wsgi pipeline or is this just a theoretical?

[Guang Yee (guang-yee)]

https://github.com/openstack/swift/blob/master/swift/common/constraints.py#47

Swift will fail if we have a service catalog header that is more than 8K.

[Guang Yee (guang-yee)]

Here's the actual enforcement.

https://github.com/openstack/swift/blob/master/swift/common/constraints.py#L83

Caller will receive a 400 if the service catalog is too big.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/51300

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/51300
Committed: http://github.com/openstack/python-keystoneclient/commit/a97b293501fa504dd154fc921809a40bc2a34049
Submitter: Jenkins
Branch: master

commit a97b293501fa504dd154fc921809a40bc2a34049
Author: guang-yee <email address hidden>
Date: Fri Oct 11 14:08:57 2013 -0700

    Opt-out of service catalog

    Introducing a config option 'include_service_catalog' to indicate whether
    service catalog is needed. If the 'include_service_catalog' option is set to
    False, middleware will not ask for service catalog on token validation and will
    not set the X-Service-Catalog header.

    This option is backward compatible as it is default to True.

    DocImpact
    Fixed bug 1228317

    Change-Id: Id8c410a7ae0443ac425d20cb9c6a24ee5bb2cb8d

[Atsuko Ito (yottatsa)]

This patch brokes nova cinder extension.

Actually, I'm on developing same approach on this HUGE TOKEN problem and there are some problems with X-Service-Catalog removal https://review.openstack.org/96725

curl -i http://compute.i.haze-pre.yandex-team.ru/v2/659f2aeaea5a43e18abea1a598557f24/os-volumes/list -X GET -H "X-Auth-Project-Id: devel" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: ..."

HTTP/1.1 500 Internal Server Error
Server: nginx/1.4.7
Date: Wed, 04 Jun 2014 14:16:02 GMT
Content-Type: application/json; charset=UTF-8
Content-Length: 128
Connection: keep-alive
X-Compute-Request-Id: req-b9dbdde1-737a-47b6-84bb-7c7fdfbe49d2

{"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}(Python)

[Dolph Mathews (dolph)]

Vladimir: the long term goal with removing the service catalog from the token also included a new endpoint on keystone to fetch a catalog, e.g. GET /v3/catalog separately from the token. When presented with a catalog-free token, keystoneclient.middleware.auth_token could call GET /v3/catalog and populate the X-Service-Catalog header as usual.

[Atsuko Ito (yottatsa)]

dolph, that will be really nice. Do you have any blueprint on it? I ready to implement it ASAP

[Atsuko Ito (yottatsa)]

PKI validation requires middleware to connect to the keystone for revocation list and CA certificate.
So we can remove catalog from encrypted token, and provide catalog on POST /v2.0/token only.
Then middleware fetch catalog when it fetches revocation list and CA certificate and populate the X-Service-Catalog header as usual.

[Guang Yee (guang-yee)]

Your propose change will definitely break endpoint-filtering extension for sure. There are multiple ways to reduce the token size already.

1. If you do not need service catalog, just asking Keystone not to return it on token request or validation by specifying the nocatalog query string. For example,

POST /v3/auth/tokens?nocatalog
GET /v3/auth/tokens?nocatalog

2. If you need the entire service catalog but don't want it to be encoded into the PKI token, use UUID token provider.

3. If you only need partial service catalog (i.e. subset of endpoints), use endpoint filtering extension.

4. If you need the entire service catalog but wants to reduce the size of PKI token in general, use either PKIZ or UUID token provider.

[Dolph Mathews (dolph)]

This discussion would be better served by a keystone-spec review :)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/142991

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/142991
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=ed2858add157b9536f157ca08f443a11dd5b1559
Submitter: Jenkins
Branch: master

commit ed2858add157b9536f157ca08f443a11dd5b1559
Author: Jamie Lennox <email address hidden>
Date: Fri Dec 19 16:06:38 2014 +1000

    Allow v3 plugins to opt out of service catalog

    The identity server supports adding ?nocatalog to auth requests and
    there are situations where we need to be able to exploit that from the
    client. Allow passing include_catalog=False to v3 plugins to fetch a
    plugin without a catalog.

    Change-Id: I4b2afbfffb71490faed4b7ef0de4d00ee208733a
    Closes-Bug: #1228317