Cinder and glance client have endpoint selection issues

Bug #1472295 reported by Bjoern Teipel on 2015-07-07
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Glance Client
Undecided
Unassigned
openstack-ansible
Undecided
Unassigned
python-cinderclient
Undecided
Unassigned
python-keystoneclient
Undecided
Unassigned

Bug Description

The cinder and glance client past 10.1.2 have endpoint selection issues and issues using the right protocol when using HTTPS and selecting publicURL endpoints.

The issue can be reproduced if we set OS_ENDPOINT_TYPE to publicURL and OS_AUTH_URL to the public keystone URL while using the HTTPS protocol.
The glance and cinder client seem to use the the href from the links array

DEBUG:keystoneclient.session:REQ: curl -g -i -X GET https://<ip>:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
DEBUG:keystoneclient.session:RESP: [200] content-length: 424 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Tue, 07 Jul 2015 14:30:23 GMT content-type: application/json
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}, {"base": "application/xml", "type": "application/vnd.openstack.identity-v2.0+xml"}], "id": "v2.0", "links": [{"href": "http://<ip>:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG:keystoneclient.auth.identity.v2:Making authentication request to http://<ip>:5000/v2.0/tokens
ERROR: Unable to establish connection to http://<ip>:5000/v2.0/tokens

I have not found an upstream issue yet but I suspect that either X-FORWARDED-PROTO is needed to help the keystone middleware selecting the protocol or we need to overwrite public_endpoint inside keystone.conf

Not working :
python-cinderclient==1.1.1
python-glanceclient==0.15.0

Working :
python-glanceclient==0.13.1
python-cinderclient==1.0.9

Changed in openstack-ansible:
status: New → Opinion
status: Opinion → Incomplete
Peter Sabaini (peter-sabaini) wrote :

This happens here independently of selecting https or endpoint type. However I call cinder, it ends up requesting tokens from what I suspect is the catalog entry of keystone instead of using os-auth-url

./bin/cinder --debug --os-auth-url=http://example-front.com/v2.0 list
DEBUG:keystoneclient.session:REQ: curl -g -i -X GET http://example-front.com/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
DEBUG:keystoneclient.session:RESP: [200] content-length: 615 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 02 Oct 2015 12:55:19 GMT content-type: application/json x-distribution: Ubuntu
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}, {"base": "application/xml", "type": "application/vnd.openstack.identity-v2.0+xml"}], "id": "v2.0", "links": [{"href": "http://internal.ip.address:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/api/openstack-identity-service/2.0/content/", "type": "text/html", "rel": "describedby"}, {"href": "http://docs.openstack.org/api/openstack-identity-service/2.0/identity-dev-guide-2.0.pdf", "type": "application/pdf", "rel": "describedby"}]}}

DEBUG:keystoneclient.auth.identity.v2:Making authentication request to http://internal.ip.address:5000/v2.0/tokens
^C... terminating cinder client

Cinder is installed via pip:

./bin/cinder --version
1.4.0

tags: added: canonical-bootstack
Bjoern Teipel (bjoern-teipel) wrote :

FYI, my current workaround is to configure public_endpoint inside keystone.conf to use the desired URL. That will update the href attribute inside the keystone catalog or token response.

Major Hayden (rackerhacker) wrote :

Is this still an issue in Kilo or Liberty?

Changed in openstack-ansible:
status: Incomplete → Opinion
status: Opinion → Incomplete
Bjoern Teipel (bjoern-teipel) wrote :

Yes this is still an issue

Changed in openstack-ansible:
status: Incomplete → New
Kairat Kushaev (kkushaev) wrote :

So glance ignores endpoint type flag IIUC. Right?
Then it may be fixed here for glanceclient: https://review.openstack.org/#/c/301214/

Bjoern Teipel (bjoern-teipel) wrote :

Yes correct. I also recall that glance was using GLANCE_ENDPOINT_TYPE, not sure if that still exist. This issue is perhaps present on more clients, possibly all of them

summary: - Juno: cinder and glance client have endpoint selection issues
+ Cinder and glance client have endpoint selection issues

This is not a bug in OpenStack-Ansible, but in the clients. I'm therefore marking this as invalid for OSA.

Changed in openstack-ansible:
status: New → Invalid
Matthew Edmonds (edmondsw) wrote :

I think this is actually working as designed. I would view comment 2 as the fix for a misconfiguration (i.e. user error) issue, not a workaround. Per the description, the clients are calling keystone's GET https://<ip>:5000/v2.0 and yet the response they get back from keystone wrongly shows http instead of https. If they get bad information from keystone, it's harsh to blame the clients...

Matthew Edmonds (edmondsw) wrote :

if there is anything to fix here, it is probably in keystoneclient

Morgan Fainberg (mdrnstm) wrote :

this bug has had no traction, is this still the case with the KSA sessions? If so, please re0open against keystoneauth.

Changed in python-keystoneclient:
status: New → Incomplete
Morgan Fainberg (mdrnstm) wrote :

This would be something to fix in keystoneauth if it is still an issue. marking as invalid/wont fix.

Changed in python-keystoneclient:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers