keystone uses incorrect OS_AUTH_URL

[Ted Elwartowski]

The keystone command does not honor the env[OS_AUTH_URL] or --os-auth-url CLI parameter. The authorization URL defaults to http://controller:35357/v2.0 regardless if a different auth URL is specified. This error was found on CentOS 6.5 (2.6.32-431.17.1.el6.x86_64) in reference to keystone verification: http://docs.openstack.org/icehouse/install-guide/install/yum/content/keystone-verify.html.

Steps to reproduce:

# export OS_USERNAME=admin
# export OS_PASSWORD=ADMIN_PASS
# export OS_TENANT_NAME=admin
# export OS_AUTH_URL=http://HOST_NAME:35357/v2.0

# keystone user-list
or
# keystone --os-auth-url=http://HOST_NAME:35357/v2.0 user-list

This produces the following error:
Unable to establish connection to http://controller:35357/v2.0/users

This command continued to fail until an entry for "controller" was added to the /etc/hosts file. While the documentation does specify to include entries in the hosts file for the controller, it is not reliable to assume static hosts are available in a production ENV.

[Ted Elwartowski (xelwarto-pub)]

Forgot to add the keystone version information:

keystone --version
0.8.0

[Ted Elwartowski (xelwarto-pub)]

This issue may be related to the service endpoint creation .. I originally created the service endpoint using the following command:

# keystone endpoint-create \
  --service-id=$(keystone service-list | awk '/ identity / {print $2}') \
  --publicurl=http://controller:5000/v2.0 \
  --internalurl=http://controller:5000/v2.0 \
  --adminurl=http://controller:35357/v2.0

After which I deleted the endpoint and recreated it with the following:

keystone endpoint-create \
  --service-id=$(keystone service-list | awk '/ identity / {print $2}') \
  --publicurl=http://HOST_NAME:5000/v2.0 \
  --internalurl=http://HOST_NAME:5000/v2.0 \
  --adminurl=http://HOST_NAME:35357/v2.0

Where HOST_NAME = controller FQDN ... creating the endpoint this way allows me to correctly specify the OS_AUTH_URL for the keystone command. The keystone verification works properly now.

[Adam Young (ayoung)]

Are you sanitizing your inputs, and putting in locally resolvable values?

export OS_AUTH_URL=http://HOST_NAME:35357/v2.0

Unless HOST_NAME is a FQDN, this would never work. There is no parameter substitution on these env vars.

Same is true with

http://controller:35357/v2.0/users

The CLI is doing HTTP calls, so the hostname must be resolvable on the machine you are running on. The fact that it runs when you add it to hosts seems to reinforce this.

[Ted Elwartowski (xelwarto-pub)]

This is the setup:

OS_INST_HOSTNAME=`hostname -f`
export OS_INST_HOSTNAME

export OS_AUTH_URL=http://$OS_INST_HOSTNAME:35357/v2.0

###

All values for the hostnames in the auth URL are the FQDN of locally resolvable hosts. I tried to reproduce the issue I encountered and it is directly related to how the service endpoint is created as I stated previously. I am guessing that this behavior is actually correct as it looks like the error is coming from a connection attempt to the service endpoint admin URL and not the auth URL specified for the keystone CLI.

The issue here is that I think the error message that is produced can be a little misleading ... which made me believe that there was an issue with the auth URL and not the service endpoint configuration.

After correcting the configuration of the service endpoint everything is working correctly. As a note ... I am not using statically set host names for the configuration of the endpoint URLs or the auth URL and I am able to preform API calls successfully.

[Dolph Mathews (dolph)]

How did the provided error message, which references a hostname present in your service catalog, lead you to believe that the "auth URL" wasn't being used? If there's a way to improve that feedback, I'd love to, but it's not clear to me how to go about it.

[Ted Elwartowski (xelwarto-pub)]

As per the documentation the step before running the verification of the keystone installation is to set the OS_AUTH_URL environment variable to the following:

# export OS_AUTH_URL=http://controller:35357/v2.0

The error message that is produced is the following:

# Unable to establish connection to http://controller:35357/v2.0/users

For me, based upon the steps, I had thought the error message was referring the authorization URL which I had just specified in the environment variable. Turns out the issue was actually with how the identity service endpoint was setup (which was incorrect). The error message provides no indication of what configuration it was using to get the connection error.

I will admit that part of this is due to my understanding of the OpenStack infrastructure as I am still trying to learn. And it was not until I probed and tested more that I found where my actual mistake was ... which was a good learning experience. Though I believe that for the purposes of troubleshooting, the error messages could be a little more descriptive to what setting this connection error is referring to. If this were a production environment, valuable time may be lost trying to trouble shoot an issue because the error message was not clear on where the problem was. To me, an error that just says "Unable to establish connection ... " is to vague.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/100006

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dolph Mathews (<email address hidden>) on branch: master
Review: https://review.openstack.org/100006
Reason: as caught by haneef, this is the wrong solution. that said, i'm not sure there's much room for improvement here as this class has no knowledge of what it's accessing beyond the URL itself (there's no context available).

[Dolph Mathews (dolph)]

As noted in code review, there's not any context available to improve this error message - just the request itself (method, URL, etc). Leaving this as incomplete in case anyone has a solution.

[Launchpad Janitor (janitor)]

[Expired for python-keystoneclient because there has been no activity for 60 days.]