keystone listens locally on admin port

[David Kranz]

I installed a vanilla devstack except for setting SERVICE_HOST in localrc so I could run tempest from another machine. Tempest fails trying to connect to adminURL and it seems to be because port 35357 is only open locally. The conf file comment says:

# The base admin endpoint URL for keystone that are advertised
# to clients (NOTE: this does NOT affect how keystone listens
# for connections) (string value)
#admin_endpoint=http://localhost:%(admin_port)s/

But this from netstat. I would expect 35357 to be the same as the others. It is also possible this is a devstack issue but
I'm not sure so starting here.

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:iscsi-target *:* LISTEN
tcp 0 0 *:40956 *:* LISTEN
tcp 0 0 localhost:35357 *:* LISTEN
tcp 0 0 *:6080 *:* LISTEN
tcp 0 0 *:6081 *:* LISTEN
tcp 0 0 *:3333 *:* LISTEN
tcp 0 0 *:8773 *:* LISTEN
tcp 0 0 *:8774 *:* LISTEN
tcp 0 0 *:8775 *:* LISTEN
tcp 0 0 *:9191 *:* LISTEN
tcp 0 0 *:8776 *:* LISTEN
tcp 0 0 *:5000 *:* LISTEN
... elided ...

And catalog:+-------------+-----------------------------------+
| Property | Value |
+-------------+-----------------------------------+
| adminURL | http://dkranz-devstack:35357/v2.0 |
| id | 39932d3dcf4340a98727294ed5ec71b8 |
| internalURL | http://dkranz-devstack:5000/v2.0 |
| publicURL | http://dkranz-devstack:5000/v2.0 |
| region | RegionOne |
+-------------+-----------------------------------+

[Wei Wang (damon-devops)]

Hi David,

I think the conf's endpoint url doesn't, can you use curl connect to keystone service from another computer? If not, pleases check iptables and make sure your keystone service is truely working.

Hope it helps

Damon

[David Kranz (david-kranz)]

This issue is caused by keystone listening globally for the public url (port 5000) but only on localhost for 35357. I poked a little more and found the cause.

Setting SERVICE_HOST in localrc causes devstack to produce these values in keystone.conf:

admin_bind_host = dkranz-devstack
admin_endpoint = http://dkranz-devstack:%(admin_port)s/
public_endpoint = http://dkranz-devstack:%(public_port)s/

I thought the purpose of this env variable was to make the catalog expose endpoints that are accessible from outside the devstack machine so it is surprising this also sets the bind host which makes it not accessible off the local machine. Is this behaviour intentional?

[Dolph Mathews (dolph)]

The history of that decision is documented in bug 1253482

[David Kranz (david-kranz)]

I see. So how can I run tempest (or anything else using the apis) on a different machine than devstack runs on? I tried

KEYSTONE_AUTH_PORT=5000

but got a bunch of errors. Surely this is a reasonable use case, no?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/75954

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/75954
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=041fa712472d887550a540dd50ade546f847c6b4
Submitter: Jenkins
Branch: master

commit 041fa712472d887550a540dd50ade546f847c6b4
Author: David Kranz <email address hidden>
Date: Mon Feb 24 13:30:59 2014 -0500

    Make admin_bind_host configurable

    The use case is running devstack inside an OpenStack vm and running tempest
    from some other machine. To make the catalog export urls that can be accessed
    from off the devstack machine, you need to set KEYSTONE_SERVICE_HOST to an
    external IP. But devstack uses that address in its setup of keystone in
    addition to exporting in the catalog. Because OpenStack has an issue where
    a vm cannot access itself through its own floating ip, devstack fails. There
    is no way to have this use case by providing an ip address. The workaround
    is to use the hostname of the devstack machine. That worked until recently
    when a change was made to set admin_bind_host to the value of
    KEYSTONE_SERVICE_HOST. The result is that port 35357 is only opened locally.
    This change allows the devstack user to restore the original behavior
    allowing this use case.

    Change-Id: I97b938b305b7dd878397e7e64462650064e59cd2
    Closes-Bug: #1283803