HA template creation fails with AWS auth

[Angus Salkeld]

Trying to create a stack with the wordpress+HA template fails if you use heat-boto, creating the non-HA templates works fine, looks like part of the resource validation in the engine is expecting keystone credentials:

heat-boto -d create wordpress_ha2 --template-file=templates/WordPress_Single_Instance_With_HA.template --parameters="InstanceType=m1.large;KeyName=root_key"
snip from /var/log/heat/engine.log

2012-08-17 15:13:45 INFO [heat.engine.resources] Validating WaitConditionHandle "WaitHandle"
2012-08-17 15:13:45 INFO [heat.engine.resources] Validating User "CfnUser"
2012-08-17 15:13:45 INFO [heat.engine.resources] Validating AccessKey "WebServerKeys"
2012-08-17 15:13:45 INFO [heat.engine.resources] Validating Instance "WikiDatabase"
2012-08-17 15:13:45 ERROR [keystoneclient.v2_0.client] Authorization Failed.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 102, in authenticate
    return_raw=True)
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/tokens.py", line 32, in authenticate
    raise ValueError('A username and password or token is required.')
ValueError: A username and password or token is required.
2012-08-17 15:13:45 ERROR [heat.engine.parser] validate
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 371, in validate
    result = res.validate()
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/instance.py", line 257, in validate
    res = super(Instance, self).validate()
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/resources.py", line 295, in validate
    self.calculate_properties()
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/resources.py", line 231, in calculate_properties
    for p, v in self.parsed_template('Properties').items():
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/resources.py", line 188, in parsed_template
    return self.stack.resolve_runtime_data(template)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 620, in resolve_runtime_data
    return resolve_runtime_data(self.t, self.resources, snippet)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 648, in resolve_runtime_data
    template.resolve_base64])
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 657, in transform
    data = t(data)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 198, in resolve_attributes
    return _resolve(lambda k, v: k == 'Fn::GetAtt', handle_getatt, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in _resolve
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in <genexpr>
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 670, in <lambda>
    recurse = lambda s: _resolve(match, handle, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in _resolve
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in <genexpr>
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 670, in <lambda>
    recurse = lambda s: _resolve(match, handle, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in _resolve
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 677, in <genexpr>
    return dict((k, recurse(v)) for k, v in snippet.items())
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 670, in <lambda>
    recurse = lambda s: _resolve(match, handle, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 679, in _resolve
    return [recurse(v) for v in snippet]
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 670, in <lambda>
    recurse = lambda s: _resolve(match, handle, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 679, in _resolve
    return [recurse(v) for v in snippet]
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 670, in <lambda>
    recurse = lambda s: _resolve(match, handle, s)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 676, in _resolve
    return handle(recurse(v))
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/parser.py", line 193, in handle_getatt
    return resources[resource].FnGetAtt(att)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/user.py", line 179, in FnGetAtt
    res = self._secret_accesskey()
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/user.py", line 158, in _secret_accesskey
    user = self._user_from_name(self.properties['UserName'])
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/user.py", line 128, in _user_from_name
    users = self.keystone().users.list(tenant_id=tenant_id)
  File "/usr/lib/python2.7/site-packages/heat-6-py2.7.egg/heat/engine/resources.py", line 218, in keystone
    auth_url=con.auth_url)
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 80, in __init__
    self.authenticate()
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 110, in authenticate
    "%s" % e)
AuthorizationFailure: Authorization Failed: A username and password or token is required.

[Zane Bitter (zaneb)]

Updated stack trace, since the code has changed quite a bit since this report:

2013-01-22 15:53:10 DEBUG [keystoneclient.client] REQ: curl -i http://127.0.0
.1:5000/v2.0/tokens -X POST -H "User-Agent: python-keystoneclient" -H "Content-T
ype: application/json" -H "X-Auth-Token: 1a9b54f53eff4be0a3f7f89baec6b255"

2013-01-22 15:53:10 DEBUG [keystoneclient.client] REQ BODY: {"auth": {"token"
: {"id": "1a9b54f53eff4be0a3f7f89baec6b255"}, "tenantName": "service"}}

2013-01-22 15:53:10 DEBUG [keystoneclient.client] RESP: {'date': 'Tue, 22 Jan
 2013 14:53:10 GMT', 'content-type': 'application/json', 'content-length': '143'
, 'status': '500', 'vary': 'X-Auth-Token'}
RESP BODY: {"error": {"message": "An unexpected error prevented the server from
fulfilling your request.", "code": 500, "title": "Internal Server Error"}}

2013-01-22 15:53:10 ERROR [keystoneclient.v2_0.client] Authorization Failed.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 10
2, in authenticate
    return_raw=True)
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/tokens.py", line 37
, in authenticate
    return self._create('/tokens', params, "access", return_raw=return_raw)
  File "/usr/lib/python2.7/site-packages/keystoneclient/base.py", line 82, in _c
reate
    resp, body = self.api.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneclient/client.py", line 168, in
 post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneclient/client.py", line 149, in
 _cs_request
    **kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneclient/client.py", line 129, in
 request
    raise exceptions.from_response(resp, body)
ClientException: An unexpected error prevented the server from fulfilling your request. (HTTP 500)
2013-01-22 15:53:10 ERROR [heat.engine.resource] create WaitConditionHandle "WaitHandle"
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/heat-grizzly.2.36.g55c9032dev-py2.7.egg/heat/engine/resource.py", line 225, in create
    self.handle_create()
  File "/usr/lib/python2.7/site-packages/heat-grizzly.2.36.g55c9032dev-py2.7.egg/heat/engine/resources/wait_condition.py", line 88, in handle_create
    user_id = self.keystone().create_stack_user(
  File "/usr/lib/python2.7/site-packages/heat-grizzly.2.36.g55c9032dev-py2.7.egg/heat/engine/resource.py", line 198, in keystone
    return self.stack.clients.keystone()
  File "/usr/lib/python2.7/site-packages/heat-grizzly.2.36.g55c9032dev-py2.7.egg/heat/engine/clients.py", line 61, in keystone
    self._keystone = hkc.KeystoneClient(self.context)
  File "/usr/lib/python2.7/site-packages/heat-grizzly.2.36.g55c9032dev-py2.7.egg/heat/common/heat_keystoneclient.py", line 53, in __init__
    self.client = kc.Client(**kwargs)
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 80, in __init__
    self.authenticate()
  File "/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py", line 110, in authenticate
    "%s" % e)
AuthorizationFailure: Authorization Failed: An unexpected error prevented the server from fulfilling yo...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/20253

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/20254

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/20253
Committed: http://github.com/openstack/heat/commit/38b7e7d501e049349ef70d2167f92e23997627b0
Submitter: Jenkins
Branch: master

commit 38b7e7d501e049349ef70d2167f92e23997627b0
Author: Steven Hardy <email address hidden>
Date: Tue Jan 22 16:30:18 2013 +0000

    heat api paste.ini auth_uri should use auth_port

    For token based auth to work, the auth_uri needs to use
    auth_port, which points at the internalURL of the keystone
    service, the current config uses publicURL which causes
    token auth requests to keystone to fail

    ref bug 1072944

    Change-Id: If7e0c2246205377f57f879ccf8bf36ff8b0d92e1
    Signed-off-by: Steven Hardy <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/20254
Committed: http://github.com/openstack/heat/commit/f8ffddca0a7014a33f3b1aec1c33fe43ff39ef12
Submitter: Jenkins
Branch: master

commit f8ffddca0a7014a33f3b1aec1c33fe43ff39ef12
Author: Steven Hardy <email address hidden>
Date: Tue Jan 22 16:33:14 2013 +0000

    heat_keystoneclient make token auth work

    username/password are ignored so don't pass them, and
    tenant needs to be the context tenant not the service
    tenant or token auth will fail.

    Fixes bug 1072944

    Change-Id: I862d0041daad278dfe7bc16d59dc733dde90e758
    Signed-off-by: Steven Hardy <email address hidden>