kolla

heat stack-create fails and returns warning about trustee

Bug #1492648 reported by Steven Dake on 2015-09-05

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	kolla	Fix Released	Critical	Steven Dake	kolla liberty-rc1 "liberty-rc1"

Bug Description

The template: http://paste.openstack.org/raw/446927/
The launch script: http://paste.openstack.org/raw/446930/

Returns the following backtrace on centos + binary:
2015-09-05 18:46:30.614 20 INFO heat.engine.service [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Creating stack steak
2015-09-05 18:46:30.629 20 INFO heat.engine.resource [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Validating Server "steak_node"
2015-09-05 18:46:30.832 20 WARNING heat.common.context [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Using the keystone_authtoken user as the heat trustee user directly is deprecated. Please add the trustee credentials you need to the trustee section of your heat.conf file.
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Exception during message handling: The resource could not be found. (HTTP 404) (HTTP 404)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 305, in wrapped
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return func(self, ctx, *args, **kwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 708, in create_stack
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher stack.store()
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 484, in store
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher trust_ctx = keystone.create_trust_context()
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/heat/common/heat_keystoneclient.py", line 210, in create_trust_context
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher self.session)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 352, in get_user_id
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return self.get_access(session).user_id
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 240, in get_access
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher self.auth_ref = self.get_auth_ref(session)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py", line 190, in get_auth_ref
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher authenticated=False, log=False, **rkwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 501, in post
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return self.request(url, 'POST', **kwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher return func(*args, **kwargs)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 401, in request
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher raise exceptions.from_response(resp, method, url)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher NotFound: The resource could not be found. (HTTP 404)
2015-09-05 18:46:30.838 20 ERROR oslo_messaging.rpc.dispatcher
2015-09-05 18:46:30.840 20 ERROR oslo_messaging._drivers.common [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Returning exception The resource could not be found. (HTTP 404) to caller
2015-09-05 18:46:30.840 20 ERROR oslo_messaging._drivers.common [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n executor_callback)\n', ' File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 305, in wrapped\n return func(self, ctx, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 708, in create_stack\n stack.store()\n', ' File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 484, in store\n trust_ctx = keystone.create_trust_context()\n', ' File "/usr/lib/python2.7/site-packages/heat/common/heat_keystoneclient.py", line 210, in create_trust_context\n self.session)\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 352, in get_user_id\n return self.get_access(session).user_id\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 240, in get_access\n self.auth_ref = self.get_auth_ref(session)\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py", line 190, in get_auth_ref\n authenticated=False, log=False, **rkwargs)\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 501, in post\n return self.request(url, \'POST\', **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner\n return func(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 401, in request\n raise exceptions.from_response(resp, method, url)\n', 'NotFound: The resource could not be found. (HTTP 404)\n']

The key warning is here:
2015-09-05 18:46:30.832 20 WARNING heat.common.context [req-55425669-1736-4d9e-bae0-cb7693e308f3 c4be44710f9e49cd94e92b48192e02eb ee3fe8d1d4234399b1fae08cf103a5de] Using the keystone_authtoken user as the heat trustee user directly is deprecated. Please add the trustee credentials you need to the trustee section of your heat.conf file.

Steven Dake (sdake) on 2015-09-05

Changed in kolla:
status:	New → Triaged
importance:	Undecided → Critical
milestone:	none → liberty-rc1
assignee:	nobody → Steven Dake (sdake)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-06: Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/220765

Changed in kolla:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-09-07: Fix merged to kolla (master)

Reviewed: https://review.openstack.org/220765
Committed: https://git.openstack.org/cgit/stackforge/kolla/commit/?id=e569d80182256f98febe499f7259bcd8dc54d07d
Submitter: Jenkins
Branch: master

commit e569d80182256f98febe499f7259bcd8dc54d07d
Author: Steven Dake <email address hidden>
Date: Sat Sep 5 18:21:41 2015 -0700

Make heat stack-create work

    The stack-create operation fails. The current implementation of
    Heat Ansible role does not implement trusts. Tell heat it should use
    the non-trusts authentication model.

    Also add a [keystone_clients] section which is required in newer
    versions of heat to properly resolve the keystone v3 API endpoint
    required for trust communication.

This fix spawns a Tech-Debt Bug to implement trusts in heat: #1492736

Change-Id: I06dd6436cc34a37b4966fddc888fade982ea51bd
Closes-Bug: #1492648

Changed in kolla:
status:	In Progress → Fix Committed

Sam Yaple (s8m) on 2015-11-20

Changed in kolla:
status:	Fix Committed → Fix Released

Revision history for this message

Ishita Dasgupta (ishitadg) wrote on 2017-06-19:

The committed git link appears to be stale. I am trying to run an MPI instance using openstack (stack-create) but it still fails with following error lines:
Expecting to find domain in user - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: )

(When looking in heat-engine.log)
Using the keystone_authtoken user as the heat trustee user directly is deprecated. Please add the trustee credentials you need to the trustee section of your heat.conf file.

Is this an issue with the openstack and heat? Is there something I might be doing wrong?

Revision history for this message

Eduardo Gonzalez (egonzalez90) wrote on 2017-06-19:

@Ishita, please open a new bug report for your issue with information about your environment, versions, etc.
Regards

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.