| 2015-09-22 20:34:48 |
Devdatta Kulkarni |
description |
The webhook-based trigger workflow is not yet supported in Solum in true end-to-end fashion. We are able to create DUs upon receiving a trigger, but heat deployment fails with 403 (Forbidden) from Heat.
Here is the stack trace from Solum's deployer.
2015-09-17 19:33:34.493 10891 ERROR solum.deployer.handlers.heat [-] ERROR: Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these
details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)
[u'
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat Traceback (most recent call last):
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/opt/stack/solum/solum/deployer/handlers/heat.py", line 349, in deploy
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat files=get_file_dict)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/v1/stacks.py", line 134, in create
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat data=kwargs, headers=headers)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/common/http.py", line 265, in json_request
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat resp = self._http_request(url, method, **kwargs)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/common/http.py", line 220, in _http_request
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat raise exc.from_response(resp)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat HTTPInternalServerError: ERROR: Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat [u'
The corresponding stack trace from the heat-api:
15-09-17 19:33:34.481 TRACE heat.common.wsgi
2015-09-17 19:33:34.488 DEBUG heat.common.serializers [req-da14c3fd-64d9-4695-9006-6d53480141ec None demo] JSON response : {"explanation": "The server has either erred or is incapable of performing the requested operation.", "code": 500, "error": {"message": "Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)\n[u'", "traceback": "Traceback (most recent call last):\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 142, in _dispatch_and_reply\\n executor_callback))\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 186, in _dispatch\\n executor_callback)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 129, in _do_dispatch\\n result = func(ctxt, **new_args)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py\", line 105, in wrapper\\n return f(*args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/common/context.py\", line 283, in wrapped\\n return func(self, ctx, *args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/engine/service.py\", line 703, in create_stack\\n stack.store()\\n', u' File \"/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py\", line 105, in wrapper\\n return f(*args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/engine/stack.py\", line 484, in store\\n trust_ctx = keystone.create_trust_context()\\n', u' File \"/opt/stack/heat/heat/common/heat_keystoneclient.py\", line 240, in create_trust_context\\n role_names=roles)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/v3/contrib/trusts.py\", line 76, in create\\n **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 73, in func\\n return f(*args, **new_kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 333, in create\\n self.key)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 151, in _create\\n return self._post(url, body, response_key, return_raw, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 165, in _post\\n resp, body = self.client.post(url, body=body, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 176, in post\\n return self.request(url, \\'POST\\', **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 206, in request\\n resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 95, in request\\n return self.session.request(url, method, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py\", line 336, in inner\\n return func(*args, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py\", line 397, in request\\n raise exceptions.from_response(resp, method, url)\\n', u'Forbidden: Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)\\n'].", "type": "RemoteError"}, "title": "Internal Server Error"} from (pid=20128) to_json /opt/stack/heat/heat/common/serializers.py:42
2015-09-17 19:33:34.491 DEBUG eventlet.wsgi.server [req-da14c3fd-64d9-4695-9006-6d53480141ec None demo] 10.0.2.15 - - [17/Sep/2015 19:33:34] "POST /v1/fd7f57c1ad5743939111144982095cc0/stacks HTTP/1.1" 500 3588 1.004695 from (pid=20128) write /opt/stack/heat/heat/common/wsgi.py:267 |
The webhook-based trigger workflow is not yet supported in Solum in true end-to-end fashion. We are able to create DUs upon receiving a trigger, but heat deployment fails with 403 (Forbidden) from Heat.
Here is the stack trace from Solum's deployer.
2015-09-17 19:33:34.493 10891 ERROR solum.deployer.handlers.heat [-] ERROR: Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these
details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)
[u'
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat Traceback (most recent call last):
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/opt/stack/solum/solum/deployer/handlers/heat.py", line 349, in deploy
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat files=get_file_dict)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/v1/stacks.py", line 134, in create
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat data=kwargs, headers=headers)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/common/http.py", line 265, in json_request
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat resp = self._http_request(url, method, **kwargs)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat File "/usr/local/lib/python2.7/dist-packages/heatclient/common/http.py", line 220, in _http_request
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat raise exc.from_response(resp)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat HTTPInternalServerError: ERROR: Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)
2015-09-17 19:33:34.493 10891 TRACE solum.deployer.handlers.heat [u'
The corresponding stack trace from the heat-api:
15-09-17 19:33:34.481 TRACE heat.common.wsgi
2015-09-17 19:33:34.488 DEBUG heat.common.serializers [req-da14c3fd-64d9-4695-9006-6d53480141ec None demo] JSON response : {"explanation": "The server has either erred or is incapable of performing the requested operation.", "code": 500, "error": {"message": "Remote error: Forbidden Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)\n[u'", "traceback": "Traceback (most recent call last):\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 142, in _dispatch_and_reply\\n executor_callback))\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 186, in _dispatch\\n executor_callback)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 129, in _do_dispatch\\n result = func(ctxt, **new_args)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py\", line 105, in wrapper\\n return f(*args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/common/context.py\", line 283, in wrapped\\n return func(self, ctx, *args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/engine/service.py\", line 703, in create_stack\\n stack.store()\\n', u' File \"/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py\", line 105, in wrapper\\n return f(*args, **kwargs)\\n', u' File \"/opt/stack/heat/heat/engine/stack.py\", line 484, in store\\n trust_ctx = keystone.create_trust_context()\\n', u' File \"/opt/stack/heat/heat/common/heat_keystoneclient.py\", line 240, in create_trust_context\\n role_names=roles)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/v3/contrib/trusts.py\", line 76, in create\\n **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 73, in func\\n return f(*args, **new_kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 333, in create\\n self.key)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 151, in _create\\n return self._post(url, body, response_key, return_raw, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py\", line 165, in _post\\n resp, body = self.client.post(url, body=body, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 176, in post\\n return self.request(url, \\'POST\\', **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 206, in request\\n resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py\", line 95, in request\\n return self.session.request(url, method, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py\", line 336, in inner\\n return func(*args, **kwargs)\\n', u' File \"/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py\", line 397, in request\\n raise exceptions.from_response(resp, method, url)\\n', u'Forbidden: Remaining redelegation depth of 0 out of allowed range of [0..3] (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-5ee8efb1-6f61-48ba-802f-8ab5c7505a3a)\\n'].", "type": "RemoteError"}, "title": "Internal Server Error"} from (pid=20128) to_json /opt/stack/heat/heat/common/serializers.py:42
2015-09-17 19:33:34.491 DEBUG eventlet.wsgi.server [req-da14c3fd-64d9-4695-9006-6d53480141ec None demo] 10.0.2.15 - - [17/Sep/2015 19:33:34] "POST /v1/fd7f57c1ad5743939111144982095cc0/stacks HTTP/1.1" 500 3588 1.004695 from (pid=20128) write /opt/stack/heat/heat/common/wsgi.py:267
This bug is to enable this support.
Basically we need to modify heat deployer to use keystone trust to grant solum service user permission to create heat calls on the user's behalf. |
|
