SMR3.1build14:creation of SVC with heat fails with authorisation failure The resource could not be found. (HTTP 404)
Provisioned a node with SM R3.1 build 14 kilo
try creating a svc with heat, it fails with authorisation failure The resource could not be found. (HTTP 404)
root@nodeg37:~# heat stack-list
+----+------------+--------------+---------------+
| id | stack_name | stack_status | creation_time |
+----+------------+--------------+---------------+
+----+------------+--------------+---------------+
root@nodeg37:~# keystone role-list
+----------------------------------+------------------+
| id | name |
+----------------------------------+------------------+
| a3f321a59c7a424cb3854628fbc50bfe | ResellerAdmin |
| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ |
| 2e1b40c838e54ce2b8e33baf184b2f8e | admin |
| 19a291db066b49dbad89eeaab3280858 | heat_stack_owner |
| 76fcc5741c3c474da49c6fd774196cc7 | heat_stack_user |
+----------------------------------+------------------+
Heat Logs while trying to create SVC
Heat-engine.log
34069 2016-08-03 22:00:48.109 5769 DEBUG keystoneclient.auth.identity.v3 [req-5e24424a-2d08-4923-aaee-bb2f1be3f941 0eac51d15cc544159fc4387cd8e47352 e128a056db4b44708172b186ebcba2b5] Making authentication request to http://10.204.217.77:5000/auth/tokens get_auth_ref /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3.py:125
34070 2016-08-03 22:00:48.116 5769 DEBUG keystoneclient.session [req-5e24424a-2d08-4923-aaee-bb2f1be3f941 0eac51d15cc544159fc4387cd8e47352 e128a056db4b44708172b186ebcba2b5] Request returned failure status: 404 request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:388
34071 2016-08-03 22:00:48.118 5769 ERROR oslo_messaging.rpc.dispatcher [req-5e24424a-2d08-4923-aaee-bb2f1be3f941 0eac51d15cc544159fc4387cd8e47352 e128a056db4b44708172b186ebcba2b5] Exception during message handling: Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)
34072 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
34073 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
34074 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher executor_callback))
34075 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
34076 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher executor_callback)
34077 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
34078 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
34079 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
34080 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
34081 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/heat/common/context.py", line 300, in wrapped
34082 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher return func(self, ctx, *args, **kwargs)
34083 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 685, in create_stack
34084 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher stack.store()
34085 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
34086 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
34087 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 462, in store
34088 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher trust_ctx = keystone.create_trust_context()
34089 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 233, in create_trust_context
34090 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher trustee_user_id = self.admin_client.auth_ref.user_id
34091 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 122, in admin_client
34092 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher c.authenticate()
34093 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
34094 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher return func(*args, **kwargs)
34095 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 503, in authenticate
34096 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher resp = self.get_raw_token_from_identity_service(**kwargs)
34097 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py", line 281, in get_raw_token_from_identity_service
34098 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher _('Authorization failed: %s') % e)
34099 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher AuthorizationFailure: Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)
34100 2016-08-03 22:00:48.118 5769 TRACE oslo_messaging.rpc.dispatcher
34101 2016-08-03 22:00:48.125 5769 DEBUG oslo_messaging._drivers.amqp [req-5e24424a-2d08-4923-aaee-bb2f1be3f941 0eac51d15cc544159fc4387cd8e47352 e128a056db4b44708172b186ebcba2b5] Pool creating new connection create /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:86
34102
Heat-api.log
2016-08-03 22:31:10.996 1792 DEBUG oslo_messaging._drivers.amqpdriver [req-27116b0d-6a4e-4116-bbfa-fb23495d3003 881bfc13fc504b9f894b95cbb68cc589 3810d898ed3e46efb3307cf32f43df41] MSG_ID is c4583183382e45889999a9683b60db24 _send /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:311
2016-08-03 22:31:10.997 1792 DEBUG oslo_messaging._drivers.amqp [req-27116b0d-6a4e-4116-bbfa-fb23495d3003 881bfc13fc504b9f894b95cbb68cc589 3810d898ed3e46efb3307cf32f43df41] UNIQUE_ID is 906013f3bf174679ab19ae7cf7563d3b. _add_unique_id /usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:252
2016-08-03 22:31:11.045 1792 ERROR heat.common.wsgi [req-27116b0d-6a4e-4116-bbfa-fb23495d3003 881bfc13fc504b9f894b95cbb68cc589 3810d898ed3e46efb3307cf32f43df41] Unexpected error occurred serving API: Remote error: Remote error: AuthorizationFailure Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)
[u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n executor_callback)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/context.py", line 300, in wrapped\n return func(self, ctx, *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 685, in create_stack\n stack.store()\n', u' File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 462, in store\n trust_ctx = keystone.create_trust_context()\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 233, in create_trust_context\n trustee_user_id = self.admin_client.auth_ref.user_id\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 122, in admin_client\n c.authenticate()\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner\n return func(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 503, in authenticate\n resp = self.get_raw_token_from_identity_service(**kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py", line 281, in get_raw_token_from_identity_service\n _(\'Authorization failed: %s\') % e)\n', u'AuthorizationFailure: Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)\n']. None
None.
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi Traceback (most recent call last):
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/heat/common/wsgi.py", line 670, in __call__
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi request, **action_args)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/heat/common/wsgi.py", line 744, in dispatch
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi return method(*args, **kwargs)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/heat/api/openstack/v1/util.py", line 38, in handle_stack_method
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi return handler(controller, req, **kwargs)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/heat/api/openstack/v1/stacks.py", line 300, in create
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 156, in call
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi retry=self.retry)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi timeout=timeout, retry=retry)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 350, in send
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi retry=retry)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 341, in _send
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi raise result
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi RemoteError: Remote error: AuthorizationFailure Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n executor_callback))\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n executor_callback)\n', u' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', u' File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/context.py", line 300, in wrapped\n return func(self, ctx, *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 685, in create_stack\n stack.store()\n', u' File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper\n return f(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/heat/engine/stack.py", line 462, in store\n trust_ctx = keystone.create_trust_context()\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 233, in create_trust_context\n trustee_user_id = self.admin_client.auth_ref.user_id\n', u' File "/usr/lib/python2.7/dist-packages/heat/common/heat_keystoneclient.py", line 122, in admin_client\n c.authenticate()\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner\n return func(*args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 503, in authenticate\n resp = self.get_raw_token_from_identity_service(**kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py", line 281, in get_raw_token_from_identity_service\n _(\'Authorization failed: %s\') % e)\n', u'AuthorizationFailure: Authorization failed: The resource could not be found. (HTTP 404) (HTTP 404)\n'].
2016-08-03 22:31:11.045 1792 TRACE heat.common.wsgi
2016-08-03 22:31:11.046
auth_uri needs to be fixed.