OpenStack Heat

Kilo Heat Authorization Failed

Bug #1471870 reported by Nastooh on 2015-07-06

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Heat	Won't Fix	Medium	Unassigned
	openstack-manuals	Fix Released	Undecided	Unassigned

Bug Description

Hi
On a multi-node ubuntu 14.04 cannot instantiate VMs, due to authorization problem. Followed http://docs.openstack.org/kilo/install-guide/install/apt/content/heat-install-controller-node.html, and below is the step by step verification: (Keystone IP:10.4.0.61 and Heat IP: 10.4.0.64)
1- Added user and owner roles

2- Verified that domain name and domain user names are not used

3- Attempted to create domain and user, with password

$ ./bin/heat-keystone-setup-domain \
> --stack-user-domain-name heat_user_domain \
> --stack-domain-admin heat_domain_admin \
> --stack-domain-admin-password 0stack
Traceback (most recent call last):
  File "./bin/heat-keystone-setup-domain", line 157, in <module>
    main()
  File "./bin/heat-keystone-setup-domain", line 147, in main
    if r.name == 'admin'][0]
IndexError: list index out of range

5- The above command seems to have failed; however, both domain and user were created:

6- Via openstack client added a password for heat_domain_admin:

openstack --os-token $OS_TOKEN --os-url=http://10.4.0.61:5000/v3 --os-identity-api-version=3 user set --password 0stack heat_domain_admin

7-Updated heat.conf

[DEFAULT]
    use_syslog = False
    debug = False
    verbose = False
    log_dir = /var/log/heat
    instance_user=ec2-user
    instance_driver=heat.engine.nova
    plugin_dirs=/usr/lib64/heat,/usr/lib/heat,/home/ubuntu/heat/contrib/heat_docker
    environment_dir=/etc/heat/environment.d
    deferred_auth_method=password
    host=heat
    auth_encryption_key=WCqnhgw972Gbxx4G
    heat_metadata_server_url = http://10.4.0.64:8000
    heat_waitcondition_server_url = http://10.4.0.64:8000/v1/waitcondition
    stack_domain_admin = heat_domain_admin
    stack_domain_admin_password = 0stack
    stack_user_domain_name = heat_user_domain
    # < Icehouse db config
    sql_connection = mysql://heat:jxJBzcLKKwBZcnK2Nd9f7yhxT3ZVd7cS@10.4.0.53/heat
    rabbit_userid = heat
    rabbit_virtual_host = openstack
    rabbit_password = 89phCHJYGshhdydfJKm5hr2pdxhkcGt2nCVTGwXtFMwRdZ8wx25ZRRcbVKHkWFTZ
    rabbit_host = 10.4.0.60
    [keystone_authtoken]
    identity_uri = http://10.4.0.61:35357
    auth_uri = http://10.4.0.61:5000/v2.0
    auth_host = 10.4.0.61
    auth_port = 35357
    auth_protocol = http
    admin_tenant_name = services
    admin_user = heat-cfn_heat
    admin_password = Fht8bfdkMkS6TTFHL9Vs52Bz4dsm5L77m3sTbG2fWyLChZn4KytZrdWGGmLBRxtw
    signing_dir = /var/cache/heat
    [ec2_authtoken]
    auth_uri = http://10.4.0.61:5000/v2.0
    keystone_ec2_uri = http://10.4.0.61:5000/v2.0/ec2tokens
    [database]
    connection = mysql://heat:jxJBzcLKKwBZcnK2Nd9f7yhxT3ZVd7cS@10.4.0.53/heat
    [paste_deploy]
    api_paste_config=/etc/heat/api-paste.ini
    [heat_api]
    bind_port=7994
    [heat_api_cfn]
    bind_port=7990

8- Restarted heat services

service heat-api restart && service heat-api-cfn restart && service heat-engine restart

9- Created a simple yaml file

heat_template_version: 2013-05-23
description: none.
parameters:
  image:
    type: string
    label: Image name or ID
    description: Image to be used.
    default: Ubuntu-14.04
  flavor:
    type: string
    label: Flavor
    description: Type of instance (flavor) to be used on the compute instance.
    default: m1.medium
  key:
    type: string
    label: Key name
    description: Name of key-pair to be installed on the compute instance.
    default: lab_admin
  private_network:
    type: string
    label: Private network name or ID
    description: Network to attach server to.
    default: priv01
resources:
node_instance:
    type: OS::Nova::Server
    properties:
      image: { get_param: image }
      flavor: { get_param: flavor }
      key_name: { get_param: key }
      networks:
        - network: { get_param: private_network }
outputs:
  ip:
    description: The IP address of the instance.
    value: { get_attr: [node_instance, first_address] }

10- Attempted orchestration

$ heat --debug stack-create test -f ../test.yaml
DEBUG (session) REQ: curl -g -i -X GET http://10.4.0.61:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
INFO (connectionpool) Starting new HTTP connection (1): 10.4.0.61
DEBUG (connectionpool) "GET /v2.0 HTTP/1.1" 200 335
DEBUG (session) RESP: [200] content-length: 335 vary: X-Auth-Token x-distribution: Ubuntu connection: keep-alive date: Fri, 03 Jul 2015 19:24:20 GMT content-type: application/json x-openstack-request-id: req-a2412dbb-081f-4420-b0be-e48ed768e9a8
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.4.0.61:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG (v2) Making authentication request to http://10.4.0.61:5000/v2.0/tokens
DEBUG (connectionpool) "POST /v2.0/tokens HTTP/1.1" 200 3902
DEBUG (session) REQ: curl -g -i -X POST http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks -H "Accept: application/json" -H "X-Auth-Token: {SHA1}5a33eb354c9b961f56916ff2e3d9f0318929275f" -H "X-Region-Name: RegionOne" -H "X-Auth-Key: 0stack" -H "X-Auth-User: admin" -H "User-Agent: python-heatclient" -H "Content-Type: application/json" -H "X-Auth-Url: http://10.4.0.61:5000/v2.0" -d '{"files": {}, "disable_rollback": true, "parameters": {}, "stack_name": "test", "environment": {}, "template": {"outputs": {"ip": {"description": "The IP address of the instance.", "value": {"get_attr": ["node_instance", "first_address"]}}}, "heat_template_version": "2013-05-23", "description": "none.", "parameters": {"private_network": {"default": "priv01", "type": "string", "description": "Network to attach server to.", "label": "Private network name or ID"}, "image": {"default": "Ubuntu-14.04", "type": "string", "description": "Image to be used.", "label": "Image name or ID"}, "flavor": {"default": "m1.medium", "type": "string", "description": "Type of instance (flavor) to be used on the compute instance.", "label": "Flavor"}, "key": {"default": "lab_admin", "type": "string", "description": "Name of key-pair to be installed on the compute instance.", "label": "Key name"}}, "resources": {"node_instance": {"type": "OS::Nova::Server", "properties": {"key_name": {"get_param": "key"}, "image": {"get_param": "image"}, "flavor": {"get_param": "flavor"}, "networks": [{"network": {"get_param": "private_network"}}]}}}}}'
INFO (connectionpool) Starting new HTTP connection (1): 10.4.0.64
DEBUG (connectionpool) "POST /v1/8f3633c4ba1640abbab3be608b6bcc88/stacks HTTP/1.1" 201 203
DEBUG (session) RESP: [201] date: Fri, 03 Jul 2015 19:24:23 GMT connection: keep-alive content-type: application/json content-length: 203 location: http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/test/6ff28ea2-aff3-4282-b626-58e7fdaa11c2
RESP BODY: {"stack": {"id": "6ff28ea2-aff3-4282-b626-58e7fdaa11c2", "links": [{"href": "http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/test/6ff28ea2-aff3-4282-b626-58e7fdaa11c2", "rel": "self"}]}}

DEBUG (session) REQ: curl -g -i -X GET http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks? -H "Accept: application/json" -H "User-Agent: python-heatclient" -H "X-Region-Name: RegionOne" -H "X-Auth-Token: {SHA1}5a33eb354c9b961f56916ff2e3d9f0318929275f" -H "Content-Type: application/json" -H "X-Auth-Url: http://10.4.0.61:5000/v2.0"
DEBUG (connectionpool) "GET /v1/8f3633c4ba1640abbab3be608b6bcc88/stacks HTTP/1.1" 200 957
DEBUG (session) RESP: [200] date: Fri, 03 Jul 2015 19:24:23 GMT content-length: 957 content-type: application/json; charset=UTF-8 connection: keep-alive
RESP BODY: {"stacks": [{"parent": null, "description": "none.", "links": [{"href": "http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/test/6ff28ea2-aff3-4282-b626-58e7fdaa11c2", "rel": "self"}], "stack_status_reason": "", "stack_name": "test", "stack_user_project_id": null, "creation_time": "2015-07-03T19:24:22Z", "updated_time": null, "stack_owner": "admin", "stack_status": "CREATE_IN_PROGRESS", "id": "6ff28ea2-aff3-4282-b626-58e7fdaa11c2"}, {"parent": null, "description": "No description", "links": [{"href": "http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/vm/77cac2ff-9102-4fd2-bc85-dedce939b102", "rel": "self"}], "stack_status_reason": "Stack CREATE completed successfully", "stack_name": "vm", "stack_user_project_id": "8f3633c4ba1640abbab3be608b6bcc88", "creation_time": "2015-06-19T01:37:18Z", "updated_time": null, "stack_owner": "admin", "stack_status": "CREATE_COMPLETE", "id": "77cac2ff-9102-4fd2-bc85-dedce939b102"}]}

+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| 6ff28ea2-aff3-4282-b626-58e7fdaa11c2 | test | CREATE_IN_PROGRESS | 2015-07-03T19:24:22Z |
+--------------------------------------+------------+--------------------+----------------------+

11- And noted the failure

$ heat --debug stack-list
DEBUG (session) REQ: curl -g -i -X GET http://10.4.0.61:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
INFO (connectionpool) Starting new HTTP connection (1): 10.4.0.61
DEBUG (connectionpool) "GET /v2.0 HTTP/1.1" 200 335
DEBUG (session) RESP: [200] content-length: 335 vary: X-Auth-Token x-distribution: Ubuntu connection: keep-alive date: Fri, 03 Jul 2015 19:24:25 GMT content-type: application/json x-openstack-request-id: req-e1a05a9c-e8a1-4443-aba5-f4d9bb727c1b
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.4.0.61:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG (v2) Making authentication request to http://10.4.0.61:5000/v2.0/tokens
DEBUG (connectionpool) "POST /v2.0/tokens HTTP/1.1" 200 3902
DEBUG (session) REQ: curl -g -i -X GET http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks? -H "Accept: application/json" -H "User-Agent: python-heatclient" -H "X-Region-Name: RegionOne" -H "X-Auth-Token: {SHA1}d5eb18c31f9b840186fd2e279679ce6a10dc278a" -H "Content-Type: application/json" -H "X-Auth-Url: http://10.4.0.61:5000/v2.0"
INFO (connectionpool) Starting new HTTP connection (1): 10.4.0.64
DEBUG (connectionpool) "GET /v1/8f3633c4ba1640abbab3be608b6bcc88/stacks HTTP/1.1" 200 973
DEBUG (session) RESP: [200] date: Fri, 03 Jul 2015 19:24:26 GMT content-length: 973 content-type: application/json; charset=UTF-8 connection: keep-alive
RESP BODY: {"stacks": [{"parent": null, "description": "none.", "links": [{"href": "http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/test/6ff28ea2-aff3-4282-b626-58e7fdaa11c2", "rel": "self"}], "stack_status_reason": "Authorization failed.", "stack_name": "test", "stack_user_project_id": null, "creation_time": "2015-07-03T19:24:22Z", "updated_time": null, "stack_owner": "admin", "stack_status": "CREATE_FAILED", "id": "6ff28ea2-aff3-4282-b626-58e7fdaa11c2"}, {"parent": null, "description": "No description", "links": [{"href": "http://10.4.0.64:8004/v1/8f3633c4ba1640abbab3be608b6bcc88/stacks/vm/77cac2ff-9102-4fd2-bc85-dedce939b102", "rel": "self"}], "stack_status_reason": "Stack CREATE completed successfully", "stack_name": "vm", "stack_user_project_id": "8f3633c4ba1640abbab3be608b6bcc88", "creation_time": "2015-06-19T01:37:18Z", "updated_time": null, "stack_owner": "admin", "stack_status": "CREATE_COMPLETE", "id": "77cac2ff-9102-4fd2-bc85-dedce939b102"}]}

+--------------------------------------+------------+-----------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+-----------------+----------------------+
| 6ff28ea2-aff3-4282-b626-58e7fdaa11c2 | test | CREATE_FAILED | 2015-07-03T19:24:22Z |
+--------------------------------------+------------+-----------------+----------------------+

P.S Problem seems to be specific to heat-keystone-setup-domain. i.e. using openstack client circumvents the problem. That is
"openstack --os-token $OS_TOKEN --os-url=http://10.4.0.61:5000/v3 --os-identity-api-version=3 role add --user heat_domain_admin --domain <domain id> Admin" works fine. (Could it be it is expecting "admin" as administrative group name, as opposed to "Admin".)

Tags: