Need Rally deployment create support on domain_id or domain_name in order to support keystone v3 test (which needs domain scoped token). The current project scoped token only works in service API but not keystone v3 API (DTUG domain, user, group...).

Bug #1441379 reported by apal on 2015-04-07
60
This bug affects 12 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Undecided
Unassigned
Rally
Medium
Unassigned

Bug Description

For keystone v3 testing, most test should based on domain scoped token but not project scoped token. And either domain_name/domain_id is required for keystone v3 domain scoped token. Rally need to pick up either domain_id or domain_name in its v3 config, in order to work with keystone v3:
Failed validating 'anyOf' in schema:
    {'anyOf': [{'properties': {'admin': {'$ref': '#/definitions/user'}},
                'required': ['type', 'auth_url', 'admin']},
               {'required': ['type', 'auth_url', 'users'],
                'users': {'items': {'$ref': '#/definitions/user'},
                          'type': 'array'}}],
     'definitions': {'user': {'oneOf': [{'properties': {'tenant_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'tenant_name']},
                                        {'properties': {'project_domain_name': {'type': 'string'},
                                                        'project_name': {'type': 'string'},
                                                        'user_domain_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'project_name']}],
                              'properties': {'password': {'type': 'string'},
                                             'username': {'type': 'string'}},
                              'type': 'object'}},
     'properties': {'auth_url': {'type': 'string'},
                    'endpoint_type': {'enum': ['admin',
                                               'internal',
                                               'public'],
                                      'type': 'string'},
                    'region_name': {'type': 'string'},
                    'type': {'type': 'string'}},
     'type': 'object'}

Here's a test:
Project scoped token deployment looks fine, but domain scoped token deployment failed. Check the following test:

1. Project scoped token deploy:
$ cat m.json
{
    "admin": {
        "project_name": "admin",
        "password": "admin",
        "project_domain_name": "Default",
        "user_domain_name": "Default",
        "username": "admin"
    },
    "auth_url": "https://localhost:5000/v3/",
    "endpoint_type": "public",
    "https_cacert": "/usr/local/share/ca-certificates/ephemeralca-cacert.crt",
    "https_insecure": False,
    "region_name": "regionOne",
    "type": "ExistingCloud",
}

$ rally deployment create --filename=m.json --name=apaltestHAKVMv3
+--------------------------------------+----------------------------+-----------------+------------------+--------+
| uuid | created_at | name | status | active |
+--------------------------------------+----------------------------+-----------------+------------------+--------+
| 01ca22c6-9ad0-422b-ae77-7e9d863234b0 | 2015-04-07 22:33:55.921296 | apaltestHAKVMv3 | deploy->finished | |
+--------------------------------------+----------------------------+-----------------+------------------+--------+
Using deployment: 01ca22c6-9ad0-422b-ae77-7e9d863234b0
~/.rally/openrc was updated

HINTS:
* To get your cloud resources, run:
        rally show [flavors|images|keypairs|networks|secgroups]

* To use standard OpenStack clients, set up your env by running:
        source ~/.rally/openrc
  OpenStack clients are now configured, e.g run:
        glance image-list

$ rally deployment check
keystone endpoints are valid and following services are available:
+------------+---------------+-----------+
| services | type | status |
+------------+---------------+-----------+
| ceilometer | metering | Available |
| cinder | volume | Available |
| cinderv2 | volumev2 | Available |
| ec2 | ec2 | Available |
| glance | image | Available |
| heat | orchestration | Available |
| keystone | identity | Available |
| neutron | network | Available |
| nova | compute | Available |
| sherpa | hp-catalog | Available |
| swift | object-store | Available |
+------------+---------------+-----------+

$ rally task start ../keystonecase/apaldebug-create-1-user.json
--------------------------------------------------------------------------------
 Preparing input task
--------------------------------------------------------------------------------

Input task is:
{
    "KeystoneBasic.create_user": [
        {
            "args": {
                "name_length": 10
            },
            "runner": {
                "type": "constant",
                "times": 1,
                "concurrency": 1
            }
        }
    ]
}

--------------------------------------------------------------------------------
 Task 76a1c52c-b085-4243-ae47-4f5cfaab8091: started
--------------------------------------------------------------------------------

Benchmarking... This can take a while...

To track task status use:

        rally task status
        or
        rally task detailed

2015-04-07 22:34:40.916 20445 WARNING rally.common.broker [-] Failed to consume a task from the queue: You are not authorized to perform the requested action: identity:get_domain (HTTP 403)

--------------------------------------------------------------------------------
Task 76a1c52c-b085-4243-ae47-4f5cfaab8091: failed
--------------------------------------------------------------------------------
<class 'rally.exceptions.ContextSetupFailure'>
Unable to setup context 'users': 'Failed to create the requested number of tenants.'

For more details run:
rally -vd task detailed 76a1c52c-b085-4243-ae47-4f5cfaab8091
Using task: 76a1c52c-b085-4243-ae47-4f5cfaab8091

2. Project scoped token deploy failed: - I just updated one line in this config -> from ""project_name": "admin"," to ""domain_name": "Default",":

$ cat u.json
{
    "admin": {
        "domain_name": "Default",
        "password": "admin",
        "project_domain_name": "Default",
        "user_domain_name": "Default",
        "username": "admin"
    },
    "auth_url": "https://localhost:5000/v3/",
    "endpoint_type": "public",
    "https_cacert": "/usr/local/share/ca-certificates/ephemeralca-cacert.crt",
    "https_insecure": False,
    "region_name": "regionOne",
    "type": "ExistingCloud",
}

$ rally deployment create --filename=u.json --name=apaltestHAKVMv3
2015-04-07 22:36:18.996 20739 ERROR rally.api [-] Deployment 44f819e6-deb9-4109-a52b-f751825e7f9c: Schema validation error.
Config schema validation error: {'endpoint_type': 'public', 'auth_url': 'https://localhost:5000/v3/', 'region_name': 'regionOne', 'https_insecure': False, 'admin': {'username': 'admin', 'user_domain_name': 'Default', 'password': 'admin', 'domain_name': 'Default', 'project_domain_name': 'Default'}, 'https_cacert': '/usr/local/share/ca-certificates/ephemeralca-cacert.crt', 'type': 'ExistingCloud'} is not valid under any of the given schemas

Failed validating 'anyOf' in schema:
    {'anyOf': [{'properties': {'admin': {'$ref': '#/definitions/user'}},
                'required': ['type', 'auth_url', 'admin']},
               {'required': ['type', 'auth_url', 'users'],
                'users': {'items': {'$ref': '#/definitions/user'},
                          'type': 'array'}}],
     'definitions': {'user': {'oneOf': [{'properties': {'tenant_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'tenant_name']},
                                        {'properties': {'project_domain_name': {'type': 'string'},
                                                        'project_name': {'type': 'string'},
                                                        'user_domain_name': {'type': 'string'}},
                                         'required': ['username',
                                                      'password',
                                                      'project_name']}],
                              'properties': {'password': {'type': 'string'},
                                             'username': {'type': 'string'}},
                              'type': 'object'}},
     'properties': {'auth_url': {'type': 'string'},
                    'endpoint_type': {'enum': ['admin',
                                               'internal',
                                               'public'],
                                      'type': 'string'},
                    'region_name': {'type': 'string'},
                    'type': {'type': 'string'}},
     'type': 'object'}

On instance:
    {'admin': {'domain_name': 'Default',
               'password': 'admin',
               'project_domain_name': 'Default',
               'user_domain_name': 'Default',
               'username': 'admin'},
     'auth_url': 'https://localhost:5000/v3/',
     'endpoint_type': 'public',
     'https_cacert': '/usr/local/share/ca-certificates/ephemeralca-cacert.crt',
     'https_insecure': False,
     'region_name': 'regionOne',
     'type': 'ExistingCloud'}.

apal (yun-song) wrote :

NOTE:
Before move on keystone v3 test, we need to get domain scoped token work in rally.
Project scoped token is only for service operation but not for keystone domain/user/group/project/role/... DTUG.

apal (yun-song) wrote :

For example, if source the project scoped token that rally config support at the current phase, role list failed with the same 401 error:
$ . ~/.rally/openrc
OS_REGION_NAME=regionOne
OS_USER_DOMAIN_NAME=Default
OS_PROJECT_NAME=default
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=admin
OS_AUTH_URL=https://localhost:5000/v3/
OS_USERNAME=admin
OS_CACERT=/usr/local/share/ca-certificates/ephemeralca-cacert.crt
OS_PROJECT_DOMAIN_NAME=Default
cetest@cer305n0016:~/ysong/rally/environments$ openstack role list
ERROR: openstack The request you have made requires authentication. (HTTP 401)

description: updated
summary: - rally deployment create failed on schema validation, if using domain
- scoped token deployment
+ Need Rally deployment create support on domain_id or domain_name in
+ order to support keystone v3 test (which needs domain scoped token). The
+ current project scoped token only works in service API but not keystone
+ v3 API (DTUG domain, user, group...).
Lin Hua Cheng (lin-hua-cheng) wrote :

Seems like all the work are in Rally. Can you add more info what change is needed from Keystone to support domain scoped testing in Rally?

Changed in keystone:
status: New → Incomplete
apal (yun-song) wrote :

LinHua,
I think keystone works fine with domain scoped token for keystone v3 APIs, like createDomain, createUser, createRole.
But Rally only supports project scoped token.
The better choice may be have Rally support domain scoped token; but not have keystone support project scoped token in keystoneAPIs (createDomain, createUser, createRole). At this point, keystone don't need to do anything if Rally can fix this issue. Your thoughts?

Thanks!
Apal

Changed in rally:
status: New → Triaged
importance: Undecided → Medium
tags: added: deployment
removed: rally
Steve Martinelli (stevemar) wrote :

sounds like this is a rally issue, marking keystone as invalid

Changed in keystone:
status: Incomplete → Invalid
Andrey Kurilin (andreykurilin) wrote :

Please, specify both project_name and domain_name. It should work for you on latest Rally

Changed in rally:
status: Triaged → Fix Committed
Dmitry Galkin (galkindmitrii) wrote :
Download full text (4.6 KiB)

Hi Andrey,

Could you please confirm that specifying both 'project_name' and 'domain_name' works with Keystone V3 API and current Rally/master branch?

I can do 'rally deployment create' with both options specified, but 'rally deployment check' results in a traceback from Keystone-Client in such case:

[root@hannibal openstack_rally]# rally deployment check
Command failed, please check log for more info
2016-03-07 15:06:30.859 29419 CRITICAL rally [-] AuthorizationFailure: Authentication cannot be scoped to multiple targets. Pick one of: project, domain, trust or unscoped
2016-03-07 15:06:30.859 29419 ERROR rally Traceback (most recent call last):
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/bin/rally", line 10, in <module>
2016-03-07 15:06:30.859 29419 ERROR rally sys.exit(main())
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/cli/main.py", line 40, in main
2016-03-07 15:06:30.859 29419 ERROR rally return cliutils.run(sys.argv, categories)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/cli/cliutils.py", line 560, in run
2016-03-07 15:06:30.859 29419 ERROR rally ret = fn(*fn_args, **fn_kwargs)
2016-03-07 15:06:30.859 29419 ERROR rally File "<string>", line 2, in check
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/cli/envutils.py", line 67, in default_from_global
2016-03-07 15:06:30.859 29419 ERROR rally return f(*args, **kwargs)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/cli/commands/deployment.py", line 226, in check
2016-03-07 15:06:30.859 29419 ERROR rally services = api.Deployment.check(deployment)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/api.py", line 139, in check
2016-03-07 15:06:30.859 29419 ERROR rally services = cls.service_list(deployment)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/api.py", line 123, in service_list
2016-03-07 15:06:30.859 29419 ERROR rally return clients.services()
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/osclients.py", line 721, in services
2016-03-07 15:06:30.859 29419 ERROR rally ks = self.keystone()
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/osclients.py", line 203, in __call__
2016-03-07 15:06:30.859 29419 ERROR rally self.cache[key] = self.create_client(*args, **kwargs)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/osclients.py", line 256, in create_client
2016-03-07 15:06:30.859 29419 ERROR rally return self._create_keystone_client(kw)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/rally/osclients.py", line 242, in _create_keystone_client
2016-03-07 15:06:30.859 29419 ERROR rally auth_ref = auth.get_access(session)
2016-03-07 15:06:30.859 29419 ERROR rally File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 240, in get_access
2016-03-07 15:06:30.859 29419 ERROR rally self.auth_ref = ...

Read more...

Andrey Kurilin (andreykurilin) wrote :

Hi Dmitry,
It looks like we should have stricter config validation.
Next config works for me in case of Keystone v3:

{
    "admin": {
        "admin_domain_name": "Default",
        "password": "password",
        "project_name": "admin",
        "user_domain_name": "Default",
        "username": "admin"
    },
    "auth_url": "http://host_name:5000/v3/",
    "endpoint_type": "public",
    "https_cacert": "",
    "https_insecure": false,
    "type": "ExistingCloud"
}

Kavitha (kavitha-subramaniam) wrote :

Hi Andrey,
The above config is with 'project scope' and not with 'domain scope', hence it works.

Agree with Dmitry that we can not simultaneously scope a token to a project and domain.

Is anybody working on this issue or is there a fix available that we can pull?

Thanks.

Michael Gale (gale-michael) wrote :

Hello,

    Is anyone working on this issue? If not I might be able to help.

Let me know, thanks.

Michael

Andrey Kurilin (andreykurilin) wrote :

@Michael: I do not know anyone how work on this issue now. I'll be really appreciate for any help

Changed in keystone:
assignee: nobody → Kavitha (kavitha-subramaniam)
assignee: Kavitha (kavitha-subramaniam) → nobody
Changed in rally:
assignee: nobody → Kavitha (kavitha-subramaniam)
status: Fix Committed → In Progress
Jay Jahns (jjahns) wrote :

Is this going to get an update soon?

Not being able to run tests because of this limitation is kind of a big deal if the cloud environment has multiple domains.

Admin needs to be able create the tenants and the users

Users need to be able to create the networks, instances, volumes, etc.

Without this, can't really do anything, unless someone has documentation they'd like to share!!!

Kavitha (kavitha-subramaniam) wrote :

I am not currently working on this issue..

Changed in rally:
assignee: Kavitha (kavitha-subramaniam) → nobody
laura (mysterious03) wrote :
Download full text (14.4 KiB)

Hi,

I met below errors when trying to use keystone v3 endpoint using rally 0.7.0. Could someone help me how to fix or workaround these errors? I plan to use rally benchmark test. Thanks!

Error 1:Here is my rally deploy file:
cat rally_deploy.json
{
  "type": "ExistingCloud",
  "auth_url": "http://**.**.**.**:35357/v3/",
  "region_name": "nova",
  "admin": {
    "username": "admin",
    "password": "admin",
    "project_name": "admin",
    "project_domain_name": "Default",
    "user_domain_name": "Default"
  },
  "users": [
    {
      "username": "tenant4_user1",
      "password": "***",
      "project_name": "tenant4"
    },
    {
      "username": "tenant3_user1",
      "password": "***",
      "project_name": "tenant3"
    },
    {
      "username": "tenant2_user1",
      "password": "***",
      "project_name": "tenant2"
    },
    {
      "username": "tenant1_user1",
      "password": "**",
      "project_name": "tenant1"
    }
  ],
  "https_insecure": True,
  "https_cacert": ""
}
rally deployment create succeeded, but rally deployment check failed as below.
rally@e2bfc2794a2a:~$ rally deployment check
2016-11-22 06:13:19.501 475 CRITICAL rally [-] BadRequest: Expecting to find domain in project - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-6b4902bc-5e8e-4b5c-8dee-17bf75990b3f)
2016-11-22 06:13:19.501 475 ERROR rally Traceback (most recent call last):
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/bin/rally", line 10, in <module>
2016-11-22 06:13:19.501 475 ERROR rally sys.exit(main())
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/main.py", line 40, in main
2016-11-22 06:13:19.501 475 ERROR rally return cliutils.run(sys.argv, categories)
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/cliutils.py", line 581, in run
2016-11-22 06:13:19.501 475 ERROR rally ret = fn(*fn_args, **fn_kwargs)
2016-11-22 06:13:19.501 475 ERROR rally File "<decorator-gen-17>", line 2, in check
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/envutils.py", line 67, in default_from_global
2016-11-22 06:13:19.501 475 ERROR rally return f(*args, **kwargs)
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/commands/deployment.py", line 227, in check
2016-11-22 06:13:19.501 475 ERROR rally services = api.Deployment.check(deployment)
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/api.py", line 144, in check
2016-11-22 06:13:19.501 475 ERROR rally osclients.Clients(objects.Credential(**endpoint_dict)).keystone()
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/osclients.py", line 208, in __call__
2016-11-22 06:13:19.501 475 ERROR rally self.cache[key] = self.create_client(*args, **kwargs)
2016-11-22 06:13:19.501 475 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/osclients.py", line 323, in create_client
2016-11...

Jay Jahns (jjahns) wrote :

So I was able to get it to work, but there needs to be and endpoint_type in the config and each user needs user_domain_name, project_domain_name, and project_name, just like admin.

Also, I used port 5000 instead of 35357.

But, this doesn't solve the problem for creating the projects/users tests, and existing users *have* to be created for this purpose.

Would be nice if rally did the following:

For rally deployment check:
* Have to run with project-scoped

For user and project creation
* Check for a domain_name in the admin config
* If not there, fail if the auth_url uses "V3"
* If it is there, then switch to domain-scoped

Problem would be solved at that point. Projects/Users would run under the domain-scoped token automatically, while instances, networks, volumes, etc. would use project_scoped tokens.

My deployment is created and checked using this json file.

{
    "type": "ExistingCloud",
    "auth_url": "http://192.168.90.200:5000/v3/",
    "region_name": "RegionOne",
    "endpoint_type": "public",
    "admin": {
        "username": "cloud_admin",
        "password": "XXXXXX",
        "user_domain_name": "admin_domain",
        "project_name": "admin",
        "project_domain_name": "admin_domain"
    },
    "https_insecure": false,
    "https_cacert": ""
}

bu i cant run the user /projects creation task, isnt there still a way to do this ?
best.

Jay Jahns (jjahns) wrote :

Any chance we can get an update on this?

Without the ability to create projects (i.e. domain-scope), raising the concurrency level on a scenario and being able to test things is ugly, if not impossible.

I don't want to have to create 100 different users to just use for rally, and there is no guarantee it works correctly.

Please advise. This has been a long standing problem that while I'll concede, could be improved from the auth side, Rally should account for!

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers