Tempest against openstack deployed with keystone v3 only, fails to initialize

Bug #1451987 reported by Imran Hayder
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
tempest
Fix Released
Medium
Andrea Frittoli

Bug Description

PROBLEM:
When running tempest against openstack that has only keystone v3 established( no identity v2) , tempest requires us to set the value of "uri" in tempest.conf even though "uri_v3" is set and it should suffice. This was observed when first:
i) uri was set to None , and uri_v3 to <keystone_url>:5001/v3 .
ii) api_v2 was set to "false" and api_v3 was set to "true".

The following stacktrace was seen:

 01:24:12 ======================================================================
01:24:12 FAIL: setUpClass (tempest.api.compute.admin.test_agents.AgentsAdminTestJSON)
01:24:12 ----------------------------------------------------------------------
01:24:12 Traceback (most recent call last):
01:24:12 testtools.testresult.real._StringException: Traceback (most recent call last):
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 255, in setUpClass
01:24:12 cls.setup_credentials()
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/api/compute/base.py", line 54, in setup_credentials
01:24:12 super(BaseComputeTest, cls).setup_credentials()
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 338, in setup_credentials
01:24:12 credential_type=credentials_type)
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 416, in get_client_manager
01:24:12 identity_version=identity_version
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/credentials.py", line 39, in get_isolated_credentials
01:24:12 identity_version=identity_version)
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/isolated_creds.py", line 151, in __init__
01:24:12 self._get_admin_clients())
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/isolated_creds.py", line 169, in _get_admin_clients
01:24:12 os = clients.Manager(self.default_admin_creds)
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/clients.py", line 168, in __init__
01:24:12 self._set_identity_clients()
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/clients.py", line 347, in _set_identity_clients
01:24:12 **self.default_params)
01:24:12 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/.venv/lib/python2.7/site-packages/tempest_lib/services/identity/v2/token_client.py", line 30, in __init__
01:24:12 if 'tokens' not in auth_url:
01:24:12 TypeError: argument of type 'NoneType' is not iterable

===========================================================================

WORKAROUND:
After that , setting uri value to same as uri_v3 ( because we only have v3) , it got past this initialization.
This suggests tempest should have a better check for :

>> if only v3 is enabled, then setting uri_v3 should suffice and uri should be ignored , based now what we set the bool for api_v2 and api_v3 parameters in tempest.conf.

Tags: v3
description: updated
tags: added: v3
summary: - Tempest against cloud deployed with keystone v3 only fails to initialize
+ Tempest against openstack deployed with keystone v3 only, fails to
+ initialize
Jin Liu (jin-t)
Changed in tempest:
status: New → Confirmed
Revision history for this message
Imran Hayder (hayderimran7) wrote :

UPDATE ON THIS BUG:

We finally hacked around tempest.conf to make it work so tempest can successfully initiate against openstack deployed with Keystone V3 ( remember our cloud has V3 only, there is no v2 so following options are only valid if your cloud is deployed in same way ie.e with keystone v3).

* First thing we did, we commented everything in tempest.conf that was related to tenant. The concept of tenant is entirely different in keystone v3 so parameters like "admin_tenant_id* , *tenant_id* , *admin_tenant_name* ALL NEED TO BE COMMENTED OUT. You should not set any value whatsoever for these params if your cloud only has v3.

* You should give some value for "uri" even though you already set "uri_v3" to correct keystone v3 url , you set api_v2=false and api_v3=true and also set auth_version=3, it WILL STILL NEED YOU TO SET VALUE OF "uri". This is a bug in tempest that needs urgent addressing, and for that workaround, we just change v3 to v2.0 in our OS_AUTH_URL and assigned it to "uri"

e.g. lets say our OS_AUTH_URL=htttp://10.0.2.15:35357/v3 so uri_v3 will be same as OIS_AUTH__URL.

but you should not leave uri empty, doesnt make sense since cloud only has v3 but hopefully this issue will be solved sooner rather than later by tempest, so lets see how to assign uri.
on bash you will do : KS_V2_URL= `echo $OS_AUTH_URL | sed 's/v3/v2.0/;' `
so KS_V2_URL will be the value of uri in tempest.conf.

=================================================================================

Revision history for this message
Imran Hayder (hayderimran7) wrote :
Download full text (3.5 KiB)

If you see following errors for all identity api v3 tests, then please be known that its not a a bug in tempest, rather you need to change keystone v3 policy.json and make it more relaxed so tempest can authorize with users created for each test with separate projects(tenants) because we set tenant_isolation to True in tempest.conf ...

=========================================================================

10:20:47 ======================================================================
10:20:47 FAIL: setUpClass (tempest.api.orchestration.stacks.test_templates.TemplateYAMLTestJSON)
10:20:47 ----------------------------------------------------------------------
10:20:47 Traceback (most recent call last):
10:20:47 testtools.testresult.real._StringException: Traceback (most recent call last):
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 255, in setUpClass
10:20:47 cls.setup_credentials()
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/api/orchestration/base.py", line 42, in setup_credentials
10:20:47 super(BaseOrchestrationTest, cls).setup_credentials()
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 338, in setup_credentials
10:20:47 credential_type=credentials_type)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/test.py", line 416, in get_client_manager
10:20:47 identity_version=identity_version
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/credentials.py", line 39, in get_isolated_credentials
10:20:47 identity_version=identity_version)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/isolated_creds.py", line 160, in __init__
10:20:47 self.identity_admin_client, self.creds_domain_name)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/isolated_creds.py", line 134, in get_creds_client
10:20:47 return V3CredsClient(identity_client, project_domain_name)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/common/isolated_creds.py", line 106, in __init__
10:20:47 params={'name': domain_name})[0]
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/tempest/services/identity/v3/json/identity_client.py", line 250, in list_domains
10:20:47 resp, body = self.get(url)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/.venv/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 267, in get
10:20:47 return self.request('GET', url, extra_headers, headers)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/.venv/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 629, in request
10:20:47 resp, resp_body)
10:20:47 File "/var/lib/jenkins/workspace/tempest_v3/tempest-repo/tempest/.venv/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 672, in _error_checker
10:20:47 raise exceptions.Forbidden(resp_body)
10:20:47 tempest_lib.exceptions.Forbidden: Forbidden
...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/182629

Changed in tempest:
assignee: nobody → Andrea Frittoli (andrea-frittoli)
status: Confirmed → In Progress
Changed in tempest:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.openstack.org/182629
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=03e546fe038f5d7617e7e450c471cd578c0c2e82
Submitter: Jenkins
Branch: master

commit 03e546fe038f5d7617e7e450c471cd578c0c2e82
Author: Andrea Frittoli (andreaf) <email address hidden>
Date: Wed May 13 12:44:47 2015 +0100

    Only create a TokenClient if api_v2 is enabled

    Token clients should only be setup if the corresponding API
    version is marked as available. If they are configured as
    available but the URI is not set, the configuration is invalid.

    Change-Id: Ib2885dbb04af3dc12724fe50bed8ce73ec3c426c
    Closes-bug: #1451987

Changed in tempest:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.