[9.x][SWARM] Command "openstack endpoint list -f json" returns exit code 1

[Alisa Tselovalnikova]

Detailed bug description:

 The issue was found by https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.command_line/141/testReport/(root)/cli_selected_nodes_deploy/

 Command 'source openrc;export OS_IDENTITY_API_VERSION=3;openstack endpoint list -f json' returns exit code 1. Traceback: http://paste.openstack.org/show/590822/

It happened, because the value of OS_DEFAULT_DOMAIN is "Default" instead "default" in openrc.

Description of the environment:

  snapshot #570 http://paste.openstack.org/show/590824/

[Timur Nurlygayanov (tnurlygayanov)]

It looks like the issue can be related to Keystone V3 API or keystone configuration.

[Timur Nurlygayanov (tnurlygayanov)]

Hi Keystone team, could you please take a look?

2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers [req-3a3c7eb8-1f69-429e-b58b-ba8a0da56da0 - - - - -] Could not find domain: Default
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers Traceback (most recent call last):
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/keystone/auth/controllers.py", line 187, in _lookup_domain
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers domain_ref = self.resource_api.get_domain(domain_id)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/keystone/common/manager.py", line 124, in wrapped
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers __ret_val = __f(*args, **kwargs)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1053, in decorate
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers should_cache_fn)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 657, in get_or_create
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers async_creator) as value:
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 158, in __enter__
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers return self._enter()
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 98, in _enter
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers generated = self._enter_create(createdtime)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py", line 149, in _enter_create
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers created = self.creator()
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 625, in gen_value
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers created_value = creator()
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/dogpile/cache/region.py", line 1049, in creator
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers return fn(*arg, **kw)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers File "/usr/lib/python2.7/dist-packages/keystone/resource/core.py", line 706, in get_domain
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers raise exception.DomainNotFound(domain_id=domain_id)
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers DomainNotFound: Could not find domain: Default
2016-11-29 03:47:10.274 22775 ERROR keystone.auth.controllers
2016-11-29 03:47:10.279 22775 WARNING keystone.common.wsgi [req-3a3c7eb8-1f69-429e-b58b-ba8a0da56da0 - - - - -] Authorization failed. The request you have made requires authentication. from 10.109.11.6

[Boris Bobrov (bbobrov)]

This looks very like https://bugs.launchpad.net/mos/+bug/1643804. Are you sure your env is up to date?

[Boris Bobrov (bbobrov)]

Although it looks like https://bugs.launchpad.net/mos/+bug/1643804, it is not it.

The problem is that openrc contains OS_DEFAULT_DOMAIN='Default'. OS_DEFAULT_DOMAIN is expected to contain domain id. Domain id is 'default', with lowercase.

[Nikita Karpin (mkarpin)]

fix is on review - https://review.openstack.org/#/c/404773/

[Timur Nurlygayanov (tnurlygayanov)]

Thank you for the fix!

[TatyanaGladysheva (tgladysheva)]

Verified on 9.2 snapshot #586.

Test is passed since #145 run:
https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.command_line/145/testReport/(root)/cli_selected_nodes_deploy/cli_selected_nodes_deploy/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-library 11.0.0.0rc1 release candidate.

[Dmitry Belyaninov (dbelyaninov)]

Passed on Newton
https://product-ci.infra.mirantis.net/view/10.0_swarm/job/10.0.system_test.ubuntu.command_line/225/