token issue fails for keystone v2 if OS_PROJECT_DOMAIN_NAME or OS_USER_DOMAIN_NAME are set

[Matthew Edmonds]

"/usr/bin/openstack --insecure token issue" fails when OS_AUTH_URL and OS_IDENTITY_API_VERSION indicate keystone v2 if OS_PROJECT_DOMAIN_NAME or OS_USER_DOMAIN_NAME are set. Since domains are irrelevant to v2, those env vars should have been ignored.

If I run with --debug, this appears to be the problem:

ERROR: openstackclient.shell Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 176, in run
    return super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 215, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 280, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/display.py", line 91, in run
    column_names, data = self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/openstackclient/identity/v2_0/token.py", line 39, in take_action
    token = self.app.client_manager.auth_ref.service_catalog.get_token()
  File "/usr/lib/python2.7/site-packages/openstackclient/common/clientmanager.py", line 174, in auth_ref
    self._auth_ref = self.auth.get_auth_ref(self.session)
  File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/generic/base.py", line 174, in get_auth_ref
    self._plugin = self._do_create_plugin(session)
  File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/generic/base.py", line 170, in _do_create_plugin
    raise exceptions.DiscoveryFailure(msg)
DiscoveryFailure: Could not determine a suitable URL for the plugin

e.g. env vars:

OS_REGION_NAME=RegionOne
OS_IDENTITY_API_VERSION=2
OS_AUTH_URL=https://myhostname:5000/v2.0/
OS_USERNAME=myuser
OS_PASSWORD=mypassword
OS_TENANT_NAME=service
OS_PROJECT_NAME=service
OS_USER_DOMAIN_NAME=Service
OS_PROJECT_DOMAIN_NAME=Service

seen with openstackclient 1.0.3 and keystoneclient 1.1.0

[Dean Troyer (dtroyer)]

This also occurs with --os-auth-type password, forcing a bypass of OSC's broken endpoint hack. It is actually a two-part problem:

1) There appears to be a design choice in Keystone client. In keystoneclient/auth/identity/generic/base.py _do_create_plugin() line 151 (in my tree) we see:

                if (_discover.version_match((2,), version) and
                        self._has_domain_scope):
                    # NOTE(jamielennox): if there are domain parameters there
                    # is no point even trying against v2 APIs.
                    continue

It appears that KSC assumes that you want v3 whenever domain args are present.

2) The user's selection of v2 is not enforced because OSC sets a default auth-type of osc_password. That really needs to be removed so discovery can work properly and --os-identity-api-version is honored.

In short, the obvious workaround is to not set domain args if you want to use v2 auth.

[Dean Troyer (dtroyer)]

I will remove the default auth-type when --os-token and --os-url are not present.

We can discuss KSC's behaviour here.

[Jamie Lennox (jamielennox)]

Dean: i was thinking of this a while ago and proposed: https://review.openstack.org/#/c/171462/ - would that solve the problem? Unfortunately it'll still take some time to filter through to OSC.

[Dean Troyer (dtroyer)]

Jamie, that's a good change but I don't think it solves this bug. Thinking about it some more I think I'm going to just have to handle this in OSC as v2 is being explicitly requested so OSC should probably not try generic, which is all osc_password knows. If I could always use the generic classes and pass in a version when user requests a specific one that would simplify things for me. I may just not know how to do it.

[Morgan Fainberg (mdrnstm)]

Based on the conversation here, Dean said he's going to handle this in openstackclient. I'm going to go ahead and mark this as "wont fix" in keystoneclient. This might be improved with KSC 2.0 and the move to "keystoneauth" (among other things).

[sameerz (sameerz)]

I have the same issue when I try to launch an instance. Both "nova boot" and "openstack server create" commands fail. I'm running a Liberty based OpenStack with v3 Keystone.

What would be the workaround in my case? Reading the comments above it seems I need to go back to keystone v2!

[Tomi Juvonen (tomi-juvonen-q)]

Hope this helps, but I could solve problem in devstack ./stack.sh run by correcting my ~/.config/openstack/clouds.yaml not to refer to v2.0 to have openstackclient working. Otherwise one can "export OS_AUTH_URL=https://myhostname:5000" instead of OS_AUTH_URL=https://myhostname:5000/v2.0/ to have CLI working.

[sameerz (sameerz)]

Unfortunately, removing the '/v3' string from OS_AUTH_URL didn't help (I'm running with keystone v3 not v2.0):

2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions [req-656f8977-9450-4fe1-8383-4505d6d39123 8e7e4050ae724f02ae0b1bfc473a3b66 2c53928125d844eab95f474de73bac21 - - -] Unexpected exception in API method
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 611, in create
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions **create_kwargs)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 149, in inner
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions rv = f(*args, **kwargs)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1581, in create
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions check_server_group_quota=check_server_group_quota)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1181, in _create_instance
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions auto_disk_config, reservation_id, max_count)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 955, in _validate_and_build_base_options
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions pci_request_info, requested_networks)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1059, in create_pci_requests_for_sriov_ports
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions neutron = get_client(context, admin=True)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 237, in get_client
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions auth_token = _ADMIN_AUTH.get_token(_SESSION)
2015-11-27 02:24:48.321 1579 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/keystonecl...

[sameerz (sameerz)]

Please ignore my comments above. The issue was due to a mis-configuration on my part. Nothing to do with this bug report. Sorry for the noise!

[Steve Martinelli (stevemar)]

sameerz, thanks for confirming you are not running into an issue, but the original bug is still valid:

export OS_IDENTITY_API_VERSION=2
export OS_AUTH_URL=https://localhost:5000/v2.0/
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_TENANT_NAME=admin
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default

stevemar@ubuntu:/opt/stack/keystoneauth$ openstack token issue --debug
START with options: ['token', 'issue', '--debug']
options: Namespace(access_token_endpoint='', auth_type='', auth_url='https://localhost:5000/v2.0/', cacert='', client_id='', client_secret='***', cloud='', debug=True, default_domain='default', deferred_help=False, domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', log_file=None, os_baremetal_api_version='1.6', os_compute_api_version='', os_data_processing_api_version='1.1', os_identity_api_version='2', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', password='***', project_domain_id='', project_domain_name='Default', project_id='', project_name='admin', protocol='', region_name='', scope='', service_provider_endpoint='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='Default', user_id='', username='admin', verbose_level=3, verify=None)
defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', 'cacert': None, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', u'interface': None, u'network_api_version': u'2', u'image_format': u'qcow2', u'image_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'1', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', u'compute_api_version': u'2', u'orchestration_api_version': u'1', u'database_api_version': u'1.0', u'interface': None, u'network_api_version': u'2', u'image_format': u'qcow2', u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': '1.6', 'auth': {'username': 'admin', 'project_name': 'admin', 'user_domain_name': 'Default', 'auth_url': 'https://localhost:5000/v2.0/', 'password': '***', 'project_domain_name': 'Default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'cacert': None, 'data_processing_api_version': '1.1', 'deferred_help': False, u'identity_api_version': '2', u'volume_api_version': u'1', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 2, cmd group open...

[Rafael Folco (rafaelfolco)]

Just hit this issue:

root@ubuntu:~# export OS_USER_DOMAIN_NAME=Default
root@ubuntu:~# export OS_PROJECT_DOMAIN_NAME=Default
root@ubuntu:~# source /opt/stack/new/devstack/openrc admin admin
root@ubuntu:~# nova flavor-list
ERROR (DiscoveryFailure): Cannot use v2 authentication with domain scope

However, it works fine if not setting DOMAIN vars.

[Haifeng.Yan (yanheven)]

thanks Tomi Juvonen‘s idea, it really works.

[Hieu LE (hieulq)]

Hi, I have tried to submit a patch set about this bug. Please take a look and give feedback: https://review.openstack.org/#/c/301014/3

[[e-mail] (mattlarkin)]

Is there any update or workaround on this bug?

[Hieu LE (hieulq)]

The patch-set was submitted in here: https://review.openstack.org/#/c/301014/ but currently received only one review.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/301014
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=59de9c477cb91573372bd363dfd0fc72af461327
Submitter: Jenkins
Branch: master

commit 59de9c477cb91573372bd363dfd0fc72af461327
Author: Hieu LE <email address hidden>
Date: Mon Apr 4 18:19:50 2016 +0700

    Ignore domain related config when using with keystone v2

    Currently, "/usr/bin/openstack --insecure token issue"
    fails when OS_AUTH_URL and OS_IDENTITY_API_VERSION
    indicate keystone v2 if OS_PROJECT_DOMAIN_NAME or
    OS_USER_DOMAIN_NAME are set. This patchset ignore domain
    related configs if using with keystone v2.

    Change-Id: If7eea2ed1a4877c60d055ed0114a5e5f31e282a0
    Closes-bug: #1447704

[Richard Theis (rtheis)]

Please see latest comment in https://review.openstack.org/#/c/301014/ at to why this bug is being re-opened.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/313979

[Hieu LE (hieulq)]

Thank you, I have summitted new patch for fixing the issue, please take a look.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/313979
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=b8432408e68855d7ce9269b7e681de3c5b733b09
Submitter: Jenkins
Branch: master

commit b8432408e68855d7ce9269b7e681de3c5b733b09
Author: Hieu LE <email address hidden>
Date: Mon May 9 12:29:44 2016 +0700

    Ignore domain related config when using with keystone v2

    Currently, "/usr/bin/openstack --insecure token issue"
    fails when OS_AUTH_URL and OS_IDENTITY_API_VERSION
    indicate keystone v2 if OS_PROJECT_DOMAIN_NAME or
    OS_USER_DOMAIN_NAME are set. This patchset ignore domain
    related configs if using with keystone v2 and print warning
    for each ignored config.

    Change-Id: I8afbda787df7855c3f8e868b0f07cbf3b9cd97fd
    Closes-bug: #1447704

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-openstackclient 2.5.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dean Troyer (<email address hidden>) on branch: master
Review: https://review.openstack.org/199661