11.2.1 : openstack client with V3 auth causes usability issues

[Bjoern]

In light of https://bugs.launchpad.net/cinder/+bug/1441922 where it states

Jin Liu (jin-t) wrote on 2015-05-07: #6

Some updates, we made keystone v3 working fine either using cli or horizon.
We figured out what parameters required in openstackclient command line. Using example in this ticket,
# openstack volume list
# openstack server list
Actually these two commands need OS_PROJECT_NAME env or --os-project-name in command line. Export OS_DOMAIN_NAME will not help. Basically OS_PROJECT_NAME is needed for most services(nova/cinder/neutron etc) except Identity. Identity requires OS_DOMAIN_NAME.
You may think export OS_PROJECT_NAME and OS_DOMAIN_NAME together. But openstackclient will complain when you do both,
"ERROR: openstack Authentication cannot be scoped to multiple targets. Pick one of: project, domain or trust".

You can find extra info about policy.json from this article https://www.mirantis.com/blog/manage-openstack-projects-using-domains-havana/

The unpleasant issue is that you have to use --domain inside the command line to do a user list like
openstack user list --domain default

unfortunately you can't just set OS_DOMAIN_NAME or ID because that will trigger the error above "ERROR: openstack Authentication cannot be scoped to multiple targets. Pick one of: project, domain or trust"

This issue can be highly misleading when not using the --domain parameter, the openstack client just comes back with

ERROR: openstack The request you have made requires authentication. (Disable debug mode to suppress these details.)

which would people to believe there is some configuration error going on.
It seems we can alter this behavior with a policy.json change and I'm asking to investigate this

[Bjoern (bjoern-t)]

This issue brings even the os-keystone-install playbook to a halt, due to the missing domain name:

TASK: [os_keystone | Ensure Admin user] ***************************************

Result from run 1 is: {'msg': 'OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: mux_client_request_session: master session id: 2\r\nTraceback (most recent call last):\n File "<stdin>", line 2895, in <module>\n File "<stdin>", line 1284, in main\n File "<stdin>", line 460, in command_router\n File "<stdin>", line 720, in ensure_user\n File "<stdin>", line 660, in _get_user\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner\n return func(*args, **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/v3/users.py", line 108, in list\n **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py", line 73, in func\n return f(*args, **new_kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py", line 361, in list\n self.collection_key)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/base.py", line 113, in _list\n resp, body = self.client.get(url, **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 170, in get\n return self.request(url, \'GET\', **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 206, in request\n resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 95, in request\n return self.session.request(url, method, **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner\n return func(*args, **kwargs)\n File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 397, in request\n raise exceptions.from_response(resp, method, url)\nkeystoneclient.openstack.common.apiclient.exceptions.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-b1e36888-0d67-4614-b4ee-c46d71fc9f0e)\n', 'failed': True, 'attempts': 1, 'parsed': False}

[Bjoern (bjoern-t)]

 git describe --tags
11.2.1-3-gfaeb18a

I'm also using LDAP as identity driver

[Jesse Pretorius (jesse-pretorius)]

I'm not sure I understand the problem here. The openstack client is required to work with the Keystone v3 API. The openstack client is an upstream product consumed by openstack-ansible. If the openstack client is confusing to use, then that is a problem which needs to be solved by either understanding how to use it, or changing it upstream. This is not a problem which can be solved by the openstack-ansible project directly.

If a deployer prefers to use the v2 API then the ability to set the openrc to do so already exists. However it is advisable for deployers to get used to how to work with the Keystone v3 API as the v2 API will eventually be removed.

[Bjoern (bjoern-t)]

I don't it is that easy since playbooks fail now and even few of our MAAS checks

[Bjoern (bjoern-t)]

Additionally I don't it is that easy because we would have to install a v2 policy, change endpoints etc etc

[Jesse Pretorius (jesse-pretorius)]

Bjoern. You still have not described a problem, other to say that the openstack cli is horrible. Can you please describe the actual problem?

[David Wilde (dave-wilde)]

Bjoern, I think this is an issue with the MaaS openrc that we're droping down. I think we need to add OS_PROJECT_DOMAIN_NAME to the MaaS openrc. Let's handle this via a GitHub issue as I don't see this is a problem with OSA.

[Bjoern (bjoern-t)]

Folks, please see my description on #1. The keystone playbooks fail with (which is up there), it is not just the openstack client usability issue

keystoneclient/session.py", line 397, in request\n raise exceptions.from_response(resp, method, url)\nkeystoneclient.openstack.common.apiclient.exceptions.Unauthorized: The request you have made requires authentication

[Bjoern (bjoern-t)]

Hi Dave, I did try that, neither domain settings work. Not OS_PROJECT_NAME, OS_DOMAIN_NAME or OS_USER_DOMAIN_NAME.

[Ian Cordasco (icordasc)]

I'll take a look at this tomorrow

[Steve Martinelli (stevemar)]

So I think most of the folks on this thread understand how to authenticate - right?

You have:
 - OS_USERNAME (and OS_USER_DOMAIN_NAME if it's in the non-default domain)
 - OS_PROJECT_NAME (and OS_PROJECT_DOMAIN_NAME if it's in the non-defualt domain)
 - OS_DOMAIN_NAME only exists if you want to scope to an entire domain, its an XOR relationship with OS_PROJECT_NAME

I think the issue here that Bjoern is trying to raise is that some of the `openstack` commands are inconsistent with regards to the --domain argument.

In some cases we use --domain, and some cases we use --user-domain and --project-domain.

These really come down to the specific command, for `openstack user list` we can only filter by user domains so we just have it as --domain. Same with `openstack group list --domain x` and `openstack project list --domain x`.

In cases where we need to be specific, like adding a role, we qualify it, for instance:

  `openstack role add member --user u --user-domain x --project p --project-domain y`

Bjoern, are you suggesting that?

[Bjoern (bjoern-t)]

Hi Steve,

thanks for your input.
What I'm asking are multiple things :

- Adding this information to documentation, since this will confuse a lot of people

- Checking why the playbooks fail

- Investigating a possible policy change as someone claimed at https://bugs.launchpad.net/cinder/+bug/1441922
  Personally I'm not sure if a policy change can fix this since this behavior is triggered when using LDAP and enabling multi domain
 support while using the Default domain.

[Bjoern (bjoern-t)]

The playbook issue has been moved to #1518351

[James Thorne (james-thorne)]

For anyone still confused with the issue at hand, here is a simple synopsis:

When LDAP is enabled as a backend for Keystone Identity (so, /etc/keystone/domains/keystone.Default.conf exists), the end user has to run "openstack user list --domain default" instead of "openstack user list". If the end user runs "openstack user list" they will get a 401 error.

Because of this, the end user could mistakenly think there is a problem with their environment. The other OpenStack project commands (nova, glance, neutron, etc.) still run as expected.

[Jesse Pretorius (jesse-pretorius)]

The action for openstack-ansible has been registered in https://bugs.launchpad.net/openstack-ansible/+bug/1518351 - so marking this as invalid for openstack-ansible.

[Bjoern (bjoern-t)]

I have submitted a upstream fix to address this issue via https://review.openstack.org/#/c/251597/.
Let's wait and see how the upstream community thinks about this patch

[Kevin Carter (kevin-carter)]

Usability issues aside, this is not a problem for openstack-ansible and the specific deficiencies we currently have are being addressed here https://review.openstack.org/#/c/248900/ and here https://review.openstack.org/#/c/238509/

This issue was marked invalid for OSA.

[Dean Troyer (dtroyer)]

I am unsure what the bug in OSC is here. The fact that the auth options change when using Identity v3 is a result of the Identity v3 API's requirements.

[Dolph Mathews (dolph)]

I raised a more direct example of this issue regarding python-openstackclient in https://bugs.launchpad.net/python-openstackclient/+bug/1592062