Admin users cannot authenticate properly via admin_domain

Bug #1649106 reported by Greg Mason
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Keystone Charm
In Progress
Medium
Liam Young
keystone (Juju Charms Collection)
Invalid
Medium
Liam Young

Bug Description

When deployed as keystone v3, auth via the domain admin_domain isn't fully functional. When trying to run "openstack domain list" or "openstack user list" keystone responds with a 403:

$ openstack user list
You are not authorized to perform the requested action: identity:list_users (HTTP 403) (Request-ID: req-xxxxxxx)

Any admin users in the domain admin_domain should be able to list domains, users, and should be able to create users.

Performing similar operations via Horizon succeed, however this is a poor workaround.

Greg Mason (gmason)
summary: - admin user cannot authenticate properly via admin_domain
+ Admin users cannot authenticate properly via admin_domain
Revision history for this message
Liam Young (gnuoy) wrote :

Are you creating a new admin user and trying to do a domain list? Or are you getting a domain scoped token from the 'admin' user in the admin domain as per https://wiki.ubuntu.com/OpenStack/OpenStackCharms/ReleaseNotes1604#Keystone_v3_API_support ?

Changed in keystone (Juju Charms Collection):
status: New → Incomplete
Revision history for this message
Liam Young (gnuoy) wrote :

I'm going to grab this bug and update the README off the back of it because this is not the first time people have had issues with authenticating against a v3 cloud.

Changed in keystone (Juju Charms Collection):
assignee: nobody → Liam Young (gnuoy)
Revision history for this message
Greg Mason (gmason) wrote :

I've tried to authenticate against the API with the following environment variables populated:

OS_REGION_NAME=redacted
OS_USER_DOMAIN_NAME=admin_domain
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=redacted
OS_DOMAIN_NAME=admin_domain
OS_AUTH_URL=http://redacted:35357/v3
OS_USERNAME=admin

I still get a 403 when I try to list users or domains via the openstack cli client.

Joey Stanford (joey)
Changed in keystone (Juju Charms Collection):
status: Incomplete → New
tags: added: bootstack-is
tags: added: canonical-bootstack
removed: bootstack-is
Revision history for this message
Greg Mason (gmason) wrote :

Here is a copy of policy.json from the environment. To me, this looks like it should work for listing users or domains, unless admin_and_matching_domain_id on line 9 is incorrect.

The templates in the charm that are used for policy.json have been edited by hand on the units to add the correct domain id for the cloud_admin rule on line 3. This is the workaround for bug #1637453, which I see is now fixed.

-Greg

Revision history for this message
Greg Mason (gmason) wrote :

This auth is failing for use with the admin_domain scope. The auth for these operations is failing for the user "admin" and any other users with the role "Admin" in the domain "admin_domain".

Revision history for this message
Liam Young (gnuoy) wrote :

Greg, when you're setting up the environment variables are you ensuring that no other OS_* variables are set ? fwiw to ensure no other OS_* env vars pollute the environment my novarc looks like this:

_OS_PARAMS=$(env | awk 'BEGIN {FS="="} /^OS_/ {print $1;}' | paste -sd ' ')
for param in $_OS_PARAMS; do
    unset $param
done
unset _OS_PARAMS
export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -f keystone`:5000/v3
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_REGION_NAME=RegionOne
export OS_DOMAIN_NAME=admin_domain
export OS_USER_DOMAIN_NAME=admin_domain
export OS_IDENTITY_API_VERSION=3
# Swift needs this:
export OS_AUTH_VERSION=3

Revision history for this message
Greg Mason (gmason) wrote :

Yes, the novarc I'm using has that function.

When I've tested different combinations of environment variables, I've been sure to use unset to clean up the environment, and checking the output of env to be sure of what the environment has set. The variables I provided earlier were in the output of "env | grep OS_" rather than a novarc.

Liam Young (gnuoy)
Changed in keystone (Juju Charms Collection):
status: New → Confirmed
James Page (james-page)
Changed in keystone (Juju Charms Collection):
importance: Undecided → Medium
status: Confirmed → In Progress
tags: added: v3
Revision history for this message
Liam Young (gnuoy) wrote :

The issue was the roles granted to the admin user by the charm. The 'admin' user should be granted the 'Admin' role against the 'admin_domain' but for some reason the 'admin' user only had the 'Member' role against the 'admin_domain'.

Querying the keystone database directly shows how the grants should look:

select a.type, c.name, d.name
from assignment a, local_user b, role c, project d
where a.actor_id = b.user_id
and b.name = 'admin'
and c.id = a.role_id
and d.id = a.target_id;

+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Member | admin |
| UserDomain | Admin | admin_domain |
+-------------+--------+--------------+

However in the case of the deployment gmason was having issues I think the query would have shown:

+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Member | admin |
| UserDomain | Member | admin_domain |
+-------------+--------+--------------+

Revision history for this message
Liam Young (gnuoy) wrote :

@gmason Could you confirm what the keystone charm setting admin-role is set to in your deployment?

Revision history for this message
Liam Young (gnuoy) wrote :

@gmason the logs from /var/log/juju/* on the keystone units would also be useful.

I have been unable to reproduce this bug so far but will keep trying.

Revision history for this message
Greg Mason (gmason) wrote :

the juju keystone admin-role is configured to "Admin"

Here's the output of the MySQL query:
+-------------+--------+--------------+
| type | name | name |
+-------------+--------+--------------+
| UserProject | Admin | admin |
| UserProject | Admin | admin_domain |
| UserDomain | Admin | admin_domain |
| UserDomain | Member | admin_domain |
+-------------+--------+--------------+

Revision history for this message
Greg Mason (gmason) wrote :

Here are sanitized versions of the requested logs from the environment.

James Page (james-page)
Changed in charm-keystone:
assignee: nobody → Liam Young (gnuoy)
importance: Undecided → Medium
status: New → In Progress
Changed in keystone (Juju Charms Collection):
status: In Progress → Invalid
Revision history for this message
Linda Guo (lihuiguo) wrote :

Hi

Is there a plan to fix this bug? we are hitting this bug that user with Admin role is unable to list role assignment but operations via Horizon succeeds

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Hi Linda

Can you verify whether that user can perform the action using the CLI rather than horizon. It would help to verify whether this is a policy issue (in horizon), or a policy/misconfiguration issue in keystone.

Also: which version of OpenStack/Ubuntu and charms is this on please?
Has this OpenStack been upgraded?

Thanks.

Revision history for this message
Linda Guo (lihuiguo) wrote (last edit ):

Hi Alex.

The user with admin role can perform action (list role) using Horizon, but CLI failed. I have reviewed the keystone policy file, it looks good.

"admin_required": "role:Admin",

"identity:get_role": "rule:admin_required",
    "identity:list_roles": "rule:admin_required",
    "identity:create_role": "rule:cloud_admin",
    "identity:update_role": "rule:cloud_admin",
    "identity:delete_role": "rule:cloud_admin",

openstack/ubuntu version:
Ubuntu 18.04 (Bionic)
OpenStack Queens

keystone charm is at version 312

Openstack hasn't been upgraded

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Hi Linda

Then it's almost certainly a mis-configuration of the admin's roles as per comment #8 in this thread. It would be useful to explore the database and see what the admin user and roles look like w.r.t. to the outputs shown in #8.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.