Keystone policy.json not matching domain_id

[Eric Miller]

Environment: Queens installed using Kolla Ansible 6.1.0 on CentOS 7.5

I created a rule in the Keystone policy.json that should match a custom role (domain_admin) and match the domain_id. I tried 4 variations, only the last variation worked, which has the domain_id hard-coded:

# "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:%(target.token.user.domain.id)s",

# "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:%(target.token.user.domain_id)s",

# "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:%(domain_id)s",

    "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:e93d848b2a274cb588676e029ae53348",

The goal was to use this rule for the project creation permission like this:
"identity:create_project": "rule:cloud_admin or rule:domain_admin_and_matching_domain_id",

However, I always got an error when creating a project with a test user who belongs to the domain_admin role and the respective domain (e93d848b2a274cb588676e029ae53348):
Forbidden: You are not authorized to perform the requested action: identity:create_project. (HTTP 403)

until I hard-coded the domain_id in the policy.json file, which led me to believe that the syntax for the variable-driven "domain_admin_and_matching_domain_id" rules is incorrect or something else is wrong.

The user has the appropriate role assignment (note that this is a test system, not production, so names and UUIDs can be publicly listed in this ticket):

openstack role assignment list --domain e93d848b2a274cb588676e029ae53348
+----------------------------------+----------------------------------+-------+---------+----------------------------------+-----------+
| Role | User | Group | Project | Domain | Inherited |
+----------------------------------+----------------------------------+-------+---------+----------------------------------+-----------+
| 13cf2d56ff594a56a9897787ab07cff5 | ad6038fe42564ba2b8278ceae52f2964 | | | e93d848b2a274cb588676e029ae53348 | False |
+----------------------------------+----------------------------------+-------+---------+----------------------------------+-----------+

The respective UUIDs are listed here (filtered by hand to only include this role):

openstack role list
+----------------------------------+-------------------------------+
| ID | Name |
+----------------------------------+-------------------------------+
| 13cf2d56ff594a56a9897787ab07cff5 | domain_admin |
+----------------------------------+-------------------------------+

openstack user list
+----------------------------------+-------------------+
| ID | Name |
+----------------------------------+-------------------+
| ad6038fe42564ba2b8278ceae52f2964 | TestDomainAdmin |
+----------------------------------+-------------------+

openstack domain list
+----------------------------------+------------------+---------+--------------------+
| ID | Name | Enabled | Description |
+----------------------------------+------------------+---------+--------------------+
| e93d848b2a274cb588676e029ae53348 | TestDomain | True | |
+----------------------------------+------------------+---------+--------------------+

Am I missing something obvious in the policy.json file?

Thanks!

Eric

[Eric Miller (erickmiller)]

I should have mentioned the command that I'm running to attempt to create a project using this account:
openstack --debug project create --domain e93d848b2a274cb588676e029ae53348 TestProject

which results in:
Forbidden: You are not authorized to perform the requested action: identity:create_project. (HTTP 403)

Changing the policy.json file so it uses this hard-coded domain permission:
    "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:e93d848b2a274cb588676e029ae53348",

results in the proper execution:

+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | |
| domain_id | e93d848b2a274cb588676e029ae53348 |
| enabled | True |
| id | 1b0cf36732fd4dee8a204721a39ab198 |
| is_domain | False |
| name | TestProject |
| parent_id | e93d848b2a274cb588676e029ae53348 |
| tags | [] |
+-------------+----------------------------------+

[Eric Miller (erickmiller)]

This is the set of export variables used for the user, in case it matters:

declare -x OS_AUTH_PLUGIN="password"
declare -x OS_AUTH_URL="http://<redacted>:5000/v3"
declare -x OS_DEFAULT_DOMAIN="e93d848b2a274cb588676e029ae53348"
declare -x OS_DOMAIN_ID="e93d848b2a274cb588676e029ae53348"
declare -x OS_IDENTITY_API_VERSION="3"
declare -x OS_INTERFACE="public"
declare -x OS_PASSWORD="test"
declare -x OS_REGION_NAME="us-central-1"
declare -x OS_USERNAME="TestDomainAdmin"
declare -x OS_USER_DOMAIN_ID="e93d848b2a274cb588676e029ae53348"

[wangxiyuan (wangxiyuan)]

So you want the users can only create projects in his domain, right?

I guess the example in keystone repo works for you: https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json#L51

[Eric Miller (erickmiller)]

That is the ultimate goal, and I am using the policy.v3cloudsample.json file, but it is the file that comes with Queens. It looks like there have been some updates in Master. I didn't want to use the Master version since I wasn't sure if it was backward-compatible with Queens.

I would still like to understand why the policy I created isn't working.

The issue I had, which led to the creation of a new role and new policies was the fact that the "admin" role gave permission for a domain admin to list all projects in the entire cloud, regardless of whether this user's role assignment being scoped to a domain. Maybe this is fixed in Rocky? I have a test system for Rocky, but haven't done enough testing with domain-specific policies.

[Eric Miller (erickmiller)]

I just changed the rule in my policy.json file to:
    "domain_admin_and_matching_domain_id": "role:domain_admin and domain_id:%(project.domain_id)s",

using %(project.domain_id)s instead of %(domain_id)s and it appears to be working ok!

I will do a bit more testing and report back.

[Eric Miller (erickmiller)]

I copied the Master copy of policy.v3cloudsample.json to my Queens test environment's Keystone, and I can create projects in the domain using the domain admin account and can also "not" see other projects in the cloud, so that's great! :)

Thanks for the help! I'll do more testing and report back if I see any issues.

[Eric Miller (erickmiller)]

Quick related question... is the policy engine the same for all projects? I don't believe Neutron or many other projects have similar policy.v3cloudpolicy.json files, so we may have to adjust them on our own. For example, Neutron allows domain admins to see all networks for the entire cloud (not good). If the oslo policy engine is used for all projects' policy.json files, then I'm assuming that the domain_id matching should work with other projects in a similar way that policy.v3cloudpolicy.json is used for Keystone.

[wangxiyuan (wangxiyuan)]

glad to see it works. I'll mark this bug as Incomplete. Feel free to change it then.

For the second question:
The policy engine the same for all projects since they all use oslo.policy.

But the resources in openstack(like vm, network etc.) are mostly project isolation. It means that most APIs for the resource work well with project-scoped token. If the request is domain-scoped, I think most APIs will raise error.

I assume the "domain admins" you mentioned here means a domain-scoped token? If so, most services do't support it for request. For the case "Neutron allows domain admins to see all networks for the entire cloud", I guess the request contains the "admin" role which is treated as "cloud admin" in Neutron by default.

[Eric Miller (erickmiller)]

Thank you for the info! Much appreciated.

Yes, "domain admin" would be a user who is logged-in using a domain-scoped token and assigned to a single role with a domain specified.

I think I need to go back to using a user account for a that is not assigned the "admin" role, so I can be sure that Neutron, and other projects that consider users who belong to the admin role, do not assume a domain admin is a cloud admin.

I will test this and report back.

[Eric Miller (erickmiller)]

While looking for a way to allow a domain admin to assign the _member_ role to a respective domain user, I found these two policy rules:

    "identity:get_domain_role": "rule:cloud_admin or rule:get_domain_roles",
    "identity:list_domain_roles": "rule:cloud_admin or rule:list_domain_roles",

Both rules come before the rule definitions for "get_domain_roles" and "list_domain_roles". Does this order matter?

[Eric Miller (erickmiller)]

Btw, I think that the policy.v3cloudpolicy.json file has come a long way for Keystone, but it honestly doesn't matter what restrictions have been placed on Keystone if a domain admin must be assigned the "admin" role, which provides that user cloud admin privileges on other projects.

I know this has been discussed many times since 2012, but it seems to be a never ending issue. Am I correct that there is really no way to create a domain admin account that only has access to a domain's resources, including its networks (for example)?

If not, then project-level permissions are all that we can count on, which is pretty limiting, especially for service providers who need to allow customers access to create their own projects, users within those projects, etc.

I'm just trying to narrow down what is, and what is not, possible so I don't spin my wheels trying to get something to work that simply isn't possible.

Thanks!

Eric

[wangxiyuan (wangxiyuan)]

According to my experience, I'd say yes, it's hard to list the resources within a domain in one request. Let's assume a case that a user want to list his VM:

The db data maintained in Nova is like:
+-----+----------+
|vm_id|project_id|
+-----|----------+
|xxx | yyy |
+-----+----------+
So:
1. if the request is project-scoped, then Nova will query the vm table by "project_id".
2. if the request is domain-scoped, Nova doesn't know the relationship between vm and domain_id, so it's hard for Nova to query all the vm in this domain with the list_vm API.
3. But "list_domain_vm" can be done with combination APIs, like, get all the projects in this domain first, then get the vms project by project, then combine them. This can be handled by portal or web-UI
4. I'm not sure how other openstack clouds do, but in our Huawei Public Cloud, each account is a domain in Keystone, so the account can create its own projects, users, etc. but when the account login, he can't see all his vms in a page (of cause we can support it by step 3, but it's not highly required), he must go into each project in his domain to see what resource in it.

So for your question: "Am I correct that there is really no way to create a domain admin account that only has access to a domain's resources, including its networks "

Openstack doesn't has the concept for domain-level resource in most services, except Keystone. But you can create projects in a domain , then when accessing a resources, use the project-scoped instead.

[Eric Miller (erickmiller)]

Thank you again for your response! Listing resources, such as VMs, isn't really a problem, and having a domain admin traverse projects for VMs isn't a problem.

For your environment, it sounds like you have a domain admin account in a Keystone domain, but can you please let me know if this account is assigned to the "admin" role"? I would guess "no"?

The reason I ask is that it appears that a domain admin user can't assign the _member_ role to a user for a new project if the domain admin is not assigned to the "admin" role.

For example, if I want a domain admin to do 3 basic things, and nothing else:
1) create a project within its domain
2) create a user within its domain
3) assign the user to the _member_ role scoped to the project

I'm assuming that a user who is not assigned to the admin role is required, but this prevents the "user create" and "role add" commands from working. I was able to change the Keystone policy.json file enough so #1 was possible, so I'm trying to make other changes to the policy.json file to get the other 2 commands to run without permission issues.

I'm assuming, in the Huawei Public Cloud, the policy.json files are custom?

Thanks!

Eric

[Eric Miller (erickmiller)]

I figured out how to do everything I needed, but I'm still testing to be sure I didn't open any holes. So far, so good.

Essentially, I created a new role called "domain_admin", adjusted various policies to lock the APIs down to the domain_admin role while also being sure the user provides their domain UUID in every command like this:
"role:domain_admin and domain_id:%(domain_id)s"

The identity:create_grant and identity:revoke_grant must be locked down to only allow domain_admins the ability to assign the _member_ role to a user:
(role:domain_admin and domain_id:%(target.project.domain_id)s and None:%(target.role.domain_id)s and '9fe2ff9ee4384b1894a90878d3e92bab':%(target.role.id)s)

The end of this rule includes a hard-coded UUID of the _member_ role.

There are a few other things that I did, but need to clean up my work before I share.

Eric

[Lance Bragstad (lbragstad)]

Hi Eric,

You're highlighting a major initiative that we're currently working to address.

For clarity, it is rare for services have more than one set of policy files. We recently started removing policies by converting the defaults to be in code, which was a cross-project initiative in Queens [0]. Keystone attempted to fix some of the issues with admin-ness that you've described by introducing a supplemental policy file that people could adopt if they were offer more granular permissions for users at the system, domain, and project levels. It became apparent that relying on policy files to fix this was problematic for deployers and wasn't thoroughly tested upstream or protected by the gate (e.g. modifications require operators to manually check that holes weren't explicitly opened on accident, just like you described).

We're working on addressing these issues in keystone [1]. As we fix more of these issues [2], we'll be removing policies from the policy.v3cloudsample.json policy file template, as it should be the behavior we're enforcing with tests in code.

Ideally, we plan to implement system-scope, domain-scope, and project-scope effectively within keystone so that we can start using it as a template for other services to do the same. This should result in OpenStack being more self-service for end-users while providing explicit tests for RBAC behavior and hopefully making maintenance easier for deployers.

[0] https://governance.openstack.org/tc/goals/queens/policy-in-code.html
[1] https://bugs.launchpad.net/keystone/+bug/1750660
[2] https://bugs.launchpad.net/keystone/+bugs?field.tag=policy

[Adam Young (ayoung)]

Just to be clear, this has always been the case. THe documentation for the cloud sample stated it needed to be edited.

Of course, I tripped over this exact problem. A few times. I once proposed reading policy values from the config file as a work around.

But this is not a bug. As Lance put, work is underway to make sure we don't need to do this in the future, but the cloudsample is just that, as sample policy file, and it needs to be edited to be correct.