RFE: Token returns Project's tag properties

Bug #1807697 reported by Yang Youseok on 2018-12-10
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Wishlist
Yang Youseok

Bug Description

From an operator perspective, there are many situations where you need to add an ACL for each project. Currently, keystore and openstack policies do not seem to have any fine-grained APIs for project-specific privilege control.

For specific, if we want to restrict some network resources per projects we have to assign neutron's rbac_policy which enable to map specific project with network sources rather than using oslo.policy.

I found that if we can handle project's extra properties in policy code, developer can check the custom properties for their own ACL logic which can be added by oslo.policy. There is already enough required code in keystone codebase for returning token with project extra property, IMHO it can be added without major changes.

Thanks in advance.

Tags: rfe Edit Tag help
Yang Youseok (ileixe) on 2018-12-10
tags: added: rfe
Morgan Fainberg (mdrnstm) wrote :

It is highly recommended that "extras" not be used. If we could do away with them, we would (with exception of the API-Contract that requires us to maintain them). These attributes are poorly implemented and it is not easy to unwind them. As is the current behavior it is not possible to unset them (behavior that must be maintained). Another concern is any/all extras could be eaten / lost if a similar column is added to the project model. It is impossible to know what operators/deployers have named the "extra" values. There are also concerns about typos leading to useless data / un-removable data without direct DB modification.

Would a vendor-specific location e.g. project['vendor_attributes'] (or similar) where similar data could be placed (and a None would unset) be usable? We could discuss adding such values into the token response.

As it stands, I am against passing the "Extra" values into the token response. Use of extras should be discouraged as implemented today.

Yang Youseok (ileixe) wrote :

@Morgan Hi Morgan.

First, thank you for the comment, I was very appreciated since I never heard of implementation detail from the developer.

Actually I tried to find right purpose of 'extra' flag in project but it exists at the very first so I did not notice the background you said (At least, it looks weird since it just json blob though). It make sense not to use of it.

Before investigating your suggestion more (new property in Project I understand), I realize there are the existed features can likely be used for my purpose. Since our deployment is very behind (we are using Newton) and I have no change to deep dive the codebase, I did not know there is such a function. I found two viable solutions.

1) https://wiki.openstack.org/wiki/HierarchicalMultitenancy
I found keystone Domain was changed to Project, and Project can have child Project recursively. From my current understanding, If I create parent Project for containing several projects, and make new policy of the parent project, I can achieve ACL per group of project. If it goes on, maybe the RFE should be changed that "Token returns parent project ID"

2) https://blueprints.launchpad.net/keystone/+spec/project-tags
We are using Tag API in Neutron to assign specific deployment property (e.g. Development / Production), and I realize it's general API which is available on cross-projects. If the Tag can be leaked from the Token, imo it can be also useful to make policy rule in oslo.policy.

I wonder how others think of it. Thanks!

Morgan Fainberg (mdrnstm) wrote :

AH! Project Tags is something more in line with something we could expose directly and I totally could see the benefits there. It is also possible for neutron to directly request the project details from Keystone via the /v3/projects api, which should return the tags (which might be the easiest and shortest path, changing the token body/format is def. a high barrier as the token format is consumed externally / is a contract. If the direct request from neutron for the project info is sufficient we can roll with that, if not then we can explore the tags being in the token.

I'm marking this bug as incomplete for now, once you respond please feel free to move it back to "new" if we need to explore tags in the token (may be a ways out to land, just forwarning...it also may not land at all).

Changed in keystone:
status: New → Incomplete
Yang Youseok (ileixe) wrote :

@Morgan. Thanks for the further explanation.

Actually, our code base is working just exactly as you said. We made paste middleware which try to get required attributes (currently project 'extra' value though but I will change to 'tags') from keystone.

I felt this solution is merely workaround and it's the reason why I made this RFE.
That's because

1. Every single API call have to make one more round trip to get the attribute. It makes more latency to use API.
2. Service(Nova, Neutron ...) have to implement keystone client API. On the Nova side, it's implemented already, but many other projects are not done and even the purpose of the keystone client is to validate the quota only (Not for general call like our purpose)

However at the same time I could not agree with you more for the risk to change external interface, so we need to consider whether this is the right direction.

IMHO, token expose somewhat scope related attributes and tag could be a scheme for grouping resources, so I think it can be building block for more abstracted APIs. (Multicloud for example)

I made a minimal required change and tested, at least it works well for policy. Although there are many related sub project to be changed (oslo.middleware, oslo.context and so on), if community has some consensus of the necessity I will be happy to get started.

Thanks

Yang Youseok (ileixe) on 2018-12-13
Changed in keystone:
status: Incomplete → New
summary: - [RFE] Token returns Project's extra properties
+ [RFE] Token returns Project's tag properties
Changed in keystone:
status: New → Triaged
importance: Undecided → Wishlist
summary: - [RFE] Token returns Project's tag properties
+ RFE: Token returns Project's tag properties

Fix proposed to branch: master
Review: https://review.opendev.org/654301

Changed in keystone:
assignee: nobody → Yang Youseok (ileixe)
status: Triaged → In Progress
Miguel Lavalle (minsel) wrote :

We got a parallel submission in Neutron on this topic: https://bugs.launchpad.net/neutron/+bug/1825336. Since this seems to need changes in several projects, let's stay in synch

Gage Hugo (gagehugo) wrote :

Project Tags their own path[0] to call that can be backed by separate policy from projects if needed. Would using this alongside the default reader role help with this issue?

[0] https://developer.openstack.org/api-ref/identity/v3/#project-tags

Putting unbound data in the token is something we're trying to avoid.

Yang Youseok (ileixe) wrote :

@Gage Hugo
imho, there would be barrier to use Tag feature if we have to request Keystone one more time in the Nova or Neutron side. We actually did like you said, but I felt It's better to keep tag in the auth token to avoid to request to Keystone.

I did not consider 'unbound' data, so I wonder what meaning of unbound?

Gage Hugo (gagehugo) wrote :

Is there a reason for trying to cut down on API request calls, other than saving time/bandwidth? I'm not following how an additional GET /projects/[id]/tags call would be a significant barrier?

If the Tags are being kept as part of the token, what is the lifetime of this token. How would we handle any tags getting changed during the token's valid lifetime, and any potential discrepancies that would result from this?

Yang Youseok (ileixe) wrote :

This is because services have to add keystone client code besides keystonemiddleware to access keystone. Some of service like Nova already have keystone client code though, it's currently only for limited use like validation of projects, and others like Neutron does not have at all.

imho, general API dependency between services should be minimized so I'm skeptical of adding client code in this respect.

For a data inconsistency, thanks for letting me know because I did not consider it deeply because I assume that currently there is no way to deal with the problem. I assume that this kind of inconsistency should be treated at client side, and actually I added keystonemiddleware to invalidate the auth token when consistency broke like this (https://bugs.launchpad.net/keystonemiddleware/+bug/1813739)

Is there any general way to keep consistency now? For example when project name is changed, will auth token having old name be invalidated?

Colleen Murphy (krinkle) wrote :

By "unbounded" we mean that projects can have effectively limitless tags, but if all of the tags are included in the token that expands the size of the token.

I don't see any issue with adding client code to services to make a request to keystone to fetch a project and its tags. There is no other way to guarantee the data is up-to-date, and adding this information to the token is not going to be acceptable.

Yang Youseok (ileixe) wrote :

@Colleen Oh, I assumed the word of 'unbounded' is related to the scope, but it does not at all. After thinking about more Gage's comments, it's not reasonable to add more mutable data because keystone have done it before.

And since the tag's mutable properties (like size) is a lot, now I fully understand adding more mutable data in the auth token is not acceptable. I will close the related reviews.

But last thing, adding client code to fetch project's tag is somewhat unclear for me since I'm not sure where do I have to add the logic. Current implementation from the similar requirements is on demand approach making request when they want to (e.g. validate quota project ID when they have to). I felt fetching tags seems to have a different character since service have to query keystone every time to use as a policy. Could you mind share what you think about it?

Anyway, thank you so much for your reviews of this proposal without any direct need.

Change abandoned by Yang Youseok (<email address hidden>) on branch: master
Review: https://review.opendev.org/654301

Colleen Murphy (krinkle) wrote :

I think using the /v3/projects/<id> API to request the project information directly is the best way forward for now, even for policy purposes. In client code, it would end up looking something like this: https://opendev.org/openstack/cinder/src/branch/master/cinder/quota_utils.py#L102-L108

You could also consider taking a different approach: perhaps you could create different roles which different privileges defined in policy, and use role assignments on specific projects (or a tree of projects) to designate the restrictions per-project. I think this would make more sense than trying to change neutron to use different policy based on project tags.

Yang Youseok (ileixe) wrote :

@Collen Murphy I don't want to bother you anymore though, you mentioned about hierarchy project, so I'm sorry but let's talk about little bit more about that. :)

As I said before Morgan Fainberg, I was trying to add two properties to auth token. One is tag, and the other is project parent id of project.

As tag's mutable property prevent the addition, how about parent project ID? I felt parent project id is not a different than project id itself, so could auth token contain the parent Id of the project?

imho, it can accelerate the introduction of hierarchy project concept from services' side.

Colleen Murphy (krinkle) wrote :

I could see possibly adding the project parent ID to the token body, however again you could just query keystone to get the project's parent.

Thanks for the answer, then I'm going to take actions at service side.

2019년 5월 17일 (금) 오전 2:21, Colleen Murphy <email address hidden>님이 작성:

> I could see possibly adding the project parent ID to the token body,
> however again you could just query keystone to get the project's parent.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1807697
>
> Title:
> RFE: Token returns Project's tag properties
>
> Status in OpenStack Identity (keystone):
> In Progress
>
> Bug description:
>
> From an operator perspective, there are many situations where you need
> to add an ACL for each project. Currently, keystore and openstack policies
> do not seem to have any fine-grained APIs for project-specific privilege
> control.
>
> For specific, if we want to restrict some network resources per
> projects we have to assign neutron's rbac_policy which enable to map
> specific project with network sources rather than using oslo.policy.
>
> I found that if we can handle project's extra properties in policy
> code, developer can check the custom properties for their own ACL
> logic which can be added by oslo.policy. There is already enough
> required code in keystone codebase for returning token with project
> extra property, IMHO it can be added without major changes.
>
> Thanks in advance.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/keystone/+bug/1807697/+subscriptions
>

Colleen Murphy (krinkle) wrote :

Okay, in that case I'll close this bug for now. If you get stuck or have questions, feel free to reopen this or contact us in #openstack-keystone or on the openstack-discuss mailing list.

Changed in keystone:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers