v3/users reports all users in all domains excepts when domain_specific_drivers_enabled is set to true.

[Sofer Athlan-Guyot]

Hi,

Setting "domain_specific_drivers_enabled=true" in the keystone.conf,
prevents my calls to "/v3/users" to works when using the admin_token:

    @token="admin_token", @url="http://127.0.0.1:35357/v3

->

    /bin/openstack user list --quiet --format csv --long'
    127.0.0.1 - - [10/Mar/2016:08:15:41 -0500] "GET /v3/users HTTP/1.1" 401 114 "-" "python-keystoneclient"

If I add a domain option to the openstack user list command, I get the
users of the domain (not 401)

If I do a project list it works and returns the complete list of all
projects in all domains:

    Debug: Executing '/bin/openstack project list --quiet --format csv --long'
    127.0.0.1 - - [10/Mar/2016:08:22:00 -0500] "GET /v3/projects HTTP/1.1" 200 471 "-" "python-keystoneclient"
    => [{:id=>"1ff87dbb8e6e45d5b43a49a812fafb88", :name=>"admin", :domain_id=>"default", :description=>"Bootstrap project for initializing the cloud.", :enabled=>"True"},
     {:id=>"60f86c662af248449c1007fbf32ed5af", :name=>"openstackv3", :domain_id=>"463e1bb751374a0586a867a73cb35330", :description=>"admin tenant", :enabled=>"True"},
     {:id=>"746e5e3d02b04d079dfa639ac5d03886", :name=>"services", :domain_id=>"default", :description=>"Tenant for the openstack services", :enabled=>"True"},
     {:id=>"bcf81b0d73b74c85b01e1b15f38be64e", :name=>"openstack", :domain_id=>"default", :description=>"admin tenant", :enabled=>"True"},
     {:id=>"e00959d5ac2545a5a77d137d20e0f9f8", :name=>"servicesv3", :domain_id=>"a43714e50901474eb328daf380ef24ee", :description=>"Tenant for the openstack services", :enabled=>"True"}]

If I try the exact same command (without the domain option) with
"domain_specific_drivers_enabled=false" in keystone.conf, I get the
list of users in all domains.

This is rather confusing. The "401", unauthorized error is confusing.
The discrepancy between user and project behavior is confusing.

So what is the "correct" behavior ?

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

This is on centos7

rpm -qa | grep keystone
python-keystonemiddleware-4.3.1-0.20160226051710.455d40c.fc23.noarch
python-keystoneclient-2.3.2-0.20160301011539.be9b585.fc23.noarch
python-keystone-9.0.0.0b4-0.20160303114032.e2ee064.el7.centos.noarch
openstack-keystone-9.0.0.0b4-0.20160303114032.e2ee064.el7.centos.noarch
python-keystoneauth1-2.3.1-0.20160302005848.943fde7.el7.centos.noarch

[Steve Martinelli (stevemar)]

Are all the users stored in a single SQL backend? They just exist in different domains?

Setting the domain_specific_drivers_enabled flag shouldn't change the behaviour of the API call. It should return all users, regardless of the domain.

The only issue I can think of is when a particular domain is backed by LDAP, then calling /v3/users will eventually timeout and return an error.

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

The users are all in the sql backend. The flag has been set to true after having successfully populated the the sql db backend with users, projects, role, domains. Then without any other modification, the list of users has been asked again from the API, giving the result I showed you. So no ldap server configured, no other backend configured at all actually, which is certainly a corner case of the usage of this flag.

[Thomas Hsiao (thomas-hsiao)]

It seems to me that this is by design in configuration.rst
"
Note
It is important to notice that by enabling either of the domain-specific configuration methods, the operations of listing all users and listing all groups are not supported, those calls will need either a domain filter to be specified or usage of a domain scoped token.
"
https://github.com/openstack/keystone/blob/master/doc/source/configuration.rst

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

Thanks for the pointer.

I understand (and half-expected) that the list of all users would be limited when using domain-specific identity backend would be somehow limited.

There is a discrepancy though between what the doc says and what I've seen, namely the list of all groups (projects) is still available, while it should have the same behaviour that user according to the doc. This may be a bug ?

As for the authentication failure as the token is not domain scoped I think it make sense, even if the admin_token may have another kind of error than 401, authorisation failure, what do you think ?

[Thomas Hsiao (thomas-hsiao)]

From my verification, both all users and all groups are not supported if domain-specific config. is enabled, as expected in the doc. both returns 401 "Unauthorized". Those calls will need either a domain filter to be specified or usage of a domain scoped token.
I think it make sense to limit both users and groups, because it may exceed the ldap server list limit easily and returns 500 errors

Listing of the rest resource data (such as projects/domians/roles/etc.) are supported if a admin-token is provided even the domain -specific config enabled.

$ curl -k -H "X-Auth-Token:ADMIN" http://localhost:35357/v3/users
{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}

$ curl -k -H "X-Auth-Token:ADMIN" http://localhost:35357/v3/groups
{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

Thanks for the detailed explanation. It makes sense and it's in the documentation as pointed out by Thomas:

"It is important to notice that by enabling either of the domain-specific configuration methods, the operations of listing all users and listing all groups are not supported, those calls will need either a domain filter to be specified or usage of a domain scoped token."

I think it's fair to close this bug as invalid.

[Steve Martinelli (stevemar)]

recently chatted with jlk on irc and he commented that using the config option may not make the most sense.

in his example there are multiple domains, but all in a single (SQL) backend. enabling the 'domain_specific_drivers_enabled' option then caused an unexpected change in behaviour.

maybe we should determine if there are indeed multiple backends before changing the behaviour?

[Jesse Keating (jesse-keating)]

Honestly, I'd prefer that it worked the same regardless of the config entry or backend configuration. End consumers of the API don't necessarily know how keystone is configured or what backends may be enabled, so they'll have to program defensively anyway and always pass the domain, just in case the configuration is such that you only get domain scoped results. Keystone should strive for the path of least surprise here.

[David Stanek (dstanek)]

I'm in favor of marking this as WONTFIX. It is the documented behavior as Thomas wrote in comment #4.

If we did want to change it I think it would be messy and confusing. To list users we'd have to iterate over all configured drivers, for which a user has authz on, and pull the lists, squash them together, etc. (This would bring up the pagination debate all over again.)

Will real users ever have the need, or access, to list all users in multiple domains? It seems like a cloud operator issue and maybe one for our future selves if reseller ever comes back around.

[David Stanek (dstanek)]

Assigning to Henry to get his opinion

[Guang Yee (guang-yee)]

Consider the use cases for domain-specific backend, which is mainly designed for (read-only) LDAP integration for identities, it is unlikely that the operation of listing all users from all LDAP domains at once would yield pleasant user experience. This is the same logic we used to argue against pagination in the first place. So iterating through all the domains do list all users in one shot is not the way to go.

On the other hand, API consistency matters, even with proper documentation on the change of API behavior depending on backend configuration. We should NOT be forcing API users to code around different backend configurations when we don't even have a way for them to discover that information programmatically. Horizon is currently facing a similar issue, which is unable to discover whether Keystone is running with a read-only LDAP backend. Therefore, it doesn't know whether to grey-out the user create/update menu. This also result in poor user experience.

I can think of the following options:

1. always make user pass in the domain filter or get the domain filter from the token scope as user management should always required a domain-scoped token
2. return HTTP 501 not implemented if domain filter is absent
3. have a way for users to discover the API behavior

[Lance Bragstad (lbragstad)]

1.) Would this break backwards compatibility? Previously, the API would assume the domain of the token used to do the operation. Is that still going to be the case here?
2.) Same concern as 1

[Dolph Mathews (dolph)]

Guang, I agree that this is primarily a user experience & discoverability issue. Having inconsistent API behaviors that depend on the contents of keystone.conf needs to be fixed.

1. Certainly breaks backwards compatibility in the API.

2. Violates API change guidelines.

3. Is a workaround that does nothing to improve the user experience. The API should be self-consistent.

[Henry Nash (henry-nash)]

So the descriptions of what is being experienced here is "as designed" (for better for worse). Here's a summary of the rationale we used at the time:

1) Hard problem - if you have multiple identity (i.e. users/groups) repositories (e.g. LDAP), each mapped to a domain, then we definitely do not want us talking to all of them in order to list all users/groups....particularly as if you DO have such a configuration, the requirement to do this is likely very low. The classic example is a public cloud with multiple customers, each in a domain, each with their own LDAP. I doubt the customers even want the ability of anyone to list all the users/groups

2) Given 1), how were we to make this restriction as "unsurprising" as possible? We did debate actually banning listing users and groups without specifying a domain_id altogether (and I know quite a few people still want us to do this). The issue is that list users is one of those common things you do on small systems - and in those cases, it kind of makes sense. We considered only requiring a domain_id when there was more than one LDAP (i.e. if all the other backends were storing their identity data in sql you wouldn't need to provide it) - however, we thought this was too hard to explain and would lead to more surprises for little gain.

3) We chose to require a domain_id (when you have enabled domain-specific configuration) for listing users & groups (but not projects) on the basis that we are restricting access only to the data that can actually be split into domain-specific repositories - i.e. identity data. Maybe this is not obvious to the caller, but we were trying to balance the restrictions vs flexibility. However, we allow this additional restriction to be added (if required by the operator) via policy files. e.g. a typical mzulti-domain policy file will require a domain_id when listing projects (unless you are cloud admin) - see police.v3cloudsample.json - so the impact of our restriction choices is really consigned to cloud admins/operator.

On balance, I think we probably still have the balance correct (although am not conceptually wedded to out current position), so currently I would vote for a WONTFIX.

[Steve Martinelli (stevemar)]

Sounds like this is a WONTFIX.