Invalid SQL Identity Assertion - Load Config from Database

[Priti Desai]

I have a default domain pointing to LDAP and ServiceDomain pointing to SQL identity backend. This kind of configuration is supported with enabling domain specific drivers. While upgrading to Kilo to leverage domain config from database capability, the same configuration is not supported.

$ openstack user list --domain 5681226a68de4f7ea8a2bd247d0fc54e
ERROR: openstack Invalid domain specific configuration: Domain specific sql drivers are not supported via the Identity API. One is specified in /domains/5681226a68de4f7ea8a2bd247d0fc54e/config (Disable debug mode to suppress these details.) (HTTP 403) (Request-ID: req-7bc89750-60de-45f2-8c80-05777a8469da)

Notice the same domain ID in request and error message.

In identity/core.py:

def _load_config_from_database(self, domain_id, specific_config):

        def _assert_not_sql_driver(domain_id, new_config):
            """Ensure this is not an sql driver.

            Due to multi-threading safety concerns, we do not currently support
            the setting of a specific identity driver to sql via the Identity
            API.

            """
            if new_config['driver'].is_sql:
                reason = _('Domain specific sql drivers are not supported via '
                           'the Identity API. One is specified in '
                           '/domains/%s/config') % domain_id
                raise exception.InvalidDomainConfig(reason=reason)

_assert_not_sql_driver causes such restriction, any domain with sql identity backend is prohibited which should be restricted to at least one.

[Morgan Fainberg (mdrnstm)]

There was actually an error in the logic for enforcing per-domain SQL drivers in the previous releases which was corrected in Kilo allowing more than one SQL driver to be loaded.

(1) Most commonly the base Identity driver is sql:

[identity]
driver=keystone.identity.backends.sql.Identity

(2) And then the deployer would specify an LDAP per-domain backend for <domain>, which might be the default domain.

(3) The deployer would then supply another per-domain backend that re-specified SQL as the driver for that domain.

The issue lies in the order the configurations were loaded. Unfortunately, there are a number of edge cases that were addressed by not allowing multiple SQL drivers, notably that you run into issues with different SQL connection strings.

--------

Does the above describe your deployment? If so, it is highly recommended that the second SQL-backed domain is merged into the main SQL backend, since the secondary backend is not really needed (it is only needed for LDAP + SQL or Multiple LDAP backends).

As a point, please keep in mind domain configs in the DB are highly experimental and subject to change over the next cycle (due to issues as they are found)

[Morgan Fainberg (mdrnstm)]

If my description of your configuration is incorrect and/or this is some other issue that is not solved with the main (from the keystone.conf, not the domain-specific backend) identity driver being set to SQL, please respond and set this ticket back to "new" status.

[Priti Desai (priti-desai)]

Thanks Morgan,

I have a little different setup:

(1) keystone.conf has set the identity backend to LDAP (for default domain):

[identity]
driver=keystone.identity.backends.ldap.Identity

(2) LDAP per-domain backend for non-default domain

(3) Per-domain backend with SQL as the driver for that domain. This domain is primarily used for service accounts. Only, one single domain has SQL as the identity backend.

[Morgan Fainberg (mdrnstm)]

Hi Priti,

I just wanted to let you know we haven't forgotten about this. I am working with Henry Nash to get some eyes on this so we can correct the problem you're describing. (I'm also a little behind on bug triage due to the summit and US Holiday earlier this week).

[Henry Nash (henry-nash)]

Hi Priti,

So you are correct in what you have deduced - we are being more restrictive in the support of SQL backends with Identity API domain configuration management. This restriction was put in place due to concerns that, in multi process/threaded keystone installations) we could get race conditions if two API calls were issue at the "same time" to set a domain to SQL...it might be possible for both to get actioned, leading to 2 domains being set to SQL (which will not work correctly with how we use the current SQLAlchemy package).

If it were possible for you to use the default domain as SQL, and all others as LDAP, then that would work round this issue. It may be that you started out with just the default domain with LDAP (since that's all we used to support) and grew your installation to add the specific domains with LDAP or SQL - so understand how you may have got here. It's true, that if we were not concerned about the race condition, we could remove the restriction we imposed (and your configuration would have worked) - so it's a bit of a balance.

In Liberty, I am working on addressing a number of the multi-process issues that arise with the Identity API Domain Configuration support, but it is not imminent. Let me know if you can work round this restriction (as described above), or whether this is not possible for you.

[Henry Nash (henry-nash)]

OK, so I'm going to work to fix this asap....will post a proposed fix this week.

[Priti Desai (priti-desai)]

Thanks Henry and Morgan for looking into this.

Henry, you are absolutely correct, we started off setting default domain with LDAP and grew our installation to add specific domains with LDAP, the main reason behind moving towards this kind of architecture is to isolate users.

Before using Domain Specific Backends:

openstack user list --domain default shows the entire list of users from user_tree.
openstack user list --domain DomainA again shows the entire list of users from default domain.

After migrating to Domain Specific Backends:

openstack user list --domain default shows the entire list of users from user_tree.
openstack user list --domain DomainA shows limited set of users from user_tree associated with DomainA.

Our customers are extremely happy with this feature and I appreciate all your efforts for building it.

For now, I have worked around this issue by modifying the if condition:

 if new_config['driver'].is_sql:

to:

if (new_config['driver'].is_sql and (self.driver.is_sql or self._any_sql)):

Cheers
Priti

[Henry Nash (henry-nash)]

Yes, that work around (while not being multi-process safe), will solve your immediate problem. I should be posting the first version of a patch (to the Liberty code stream) in a few days....and we'll look at back porting it once it gets approved

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/191976

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/191976
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1696b875dd7b37c5d9569a1a5d5ad25e4410a1ef
Submitter: Jenkins
Branch: master

commit 1696b875dd7b37c5d9569a1a5d5ad25e4410a1ef
Author: Henry Nash <email address hidden>
Date: Mon Jun 15 21:57:47 2015 +0100

    Relax newly imposed sql driver restriction for domain config

    The first implementation of the domain config API introduced a
    new restriction in terms of which identity drivers could be SQL
    backend (to avoid potential multi-process race conditions). This
    restriction specified that only the default domain could be SQL
    backend.

    This patch resolves these issues and hence ensures the restrictions
    in terms of SQL identity drivers are the same as the original config
    file based approach (i.e. only one driver can be SQL backend, but
    it can be for any given domain).

    Although this patch changes the V8 driver interface, this is for
    the domain_config backend which is still marked as experimental.

    Closes-Bug: #1450344
    Change-Id: Id1b4415cb5f263dc7afd9e25bd0da2b667f1c90a