keystone integration with LDAP broken in containerized deployment, manually keystone restart needed

[Raildo Mascena de Sousa Filho]

after successful fresh Openstack Pike deploy, the keystone v3 domain for the Active Directory environment was created, however, I could not get any users to return. Once I restarted the keystone processes within the keystone docker container, it started working:

[root@controller1 heat-admin]# docker exec -it keystone pkill -HUP -f keystone

Version-Release number of selected component (if applicable):

How reproducible:

(overcloud) [stack@openstack ~]$ openstack domain list
+----------------------------------+------------+---------+--------------------+
| ID | Name | Enabled | Description |
+----------------------------------+------------+---------+--------------------+
| 58acbdc9da0b4ada8fdf4446ce8e0ca4 | LAB | True | |
| c31ba5db93e649f888e3d3c2aa92a929 | heat_stack | True | |
| default | Default | True | The default domain |
+----------------------------------+------------+---------+--------------------+
(overcloud) [stack@openstack ~]$ openstack user list --domain lab

(overcloud) [stack@openstack ~]$ openstack user list --domain lab

(overcloud) [stack@openstack ~]$ openstack user list --domain lab

(overcloud) [stack@openstack ~]$ openstack user list --domain lab

(overcloud) [stack@openstack ~]$ openstack user list --domain lab

[root@controller1 heat-admin]# docker exec -it keystone pkill -HUP -f keystone

(overcloud) [stack@openstack ~]$ openstack user list --domain lab

+------------------------------------------------------------------+---------------+
| ID | Name |
+------------------------------------------------------------------+---------------+
| 7ebf8923d6a15322c3e7b611d8e9028bd4a70715199f64bba59b83c434b3ab36 | Administrator |
| 49c22aa306719865a691b875f70de6dfcfc41da0e3bad2d82f15abdae6912c7a | Guest |
| aae9ca159631e25687a6fdbca64fbd1933b2380fe00a0ea3c05d539c54099440 | WIN2K8SVR$ |
| 809b820c83b179dec31864db0d17ba3341422765632aec4be9efe2df2ee27502 | krbtgt |
| 7662775af3175eda8bad21fd8766d848c31a66f6679fe85e6a6c2092364aedd0 | svc.acct1 |
| 11d4448b62a3f6b49b8ce483e7791ecba43525c2ccffbea5407fc72e19b58f35 | svc.acct2 |
| 47540c6e6c444eb03f03bebbc48730d5d1d3866811260e8f84c5d085aa40b3c6 | kholden |
| ee7b53d02827adb2ed6f808a07c3b9e20ee9f56b0b250c3891948f6aa5400078 | svc.acct3 |
+------------------------------------------------------------------+---------------+
(overcloud) [stack@stack@openstack ~]$

Steps to Reproduce:
1. on directory, run `sed -i 's/puppet_tags\: keystone_config/puppet_tags\: keystone_config,keystone_domain_config/' /usr/share/openstack-tripleo-heat-templates/docker/services/keystone.yaml`

2. deploy fresh OpenStack Pike using keystone_domain_specific_ldap_backend.yaml template
3. run source ~/overcloudrc.v3; openstack user list --domain DOMAIN_NAME. this will result in no users being returned

Additional info:
This is how I fixed it after deployment:
1. ssh to all controllers and run sudo docker exec -it keystone pkill -HUP -f keystone

2. run source ~/overcloudrc.v3; openstack user list --domain DOMAIN_NAME. this will result in LDAP users returned (given your LDAP configs are correct)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/557736

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/557736
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=ffc14e3067e6eb0039dceec9656f07d7663dc87f
Submitter: Zuul
Branch: master

commit ffc14e3067e6eb0039dceec9656f07d7663dc87f
Author: Juan Antonio Osorio Robles <email address hidden>
Date: Thu Mar 29 12:41:51 2018 +0000

    Refresh keystone after deployment

    This is necessary for certain setups (such as enabling multiple LDAP
    domains). So, instead of always adding checks every time to see if
    we need to refresh or not, lets just do it always, thus simplifying
    the already convoluted logic here.

    Change-Id: Ie1a0b9740ed18663451a3907ec3e3575adb4e778
    Closes-Bug: #1748219
    Co-Authored-By: Raildo Mascena <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/558547

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/558548

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/558548
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=1303503457df6466b159924b87bf08b418927d34
Submitter: Zuul
Branch: stable/pike

commit 1303503457df6466b159924b87bf08b418927d34
Author: Juan Antonio Osorio Robles <email address hidden>
Date: Thu Mar 29 12:41:51 2018 +0000

    Refresh keystone after deployment

    This is necessary for certain setups (such as enabling multiple LDAP
    domains). So, instead of always adding checks every time to see if
    we need to refresh or not, lets just do it always, thus simplifying
    the already convoluted logic here.

    Change-Id: Ie1a0b9740ed18663451a3907ec3e3575adb4e778
    Closes-Bug: #1748219
    Co-Authored-By: Raildo Mascena <email address hidden>
    (cherry picked from commit ffc14e3067e6eb0039dceec9656f07d7663dc87f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/558547
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=c454aea43ae01bc227c46850baff886e261d3677
Submitter: Zuul
Branch: stable/queens

commit c454aea43ae01bc227c46850baff886e261d3677
Author: Juan Antonio Osorio Robles <email address hidden>
Date: Thu Mar 29 12:41:51 2018 +0000

    Refresh keystone after deployment

    This is necessary for certain setups (such as enabling multiple LDAP
    domains). So, instead of always adding checks every time to see if
    we need to refresh or not, lets just do it always, thus simplifying
    the already convoluted logic here.

    Change-Id: Ie1a0b9740ed18663451a3907ec3e3575adb4e778
    Closes-Bug: #1748219
    Co-Authored-By: Raildo Mascena <email address hidden>
    (cherry picked from commit ffc14e3067e6eb0039dceec9656f07d7663dc87f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.12 release.