Test that create or delete user in an LDAP backed user source have to be blacklisted

[Doug Schveninger]

If the keystone user source is immutable, like an LDAP active directory implementation, the tempest test cases that try to create or delete a user will fail. We would like tempest test to skip in this case. We would like to propose a conf setting in the identity group called immutable-user-source that is defaulted to false. If the value is true and the rest client for keystone create or delete user is called a skip check exception will be raised.

This is a simple change so I defined it as a bug. Let me know if you would like to address it as a specification.

[Trevor McCasland (twm2016)]

I think we should do three things.
1. add an option in tempest.conf called 'immutable_user_source=True/False' or something we all agree on.
2. check for immutable_user_source in the create_test_user method in the temepest base implementation and skip the test if it is True. https://github.com/openstack/tempest/blob/2d6bcaa7e55be9fd070bade12824709ac848f8fe/tempest/api/identity/base.py#L70
3. Add a similar skip if force_tenant_isolation is True and immutable_user_source is True in test.py skip_checks, kind of similar to what is done here https://github.com/openstack/tempest/blob/2d6bcaa7e55be9fd070bade12824709ac848f8fe/tempest/test.py#L283

Once those are done, I think that will cover most of the issues.

[Doug Schveninger (ds6901)]

Trevor I agree with new conf value and the create and delete rest client ship check.

I do not see a need for number 3 since the create and delete rest client skip check will also cover the skipping of force_tenant_isolation is true test cases.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/585536

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/586603

[OpenStack Infra (hudson-openstack)]

Change abandoned by Anna Pankiewicz (<email address hidden>) on branch: master
Review: https://review.openstack.org/586603

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/587846

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/587878

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/588393

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/588395

[OpenStack Infra (hudson-openstack)]

Change abandoned by Anna Pankiewicz (<email address hidden>) on branch: master
Review: https://review.openstack.org/587846

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/589279

[Ghanshyam Mann (ghanshyammann)]

Can we get more information from keystone side about when and what basis keystone does not allow to create the resources(user in this case) ?

Any user documentation or interfaces link will be helpful to understand that in what condition keystone will control this behavior and accordingly we can check if that is right things to do in Tempest also.

[Ghanshyam Mann (ghanshyammann)]

for user resource as immutable, dynamic cred or force_tenant_isolation tests will not work for LDAP right ? is this bug solving the issue for per-provisioned cred only ?

Also if there are more immutable source from keystone in case of LDAP or other, then it might end up with skipping the most of the tests.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/585536
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=a0710668324415497b3ae1b3968156333cc58644
Submitter: Zuul
Branch: master

commit a0710668324415497b3ae1b3968156333cc58644
Author: Anna Pankiewicz <email address hidden>
Date: Tue Jul 24 14:56:42 2018 -0500

    Support immutable user source

    If the keystone user source is immutable, such as an LDAP active
    directory implementation, tempest tests that try to create, delete
    or modify a user will fail. Instead of failing, we would like them to
    skip. Proposed is an additional config setting in the auth group
    called immutable_user_source that is defaulted to false.

    To handle this on a test by test basis and avoid modifying behavior
    that depends on the identity base class, we propose using a
    testtools decorator to skip tests based on the new config setting.

    One test class so far has been adjusted to use the decorator, and if
    the config setting is set to true, the tests will skip accordingly.

    Co-Authored-By: Michael Beaver <email address hidden>
    Partial-Bug: 1777047
    Change-Id: Idc09d6272386f026a899787c2151745916a7228d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/588393
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=11a2dd79190660a2a463440e610b82217afda04e
Submitter: Zuul
Branch: master

commit 11a2dd79190660a2a463440e610b82217afda04e
Author: Anna Pankiewicz <email address hidden>
Date: Thu Aug 2 16:03:06 2018 -0500

    Immutable user source: test_default_project_id

    If the keystone user source is immutable, such as an LDAP
    active directory implementation, tempest tests that try
    to create or delete a user will fail. Instead of failing,
    we would like them to skip. This change uses a testtools
    decorator to avoid unnecessary modifications and allow those tests
    to skip. In [1], I introduced the config setting that allows
    this to happen.

    [1] https://review.openstack.org/#/c/585536/

    Change-Id: I4d928fdbf04de65a73977dbe4a818dff4cb2aa8d
    Partial-Bug: 1777047

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625714

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625715

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625718

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625722

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625723

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/631582
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=566237c0a1c1fc8421415fd414ce845956b41d90
Submitter: Zuul
Branch: master

commit 566237c0a1c1fc8421415fd414ce845956b41d90
Author: Trevor McCasland <email address hidden>
Date: Thu Jan 17 11:13:34 2019 -0600

    Immutable user source: v3 test_groups

    If the keystone user source is immutable, such as an LDAP
    active directory implementation, tempest tests that try
    to create or delete a user will fail. Instead of failing,
    we would like them to skip. This change uses a testtools
    decorator to avoid unnecessary modifications and allow those tests
    to skip.

    Partial-Bug: #1777047

    Change-Id: I8a0d5935b695a1649b204bd3c3351f34a80e9059

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/631530
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=c3f07b441f6a077efc478fb50dd1bb8f04700066
Submitter: Zuul
Branch: master

commit c3f07b441f6a077efc478fb50dd1bb8f04700066
Author: Trevor McCasland <email address hidden>
Date: Thu Jan 17 08:53:24 2019 -0600

    Immutable user source: v3 test_inherits

    If the keystone user source is immutable, such as an LDAP
    active directory implementation, tempest tests that try
    to create or delete a user will fail. Instead of failing,
    we would like them to skip. This change uses a testtools
    decorator to avoid unnecessary modifications and allow those tests
    to skip.

    Parital-Bug: #1777047

    Change-Id: I1d8105c24eb9b20e563de962f9e4d5776937126a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/625718
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=c98a3254c84c00f05db63a74f26010ab80e699c6
Submitter: Zuul
Branch: master

commit c98a3254c84c00f05db63a74f26010ab80e699c6
Author: nicolas <email address hidden>
Date: Mon Dec 17 13:06:02 2018 -0800

    Immutable user source: Non-admin V2 and V3 tests

    If the keystone user source is immutable, such as an LDAP
    active directory implementation, tempest tests that try
    to create or delete a user will fail. Instead of failing,
    we would like them to skip. This change uses a testtools
    decorator to avoid unnecessary modifications and allow those tests
    to skip. In [1], I introduced the config setting that allows
    this to happen.

    [1] https://review.openstack.org/#/c/585536/

    Change-Id: I786499204acdb929ee341e014bcb0459d2f27483
    Partial-Bug: 1777047

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/672798

[OpenStack Infra (hudson-openstack)]

Change abandoned by Nicolas Helgeson (<email address hidden>) on branch: master
Review: https://review.opendev.org/625723

[OpenStack Infra (hudson-openstack)]

Change abandoned by Nicolas Helgeson (<email address hidden>) on branch: master
Review: https://review.opendev.org/625722

[OpenStack Infra (hudson-openstack)]

Change abandoned by Nicolas Helgeson (<email address hidden>) on branch: master
Review: https://review.opendev.org/625715

[OpenStack Infra (hudson-openstack)]

Change abandoned by Nicolas Helgeson (<email address hidden>) on branch: master
Review: https://review.opendev.org/625714

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/670590
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=bae11b6dc0c776ef008fe09c5885e53392ded002
Submitter: Zuul
Branch: master

commit bae11b6dc0c776ef008fe09c5885e53392ded002
Author: Leo Henken <email address hidden>
Date: Fri Jul 12 10:42:58 2019 -0500

    Skip tests in TrustsV3TestJSON with immutable user source

    Adds a simple condition in skip_checks to skip the entire test class
    if the environment is configured to have an immutable user source. If
    so, a skipException is raised and the entire class is not executed.

    Partial-Bug: 1777047

    Change-Id: I8c1c1df25401157c667bce97d4995f738eec8e05

[OpenStack Infra (hudson-openstack)]

Change abandoned by Leo Henken (<email address hidden>) on branch: master
Review: https://review.opendev.org/672798
Reason: Abandoned in order to split into two commits. (lh236s)

[Doug Schveninger (ds6901)]

I believe that all of these are done and this can be closed

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ghanshyam <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tempest/+/589279

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ghanshyam <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tempest/+/588395

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ghanshyam <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tempest/+/587878