tempest_tempest_conf_overrides ignored in the featureset if set in job featureset_override

[Marios Andreou]

as first discussed in https://bugs.launchpad.net/tripleo/+bug/1867807/comments/5 for jobs that use featureset 052 (all standalone jobs) when a job definition contains the "tempest_tempest_conf_overrides" in the job vars featureset_override, the values given in the featureset are ignored and only the ones used in the job are used.

Example: fs52 [1] has the following:

        tempest_tempest_conf_overrides:
          auth.use_dynamic_credentials: true
          image.image_path: http://images.rdoproject.org/cirros/cirros-0.4.0-x86_64-disk.img

The job definition for periodic-tripleo-ci-centos-8-scenario004-standalone-master [2] has the following:

        tempest_tempest_conf_overrides:
          'auth.tempest_roles': 'Member'
          'share.multitenancy_enabled': 'False'
          'share.enable_protocols': 'cephfs'
          'share.capability_snapshot_support': 'False'
          'share.capability_create_share_from_snapshot_support': 'False'

However, in the resulting job run [3] only the values from the job definition are used:

        overrides:
            auth.tempest_roles: Member
            share.capability_create_share_from_snapshot_support: 'False'
            share.capability_snapshot_support: 'False'
            share.enable_protocols: cephfs
            share.multitenancy_enabled: 'False'

From a search it looks like the problem is limited to featureset 052 and standalone jobs both check/gate and periodic [4][5]. This makes sense since the featureset_override exceptions were added because the standalones share featureset052 but need to override individual values.

I think we need to ensure that the tempest_tempest_conf_overrides from featureset052 are well represented in the jobs themselves, and then remove it from the featureset.

[1] https://github.com/openstack/tripleo-quickstart/blob/16beb60b8f6f6dc7233dd3a8e2e839ec48ee8a64/config/general_config/featureset052.yml#L67
[2] https://github.com/rdo-infra/rdo-jobs/blob/f968638e42e4edce27a4ffdad09929ab0a462346/zuul.d/standalone-jobs.yaml#L267-L282
[3] https://logserver.rdoproject.org/openstack-component-cinder/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario004-standalone-cinder-master/6eeb8cc/logs/undercloud/home/zuul/tempest/etc/profile.yaml.txt.gz
[4] http://codesearch.openstack.org/?q=tempest_tempest_conf_overrides
[5] https://codesearch.rdoproject.org/?q=tempest_tempest_conf_overrides

[Marios Andreou (marios-b)]

posted a test change yesterday there https://review.rdoproject.org/r/26076/2 and tested it with testproject at https://review.rdoproject.org/r/26078

results are good

        * https://logserver.rdoproject.org/78/26078/1/check/periodic-tripleo-ci-centos-8-scenario004-standalone-cinder-master/53c1def/logs/undercloud/home/zuul/tempest/etc/profile.yaml.txt.gz

overrides:
    auth.tempest_roles: Member
    image.image_path: http://images.rdoproject.org/cirros/cirros-0.4.0-x86_64-disk.img
    share.capability_create_share_from_snapshot_support: 'False'
    share.capability_snapshot_support: 'False'
    share.enable_protocols: cephfs
    share.multitenancy_enabled: 'False'

going to add the rest

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/714909

[Marios Andreou (marios-b)]

so just discussed with sshnaidm and chkumar on irc

I posted that "tripleo-ci master: Add tempest_tempest_conf_overrides from featureset into jobs vars" https://review.opendev.org/714905 and was about to post the same for rdo-jobs standalone jobs

however chkumar will make the image location default in tempest with https://review.opendev.org/#/q/topic:tripleocirros+(status:open+OR+status:merged

and for the other value (dynamic credentials) it is already default in tempest

12:01 < chandankumar> marios, https://opendev.org/openstack/tempest/src/branch/master/tempest/config.py#L59 it is set to true by default
12:02 < chandankumar> marios, I checked the code,
                      https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L186
12:02 < chandankumar> marios, basically it is already true, that is not needed
12:03 < chandankumar> https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L198

so i posted that one to remove them from the featureset alltogether tripleo-quickstart master: Remove tempest_tempest_conf_overrides from featureset 52 https://review.opendev.org/714909

(don't think we'll need that one then https://review.opendev.org/714905)

[chandan kumar (chkumar246)]

tempest_tempest_conf_overrides:
   auth.use_dynamic_credentials: true

above overrides is also no longer needed, as in python-tempestconf, it is setted by default to true.
in tempest https://opendev.org/openstack/tempest/src/branch/master/tempest/config.py#L59
and in config_tempest for non-admin and accounts file case it is set to false otherwise it is true.

https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L186 and https://opendev.org/openstack/python-tempestconf/src/branch/master/config_tempest/main.py#L198

From featureset_overrides it needs to be cleanup.

[Marios Andreou (marios-b)]

https://review.opendev.org/#/c/714601/ and https://review.opendev.org/#/c/714609/
changes from chkumar to make the image location default to the one we need.

for the credentials its already default see comment #4

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/714909
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=2c4f3d0cf92bc7fc962c13c3b5339f72c601fe23
Submitter: Zuul
Branch: master

commit 2c4f3d0cf92bc7fc962c13c3b5339f72c601fe23
Author: Marios Andreou <email address hidden>
Date: Wed Mar 25 12:01:59 2020 +0200

    Remove tempest_tempest_conf_overrides from featureset 52

    As described in related bug these values are dropped when the
    same is defined via featureset override on the job. We should
    drop them from the featureset to avoid confusion. They will
    be set as defaults in tempestconf

    Change-Id: I5aec865389be13ec2a42a1cbc72b9e7e30a4805f
    Related-Bug: 1868938

[Marios Andreou (marios-b)]

moving this to fix-released since we merged those * https://review.opendev.org/c/openstack/tripleo-quickstart/+/714909/ https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/714601 https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/714609