common/config/generator.py does not properly handle the same option name in multiple groups

[Steven Dake]

The generator.py in master of oslo-incubator does not properly handle the same option names in different groups.

For example if there is a group "heat_api" with bind_host and "heat_api_cfn" with bind_host, one of them will not generate some of the options. In other cases, entire groups are ignored.

To verify, sync the config module into master of heat and check out the difference of the config file.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/78535

[Steven Dake (sdake)]

Doug,

Stepping back from fixing this problem. The only way I know how to fix it is to rewrite the config generator, and I don't think I'm the appropriate person to do that. There may be a surgical fix, but I'm not sure what it is.

[Zane Bitter (zaneb)]

The cause appears to be the fix for bug 1262148. Previously, when looking for an option in a group, it was compared for identity (since __eq__ was not implemented, '==' acted like 'is'). Now that options are compared only for equality, when equal (but not identical) options appear in multiple groups the option will show up only in the first group in the output.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/79738

[Doug Hellmann (doug-hellmann)]

Heat has all of their options defined in one file, so I proposed using the new entry point hook to generate their configuration because it won't require potentially breaking changes to the config generator and all of the work can be done in the heat repository. We still need to look at this in oslo in the future, so I am leaving the bug open.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/84594

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/84594
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=df9fa6c0c4cedf9577edfbd6951ee67b33386aa1
Submitter: Jenkins
Branch: master

commit df9fa6c0c4cedf9577edfbd6951ee67b33386aa1
Author: Steve Baker <email address hidden>
Date: Wed Apr 2 13:22:45 2014 +1300

    Disable check_uptodate.sh check in pep8

    Running generate_sample.sh currently produces an invalid heat.conf.sample
    due to bug #1288586.

    This change disables the check_uptodate.sh in the tox pep8 so that the
    gate-heat-pep8 job passes while we find a more appropriate solution.

    Change-Id: Ib20c4f892df68f7112d969d801a18a1a5bfbd474
    Related-Bug: #1288586

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/85915

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/85915
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=79d8191c75e3dc5578686fdb68158cae36760641
Submitter: Jenkins
Branch: milestone-proposed

commit 79d8191c75e3dc5578686fdb68158cae36760641
Author: Steve Baker <email address hidden>
Date: Wed Apr 2 13:22:45 2014 +1300

    Disable check_uptodate.sh check in pep8

    Running generate_sample.sh currently produces an invalid heat.conf.sample
    due to bug #1288586.

    This change disables the check_uptodate.sh in the tox pep8 so that the
    gate-heat-pep8 job passes while we find a more appropriate solution.

    Change-Id: Ib20c4f892df68f7112d969d801a18a1a5bfbd474
    Related-Bug: #1288586

[Openstack Gerrit (openstack-gerrit)]

Related fix proposed to branch: stable/havana
Review: https://review.openstack.org/91684

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/91684
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=58de9e6415f5bdabde708c8584b21b59b7e96a88
Submitter: Jenkins
Branch: stable/havana

commit 58de9e6415f5bdabde708c8584b21b59b7e96a88
Author: Steve Baker <email address hidden>
Date: Wed Apr 2 13:22:45 2014 +1300

    Disable check_uptodate.sh check in pep8

    Running generate_sample.sh currently produces an invalid heat.conf.sample
    due to bug #1288586.

    This change disables the check_uptodate.sh in the tox pep8 so that the
    gate-heat-pep8 job passes while we find a more appropriate solution.

    Related-Bug: #1288586
    (cherry picked from commit 79d8191c75e3dc5578686fdb68158cae36760641)

    Conflicts:
     tox.ini

    Change-Id: Ib20c4f892df68f7112d969d801a18a1a5bfbd474

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/95178

[Doug Hellmann (doug-hellmann)]

Does using entry points solve this problem?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/95178
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=52ad1edea2bb6f4209879064488d1bafb6eaef13
Submitter: Jenkins
Branch: master

commit 52ad1edea2bb6f4209879064488d1bafb6eaef13
Author: Thomas Herve <email address hidden>
Date: Tue May 20 10:45:51 2014 +0200

    Use entry points for config generation

    This starts using entry points to generate configuration for the
    common.config and the common.wsgi modules, as they use entries with the
    same name in different groups.
    This also removes configuration for sslutils which is unused, and
    re-enable the check for up-to-date configuration.

    Co-Authored-By: <email address hidden>
    Closes-Bug: #1288586
    Change-Id: If48c62bbb2b1fb641791dad56f7f905e483080fa

[Zane Bitter (zaneb)]

Using entry points solves the immediate problem for Heat by allowing us to ignore the options we don't want. The underlying issue is still there, though.

[Doug Hellmann (doug-hellmann)]

OK, the underlying issue is really a change in the way options are being used now vs. when oslo.config and the config generator were conceived. If entry points makes it possible to reuse options in multiple sections for the config generator, then I'm going to count this as closed.