config.generator could not handle split configs

Bug #1300546 reported by Zhi Yan Liu
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Fix Released
Medium
Zhi Yan Liu
oslo-incubator
Fix Released
High
Unassigned

Bug Description

Current config.generator could not handle split configs for the different services within a project, all configurations be collected and save to a single large template file. But for most project, the code repo contains more then one service, like nova repo/project contains nova-api, nova-compute. So the single template file is hard to be used/maintained for separated service. IMO, it will be cool if config.generator could generate separated configs based on service instead of project.

Example of "split configs" is glance-api.conf versus glance-registry.conf

Tags: config
Revision history for this message
Ben Nemec (bnemec) wrote :

This is blocking Glance's adoption of the config generator, so we need to figure out some solution here.

Changed in oslo:
status: New → Triaged
importance: Undecided → High
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

There are a couple of ways to handle this.

The config generator takes as argument the files and directories to scan. Glance could call it multiple times, passing the right set of files each time.

The generator also has a new feature that lets it load entry points that declare sets of options. This was intended for use in libraries, but could be repurposed for having multiple sets of options registered to different apps. See oslo.messaging for an example of the entry points.

Along with this, Mark asked about how to ensure a service only uses the options it declares it is using. The best way to do that is to register the options at runtime, instead of import time. Doing that ensures that a module can't use an option just because it is being imported as a side-effect of importing another module.

tags: added: config
Mark McLoughlin (markmc)
description: updated
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

Would the -l option to the generator help with this? Glance could define different entry points for each service and then run the generator only specifying those entry points and not using the code-scanning mode. There's an example of setting up the entry points in http://git.openstack.org/cgit/openstack/oslo.messaging/tree/oslo/messaging/opts.py#n56 and http://git.openstack.org/cgit/openstack/oslo.messaging/tree/setup.cfg#n53

Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

Is this still an issue for glance?

Revision history for this message
Zhi Yan Liu (lzy-dev) wrote :

Hi Doung, yes, I'm taking this for glance and will try -l option approach.

Revision history for this message
Zhi Yan Liu (lzy-dev) wrote :

Hello Doung, I tried -l approach with code practice, but it seems it can only controls generate_sample.sh tool whether involving extra library options for discovery, those options which in code base can be generated in any case still. Any suggests?

Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

If you want to generate separate config files with only some of the options, you will need to use *only* the -l option and not have the generator scan the code, or at least have it ignore some of the files when doing the scanning. Heat does something similar, so you can look at their tools/config/oslo.config.generator.rc for an example.

Zhi Yan Liu (lzy-dev)
Changed in glance:
assignee: nobody → Zhi Yan Liu (lzy-dev)
Revision history for this message
Zhi Yan Liu (lzy-dev) wrote :

Adding oslo-config-generator: https://review.openstack.org/#/c/100946/

Changed in oslo:
status: Triaged → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/104476

Changed in glance:
status: New → In Progress
Changed in oslo:
milestone: none → juno-2
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.openstack.org/104476
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=819f28a0b8863bd18f8a14491b5966c8b2723432
Submitter: Jenkins
Branch: master

commit 819f28a0b8863bd18f8a14491b5966c8b2723432
Author: Zhi Yan Liu <email address hidden>
Date: Thu Mar 27 17:39:14 2014 +0800

    Enabling separated sample config file generation

    As a common approach most projects used now and Oslo preferred, this
    change enabled sample configuration file generation mechanism for
    each Glance services.

    This change, as an enhancement, allows generating separated sample
    configuration files for each Glance major services, e.g.:

        etc/glance-api.conf.sample
        etc/glance-cache.conf.sample
        etc/glance-manage.conf.sample
        etc/glance-registry.conf.sample
        etc/glance-scrubber.conf.sample

    It is different than I94d486d6686815c45705a7a9b00fb26062e1eb63
    which only supports generating an unified sample configuration
    file to including all Glance available options.

    This mechanism not only can help auditing by packager, milestone
    maintainer or developer as a function (testenv) of tox, but also
    those separated configuration files could make deployment be easy.
    And it helps keeping sample configuration files be update with
    code change.

    The change added "genconfigs" function as a tox testenv (-egenconfigs),
    and it dependes on oslo-config-generator function of oslo.config.

    The change doesn't introduce those sample files Glance repo, so
    next step is to investigate if we can generate them in gate
    automatically when a change was merged.

    Related-Change-Id: I15686708fc9460948a58cfea3d18dae40ba1fda9
    Related-Change-Id: Iae31856d5886ee78786972d80c7c103c3460a2b3
    Related-Change-Id: I76043b08e2872867e5af2a5ac902e4d092fda5c8
    Closes-Bug: #1300546
    Closes-Bug: #1361963

    Change-Id: Ibe03a3fe80b96ca32acb1a6bea7e38e6075951bb
    Signed-off-by: Zhi Yan Liu <email address hidden>

Changed in glance:
status: In Progress → Fix Committed
Dolph Mathews (dolph)
Changed in glance:
milestone: none → juno-rc1
importance: Undecided → Medium
Thierry Carrez (ttx)
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in glance:
milestone: juno-rc1 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.