Glance config generator files need to be updated for oslo_db

[Peter Nordquist]

The current config generator files at etc/oslo-config-generator/* need to be updated to import the correct packages for oslo_db.concurrency. They currently import oslo.db.concurrency which does not exist in the Mitaka version of oslo.db. I'm currently using the python2-oslo-db-4.6.0-1 package in the RDO repositories to attempt to build these configuration files.

[Peter Nordquist (pnnl-plnordquist)]

Here's the commit that removes the old package files https://github.com/openstack/oslo.db/commit/5e9b3af32e51b4daff4423380d0621b753d80b82. Changing these files seemed to get the generator working but the new files were missing the thread pool settings.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/316898

[Doug Hellmann (doug-hellmann)]

You are confusing the python namespace package with the name of the entry point the config generator looks for. The entry points defined in oslo.db are "oslo.db" and "oslo.db.migration". Those names are independent of the names of the files they point to, or the python package names. You should not need to change the configuration file in glance to "oslo_db" because that is not a valid name for an entry point.

[Cyril Roelandt (cyril-roelandt)]

@Doug: thanks for your comment!

@Peter: this bug seems invalid, could you elaborate on what settings are missing? If I run "tox -egenconfig" on master, I see a lot of things related to the thread pool in the generated files:

$ grep "thread pool" etc/*.conf.sample
etc/glance-api.conf.sample:# The size of thread pool to be used for scrubbing images. The default
etc/glance-api.conf.sample:# Size of executor thread pool. (integer value)
etc/glance-api.conf.sample:# Enable the experimental use of thread pooling for all DB API calls
etc/glance-api.conf.sample:# The number of thread pools to perform a multipart upload in S3.
etc/glance-cache.conf.sample:# The number of thread pools to perform a multipart upload in S3.
etc/glance-glare.conf.sample:# Enable the experimental use of thread pooling for all DB API calls
etc/glance-glare.conf.sample:# The number of thread pools to perform a multipart upload in S3.
etc/glance-manage.conf.sample:# Enable the experimental use of thread pooling for all DB API calls
etc/glance-registry.conf.sample:# Size of executor thread pool. (integer value)
etc/glance-registry.conf.sample:# Enable the experimental use of thread pooling for all DB API calls
etc/glance-scrubber.conf.sample:# The size of thread pool to be used for scrubbing images. The default
etc/glance-scrubber.conf.sample:# Enable the experimental use of thread pooling for all DB API calls
etc/glance-scrubber.conf.sample:# The number of thread pools to perform a multipart upload in S3.

[Peter Nordquist (pnnl-plnordquist)]

Ok I think this was my mistake here, I tried to generate the configuration from packaged RPMs in CentOS but the tox -egenconfig works properly. Seems like it might be a packaging bug for oslo.db from RDO.

[David Comay (comay)]

Just to note I'm seeing similar issues with Cinder on Mitaka. The stack trace I eventually see is of the form:

$ oslo-config-generator --namespace oslo.db.concurrency
Traceback (most recent call last):
  File "/usr/bin/oslo-config-generator", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/vendor-packages/oslo_config/generator.py", line 419, in main
    generate(conf)
  File "/usr/lib/python2.7/vendor-packages/oslo_config/generator.py", line 401, in generate
    groups = _get_groups(_list_opts(conf.namespace))
  File "/usr/lib/python2.7/vendor-packages/oslo_config/generator.py", line 333, in _list_opts
    loaders = _get_raw_opts_loaders(namespaces)
  File "/usr/lib/python2.7/vendor-packages/oslo_config/generator.py", line 305, in _get_raw_opts_loaders
    invoke_on_load=False)
  File "/usr/lib/python2.7/vendor-packages/stevedore/named.py", line 55, in __init__
    verify_requirements)
  File "/usr/lib/python2.7/vendor-packages/stevedore/extension.py", line 171, in _load_plugins
    self._on_load_failure_callback(self, ep, err)
  File "/usr/lib/python2.7/vendor-packages/stevedore/extension.py", line 163, in _load_plugins
    verify_requirements,
  File "/usr/lib/python2.7/vendor-packages/stevedore/named.py", line 123, in _load_one_plugin
    verify_requirements,
  File "/usr/lib/python2.7/vendor-packages/stevedore/extension.py", line 184, in _load_one_plugin
    plugin = ep.resolve()
  File "/usr/lib/python2.7/vendor-packages/pkg_resources/__init__.py", line 2386, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ImportError: No module named oslo.db.concurrency

This is with Solaris packages, in particular oslo.config 3.9.0, stevedore 1.12.0, and setuptools 19.2.

[David Comay (comay)]

Oh, and oslo.db 4.7.0.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Cyril Roelandt (<email address hidden>) on branch: master
Review: https://review.openstack.org/316898