duplicated definition of config option memcached_servers

[Lianhao Lu]

When using the latest config generator from oslo-incubator, see https://review.openstack.org/62815, and trying to generate the nova.conf.sample by using the following commands:

NOVA_CONFIG_GENERATOR_EXTRA_MODULES=keystoneclient.middleware.auth_token tools/config/generate_sample.sh -p nova

2013-12-18 18:22:03.187 30506 CRITICAL nova [-] Unable to find group for option memcached_servers, maybe it's defined twice in the same group?

This is because the config option memcached_servers is defined both in the python module keystoneclient.middleware.auth_token and nova.openstack.common.memorycache.

[Lianhao Lu (lianhao-lu)]

Maybe the keystoneclient should switch to oslo's momorycache implementation or rename the option?

[Lianhao Lu (lianhao-lu)]

I dig into this a little bit.

The config.generator works by first importing all the python modules, then for each imported python module, it will try to find all the CfgOpt options in it and print it.

During the finding process, the config.generator._guess_groups() will try to find if the CfgOpt is belonging to the 'DEFAULT' group or other groups, this is where the error happens.

The reason behind this is that "keystoneclient.middleware.auth_token" imports "keystoneclient.openstack.common.memorycache" which registers an option "memcached_servers". Later, when importing "nova.openstack.common.memorycache" which also has the same option "memcached_servers" , that option is never registered because we already have the same option from keystoneclient registered.

When the config.generator "_guess_groups()" the option "memcached_servers" from "nova.openstack.common.memorycache", it uses operator '==' to compare if two options are the same. But the oslo.cfg.Opt class doesn't implement the __eq__ method, it only implements the __ne__ method. That's why the '==' operator doesn't give us the expected result.

We should implement the __eq__ method in oslo.cfg. Before that, we should use 'not a == b' in the config.generator as a temporary workaround.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/63073

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/63073
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=7c18261df5609d689a32c697c77c0a97512342bc
Submitter: Jenkins
Branch: master

commit 7c18261df5609d689a32c697c77c0a97512342bc
Author: Lianhao Lu <email address hidden>
Date: Thu Dec 19 16:44:10 2013 +0800

    Temporary workaround for config generator

    Current oslo.config.cfg.Opt doesn't support operator '=='. So use 'not
    !=' instead. This is a temporary workaround before the newly released
    oslo.config support the operator '=='.

    Partial-Bug: 1262148

    Change-Id: I19279cd211594bb107777d6b98ebe6990063ebc7

[Zane Bitter (zaneb)]

Can you explain why this is not a bug in Nova? On the face of it, the code in Oslo is doing exactly what it intended: finding options that have been registered twice in the same group and raising the error:

  "Unable to find group for option memcached_servers, maybe it's defined twice in the same group?"

After this change, the code no longer detects these errors as intended. Worse, now if a similar option is defined in a different group, that option will fail to show up in the output (see bug 1288586).

It seems like one or both of these Nova options should be in a separate group, so that they can be configured independently.

[Lianhao Lu (lianhao-lu)]

I'm just following same logic at https://github.com/openstack/oslo.config/blob/master/oslo/config/cfg.py#L515 to detect if an opt is registered in the same group.

[Zane Bitter (zaneb)]

Ah, interesting, thank you.

So it seems like we need to stop _guessing_ which group an option belongs to and actually find out. In the general case of equivalent options in different groups each being registered multiple times, that appears to be impossible though :( Oslo.config would have to store the group in the option at the time it is registered.

[Zane Bitter (zaneb)]

BTW I've just noticed that when the option gets registered twice it now appears in the sample config file twice too, which is probably undesirable. I'll work on a fix as part of bug 1288586.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/81412

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/81414

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/81415

[Morgan Fainberg (mdrnstm)]

All new development has been moved to keystonemiddleware. If this is a continuing issue please reopen/open new against keystonemiddleware.