An option in deprecated name causes failure on overriding the value

[Kent Wu]

This occurs in Freezer when it calls oslo.config to override(set_override) an option in deprecated name. Here's the way to reproduce it.

>>> from oslo_config import cfg
>>> CONF=cfg.CONF
>>> CONF.register_opt(cfg.StrOpt('os-identity-api-version', deprecated_name='os-auth-ver', dest='os_identity_api_version'))
True
>>> CONF._deprecated_opts
{'DEFAULT': {'os_auth_ver': {'opt': <oslo_config.cfg.StrOpt object at 0x7f478ed02b50>, 'group': None}}}
>>> CONF.set_override('os_auth_ver','3')
No handlers could be found for logger "oslo_config.cfg"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/debtcollector/removals.py", line 261, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2402, in __inner
    result = f(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2727, in set_override
    opt_info = self._get_opt_info(name, group)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 3113, in _get_opt_info
    return opts[opt_name]
KeyError: 'os-identity-api-version'

This is because the key stored in self._opts has replaced '-' to '_'. Will introduce the fix in short.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/571900

[Ben Nemec (bnemec)]

I'm not sure I agree that this is a bug. You registered a deprecated opt named os-auth-ver, you should set the override using the same name. Only by looking at an internal data structure do you see the _ form.

That said, it might be okay to change this behavior. I need to give a little more thought to the implications.

[Doug Hellmann (doug-hellmann)]

I don't think that behavior really something we want to change. The library is working as designed. We expect the application owner to change internal references to the old option name when the option is renamed. And we do not expect the application to use options that it doesn't own and might be changed out from under it.

[Kent Wu (mimic1983)]

The reason to propose such change is because:
# It seems like the internal data structure(self._opts) stores options by using 'opt.dest' instead of 'opt.name' while registering options.(https://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py?h=refs/heads/master#n2628)

# The function set_override() is calling private function _get_opt_info() and passing in deprecated name as the 'opt_name'. In this case, the internal container(self._opts) will be used.(https://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py?h=refs/heads/master#n3121) It errors out because 'self._opts' doesn't has the key.(https://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py?h=refs/heads/master#n3144)

The change I proposed might be able to handle such kind of situation without harming the original logic.

[Ben Nemec (bnemec)]

It doesn't matter what the internal data structure of oslo.config looks like though. The opt name is the interface that applications are supposed to use here.

Is there a situation you're seeing where it isn't possible to use the original name?

[Kent Wu (mimic1983)]

It is because the client side is feeding a deprecated name instead of the original name.(i.e. os-identity-api-version) Here's the way to reproduce it on Freezer end.

>>> from freezer.common import config as freezer_config
>>> freezer_config.config()
>>> freezer_config.CONF.set_override('os_auth_ver', '2')

Config option None.os_auth_ver is deprecated. Use option DEFAULT.os-identity-api-version instead.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/debtcollector/removals.py", line 261, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2402, in __inner
    result = f(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2727, in set_override
    opt_info = self._get_opt_info(name, group)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 3115, in _get_opt_info
    return opts[opt_name]
KeyError: 'os-identity-api-version'
>>>

As for the comment above, it seems like a proposed fix on the service side(Freezer) would be the feasible way OR force the client to use non-deprecated opt instead.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/575950
Committed: https://git.openstack.org/cgit/openstack/freezer/commit/?id=c1499e54cdb06dab5e54686035bf56a537208d61
Submitter: Zuul
Branch: master

commit c1499e54cdb06dab5e54686035bf56a537208d61
Author: Kent Wu <email address hidden>
Date: Sat Jun 16 19:09:01 2018 -0600

    Fix KeyError if a deprecated key is given

    It happens when a deprecated key is given(i.e. os-auth-ver). The service shall
    change the reference before calling oslo.config.

    Change-Id: I841f027f90e353b7ea5e1ec059286e8fa9b94f49
    Closes-Bug: #1774722

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/581052

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/581052
Committed: https://git.openstack.org/cgit/openstack/freezer/commit/?id=3f7f3f2d1cf6680c2862ad8d00cc15bbc7c652b0
Submitter: Zuul
Branch: stable/pike

commit 3f7f3f2d1cf6680c2862ad8d00cc15bbc7c652b0
Author: Kent Wu <email address hidden>
Date: Sat Jun 16 19:09:01 2018 -0600

    Fix KeyError if a deprecated key is given

    It happens when a deprecated key is given(i.e. os-auth-ver). The service shall
    change the reference before calling oslo.config.

    Change-Id: I841f027f90e353b7ea5e1ec059286e8fa9b94f49
    Closes-Bug: #1774722
    (cherry picked from commit c1499e54cdb06dab5e54686035bf56a537208d61)

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kent Wu (<email address hidden>) on branch: master
Review: https://review.openstack.org/571900

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/freezer 7.0.0 release.