neutron returns objects other than oslo_config.cfg.Opt instances from list_opts

Bug #1548433 reported by Doug Hellmann
28
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Pavlo Shchelokovskyy
keystoneauth
Invalid
Undecided
Unassigned
neutron
Fix Released
High
Jamie Lennox

Bug Description

The neutron function for listing options for use with the configuration generator returns things that are not compliant with the oslo_config.cfg.Opt class API. At the very least this includes the options from keystoneauth1, but I haven't looked to find if there are others.

We'll work around this for now in the configuration generator code, but in the future we will more strictly enforce the API compliance by refusing to generate a configuration file or by leaving options out of the output.

The change blocked by this issue is: https://review.openstack.org/#/c/282435/5

One failure log showing the issue is: http://logs.openstack.org/35/282435/5/check/gate-tempest-dsvm-neutron-src-oslo.config/77044c6/logs/devstacklog.txt.gz

The neutron code triggering the issue is in: http://git.openstack.org/cgit/openstack/neutron/tree/neutron/opts.py#n279

The best solution would be to fix keystoneauth to support option discovery natively using proper oslo.config Opts.

Changed in keystoneauth:
status: New → Won't Fix
status: Won't Fix → Incomplete
Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

Keystoneauth will not natively depend on oslo_config for everything (unless there is a drastic change) due to the fact that it has made efforts to not be locked into depending on oslo.* libraries. This was a key factor in it's original design to ensure we didn't get into any weird dependency issues and allowing it to be part of swift/swiftclient code bases.

Keeping this design in mind, what would be the correct approach? I would prefer to see oslo_config understand a clear "template" that can be converted into a proper opt for the generator or similar.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

The function that's being called here is an API hook defined by oslo.config. The best solution is to figure out some way to return the right kind of object from that function, with the understanding that at the point it is being called oslo.config is available because it's oslo.config that's calling the function.

Revision history for this message
Jamie Lennox (jamielennox) wrote :

Keystoneauth decided not to have a dependency on oslo_config. This is a choice and won't be reverted.

A keystoneauth.Opt is NOT a substitute for an oslo_config opt. You can generate an oslo_config opt from a keystoneauth Opt using the function [1] available as keystoneauth1.loading.get_auth_plugin_conf_options. This function can take either a string name or an existing plugin (yes, they probably should have been two functions).

This is a neutron bug for returning keystoneauth Opts instead of oslo_config Opts.

[1] https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/loading/conf.py#L45
[2] https://github.com/openstack/keystoneauth/blob/master/keystoneauth1/loading/__init__.py#L25

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/292640

Changed in neutron:
assignee: nobody → Jamie Lennox (jamielennox)
status: New → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :
Changed in nova:
status: New → Fix Released
Matt Riedemann (mriedem)
Changed in nova:
assignee: nobody → Pavlo Shchelokovskyy (pshchelo)
importance: Undecided → High
Henry Gessau (gessau)
Changed in neutron:
importance: Undecided → Low
tags: added: config mitaka-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/292640
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c3db0707eff70f381913643891ba4e148977407d
Submitter: Jenkins
Branch: master

commit c3db0707eff70f381913643891ba4e148977407d
Author: Jamie Lennox <email address hidden>
Date: Tue Mar 15 10:05:29 2016 +1100

    Return oslo_config Opts to config generator

    We shouldn't be returning keystoneauth Opts to the oslo_config
    generator. Whilst it mostly works these objects are not interchangable
    and it can result in problems. You can see this by entries such as:

      # Warning: Failed to format sample for tenant_name
      # isinstance() arg 2 must be a class, type, or tuple of classes
        and types

    in the currently generated config files.

    Keystoneauth provides a function that returns oslo_config options so
    fetch, process and return those instead.

    Change-Id: Ie3fad2381467b19189cbb332c41cea8b6cf6e264
    Closes-Bug: #1548433

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/302728

Revision history for this message
Ihar Hrachyshka (ihar-hrachyshka) wrote :

I don't think releasing code with this bug was very wise. This bug should have been considered a blocker for Mitaka final release. I moved the bug to High to raise visibility.

Changed in neutron:
importance: Low → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/302728
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9679285f547b301a511e02ebc763b5406fb03ffc
Submitter: Jenkins
Branch: stable/mitaka

commit 9679285f547b301a511e02ebc763b5406fb03ffc
Author: Jamie Lennox <email address hidden>
Date: Tue Mar 15 10:05:29 2016 +1100

    Return oslo_config Opts to config generator

    We shouldn't be returning keystoneauth Opts to the oslo_config
    generator. Whilst it mostly works these objects are not interchangable
    and it can result in problems. You can see this by entries such as:

      # Warning: Failed to format sample for tenant_name
      # isinstance() arg 2 must be a class, type, or tuple of classes
        and types

    in the currently generated config files.

    Keystoneauth provides a function that returns oslo_config options so
    fetch, process and return those instead.

    Change-Id: Ie3fad2381467b19189cbb332c41cea8b6cf6e264
    Closes-Bug: #1548433
    (cherry picked from commit c3db0707eff70f381913643891ba4e148977407d)

tags: added: in-stable-mitaka
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 8.1.0

This issue was fixed in the openstack/neutron 8.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/314250

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)
Download full text (36.9 KiB)

Reviewed: https://review.openstack.org/314250
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3bf73801df169de40d365e6240e045266392ca63
Submitter: Jenkins
Branch: master

commit a323769143001d67fd1b3b4ba294e59accd09e0e
Author: Ryan Moats <email address hidden>
Date: Tue Oct 20 15:51:37 2015 +0000

    Revert "Improve performance of ensure_namespace"

    This reverts commit 81823e86328e62850a89aef9f0b609bfc0a6dacd.

    Unneeded optimization: this commit only improves execution
    time on the order of milliseconds, which is less than 1% of
    the total router update execution time at the network node.

    This also

    Closes-bug: #1574881

    Change-Id: Icbcdf4725ba7d2e743bb6761c9799ae436bd953b

commit 7fcf0253246832300f13b0aa4cea397215700572
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Apr 21 07:05:16 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I9e930750dde85a9beb0b6f85eeea8a0962d3e020

commit 643b4431606421b09d05eb0ccde130adbf88df64
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Apr 19 06:52:48 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I52d7460b3265b5460b9089e1cc58624640dc7230

commit 1ffea42ccdc14b7a6162c1895bd8f2aae48d5dae
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Apr 18 15:03:30 2016 +0000

    Updated from global requirements

    Change-Id: Icb27945b3f222af1d9ab2b62bf2169d82b6ae26c

commit b970ed5bdac60c0fa227f2fddaa9b842ba4f51a7
Author: Kevin Benton <email address hidden>
Date: Fri Apr 8 17:52:14 2016 -0700

    Clear DVR MAC on last agent deletion from host

    Once all agents are deleted from a host, the DVR MAC generated
    for that host should be deleted as well to prevent a buildup of
    pointless flows generated in the OVS agent for hosts that don't
    exist.

    Closes-Bug: #1568206
    Change-Id: I51e736aa0431980a595ecf810f148ca62d990d20
    (cherry picked from commit 92527c2de2afaf4862fddc101143e4d02858924d)

commit eee9e58ed258a48c69effef121f55fdaa5b68bd6
Author: Mike Bayer <email address hidden>
Date: Tue Feb 9 13:10:57 2016 -0500

    Add an option for WSGI pool size

    Neutron currently hardcodes the number of
    greenlets used to process requests in a process to 1000.
    As detailed in
    http://lists.openstack.org/pipermail/openstack-dev/2015-December/082717.html

    this can cause requests to wait within one process
    for available database connection while other processes
    remain available.

    By adding a wsgi_default_pool_size option functionally
    identical to that of Nova, we can lower the number of
    greenlets per process to be more in line with a typical
    max database connection pool size.

    DocImpact: a previously unused configuration value
               wsgi_default_pool_size is now used to a...

tags: added: neutron-proactive-backport-potential
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 9.0.0.0b1

This issue was fixed in the openstack/neutron 9.0.0.0b1 development milestone.

Changed in keystoneauth:
status: Incomplete → Invalid
tags: removed: neutron-proactive-backport-potential
tags: removed: mitaka-backport-potential
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.