Numerous config options ignored due to CONF used in import context

[Matthew Booth]

In general[1] it is incorrect to use the value of a config variable at import time, because although the config variable may have been registered, its value will not have been loaded. The result will always be the default value, regardless of the contents of the relevant config file.

I did a quick scan of Nova, and found the following instances of config variables being used in import context:

nova/api/openstack/common.py:limited()
nova/api/openstack/common.py:get_limit_and_marker()
nova/compute/manager.py:_heal_instance_info_cache()
nova/compute/manager.py:_poll_shelved_instances()
nova/compute/manager.py:_poll_bandwidth_usage()
nova/compute/manager.py:_poll_volume_usage()
nova/compute/manager.py:_sync_power_states()
nova/compute/manager.py:_cleanup_running_deleted_instances()
nova/compute/manager.py:_run_image_cache_manager_pass()
nova/compute/manager.py:_run_pending_deletes()
nova/network/manager.py:_periodic_update_dns()
nova/scheduler/manager.py:_run_periodic_tasks()

Consequently, it appears that the given values of the following config variables are being ignored:

osapi_max_limit
heal_instance_info_cache_interval
shelved_poll_interval
bandwidth_poll_interval
volume_usage_poll_interval
sync_power_state_interval
running_deleted_instance_poll_interval
image_cache_manager_interval
instance_delete_interval
dns_update_periodic_interval
scheduler_driver_task_period

[1] This doesn't apply to drivers, which are loaded dynamically after the config has been loaded. However, relying on that seems even nastier.

[Sean Dague (sdague)]

This seems like somethign that should get addressed sooner rather than later

[Boden R (boden)]

All of the CONF usages outlined in the bug description are in fact referenced after the configuration is loaded and therefore all conf values are honored -- there is no functional deficiency here.

However as noted this is bad practice and we likely need a better way to handle these cases.

Lowing importance based on my statements above.

[Boden R (boden)]

Moving to low based on https://wiki.openstack.org/wiki/BugTriage there are no functional deficiencies here. Removing myself as the assignee as there are more important bugs to address right now.

[Ganesh (ganeshna)]

I would like to work on this bug. Can someone please give a correct expected usage example for the config values, so that I can start making the changes ?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/241361

[Stephen Finucane (stephenfinucane)]

Some more information:

A variable used as a function parameter results becomes immutable: changes to the variable after the function has been registered do not get reflected in the function definition. For example:

    >>> class Config(object):
    ... x = 5
    >>> config = Config()
    >>>
    >>> def func(param=config.x):
    ... print(param)
    >>> func()
    5
    >>> config.x = 9
    >>> func()
    5

Currently, this mistake when configuration settings are passed as arguments to functions and decorators. Should these settings change, the updated values will not be reflected in the functions/decorators. However, this is not an issue for methods or constructors as these are only evaluated when an instance of a class is created. Correct this oversight by ensuring these settings are set at runtime in affected functions.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/241361
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a9716c880e4dd6ff2e8e6a15405d90d1402c816e
Submitter: Jenkins
Branch: master

commit a9716c880e4dd6ff2e8e6a15405d90d1402c816e
Author: Stephen Finucane <email address hidden>
Date: Tue Nov 3 17:18:02 2015 +0000

    Rework 'limited' and 'get_limit_and_marker'

    The 'max_limit' parameter in each of these functions are only used
    in one test. Remove these. In addition, make consistent use of the
    other helper functions within these functions.

    Partial-bug: #1354403
    Change-Id: Ifab24647396a868bde123623e39bbba3bd501dfa

[Stephen Finucane (stephenfinucane)]

I have some of this done but there is yet more to do

[Maciej Szankin (mszankin)]

Tagging as New. It is old and requires to be verified.

[Sean Dague (sdague)]

If anyone is interested in keeping this bug alive, can they provide a script that can be run on the Nova code to ensure that we know when it's done?

[Launchpad Janitor (janitor)]

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]