fixtures in neutron.tests.base blow away default database config

[Mike Bayer]

Really trying to narrow this one down fully, and just putting this up because this is as far as I've gotten.

Basically, the lines in neutron/tests/base.py:

  line 159: self.addCleanup(CONF.reset)
  line 182: self.useFixture(self.messaging_conf)

cause cfg.CONF to get totally wiped out in the "database" config. I don't yet understand why this is the case.

if you then run any test that extends BaseTestCase, and then run neutron/tests/unit/test_db_plugin.py -> NeutronDbPluginV2AsMixinTestCase in the same process, these two tests fail:

Traceback (most recent call last):
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/tests/unit/test_db_plugin.py", line 3943, in setUp
    self.plugin = importutils.import_object(DB_PLUGIN_KLASS)
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/openstack/common/importutils.py", line 38, in import_object
    return import_class(import_str)(*args, **kwargs)
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/db/db_base_plugin_v2.py", line 72, in __init__
    db.configure_db()
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/db/api.py", line 45, in configure_db
    register_models()
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/db/api.py", line 68, in register_models
    facade = _create_facade_lazily()
  File "/Users/classic/dev/redhat/openstack/neutron/neutron/db/api.py", line 34, in _create_facade_lazily
    _FACADE = session.EngineFacade.from_config(cfg.CONF, sqlite_fk=True)
  File "/Users/classic/dev/redhat/openstack/neutron/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 977, in from_config
    retry_interval=conf.database.retry_interval)
  File "/Users/classic/dev/redhat/openstack/neutron/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 893, in __init__
    **engine_kwargs)
  File "/Users/classic/dev/redhat/openstack/neutron/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py", line 650, in create_engine
    if "sqlite" in connection_dict.drivername:
AttributeError: 'NoneType' object has no attribute 'drivername'

I'm getting this error running tox on a subset of tests, however it's difficult to reproduce as the subprocesses have to work out just right.

To reproduce, just install nose and do:

.tox/py27/bin/nosetests -v neutron.tests.unit.test_db_plugin:DbModelTestCase neutron.tests.unit.test_db_plugin:NeutronDbPluginV2AsMixinTestCase

That is, DbModelTestCase is a harmless test but because it runs base.BaseTestCase first, cfg.CONF gets blown away.

I don't know what the solution should be here, cfg.CONF shouldn't be reset but I don't know what "messaging_conffixture.ConfFixture" is or how "CONF.reset" was supposed to work as it blows away DB config. The cfg.CONF in the first place seems to get set up via this path:

  <string>(7)exec2()
  /Users/classic/dev/redhat/openstack/neutron/neutron/tests/unit/test_db_plugin.py(26)<module>()
-> from neutron.api import extensions
  /Users/classic/dev/redhat/openstack/neutron/neutron/api/extensions.py(31)<module>()
-> from neutron import manager
  /Users/classic/dev/redhat/openstack/neutron/neutron/manager.py(20)<module>()
-> from neutron.common import rpc as n_rpc
  /Users/classic/dev/redhat/openstack/neutron/neutron/common/rpc.py(22)<module>()
-> from neutron import context
  /Users/classic/dev/redhat/openstack/neutron/neutron/context.py(26)<module>()
-> from neutron import policy
  /Users/classic/dev/redhat/openstack/neutron/neutron/policy.py(55)<module>()
-> cfg.CONF.import_opt('policy_file', 'neutron.common.config')
  /Users/classic/dev/redhat/openstack/neutron/.tox/py27/lib/python2.7/site-packages/oslo/config/cfg.py(1764)import_opt()
-> __import__(module_str)
  /Users/classic/dev/redhat/openstack/neutron/neutron/common/config.py(135)<module>()
-> max_overflow=20, pool_timeout=10)
> /Users/classic/dev/redhat/openstack/neutron/.tox/py27/lib/python2.7/site-packages/oslo/db/options.py(145)set_defaults()
-> conf.register_opts(database_opts, group='database')

e.g. oslo.db set_defaults() sets it up.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/108544

[Kevin Benton (kevinbenton)]

Hi Mike,

I believe the root cause was the model test. Can you check my fix out and confirm?

Thanks

[Mike Bayer (zzzeek)]

hi Kevin -

That particular fix only repairs the issue by accident. It works because it ultimately calls neutron.db.api._create_facade_lazily(), which then establishes the global variable _FACADE so that we no longer need to look at cfg.CONF. If that's to be the mechanics of how this should work, then _create_facade_lazily() should be called up front before any tests run.

The solution should definitely not be part of DbModelTestCase because it is not the cause. If you add a test case just like this:

class AnythingTestCase(base.BaseTestCase):
    def test_nothing(self):
        return

and run that before the NeutronDbPluginV2AsMixinTestCase, same failure. The BaseTestCase fixture destroys configuration that is not necessarily set up yet. So either the suite must guarantee this config is set up by _create_facade_lazily() ultimately being called up front no matter what, or fixing BaseTestCase to do a more appropriate teardown operation.

[Kevin Benton (kevinbenton)]

Wow, my fix was embarrassingly wrong! :-) I think the bug here might actually be with the mixin test depending on something else having setup the DB for it before the config is wiped out.

[Kevin Benton (kevinbenton)]

I pushed up another fix to add a config parse call to the setup of the mixin.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/108544
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d98ca642402bafbaa15ac3df588ed34d58d8456b
Submitter: Jenkins
Branch: master

commit d98ca642402bafbaa15ac3df588ed34d58d8456b
Author: Kevin Benton <email address hidden>
Date: Sat Jul 19 00:54:09 2014 -0700

    Call config_parse in base test setup

    Some of the tests (e.g. NeutronDbPluginV2AsMixinTestCase) do not call
    config_parse so if the database engine is not already been setup by another
    test before the config object is reset on teardown, the database connection
    will fail since the database config is then empty.

    This patch adds a new setup_config method called during the base test case
    setUp method which calls config_parse by default to load the default config.
    Tests that couldn't use the default config were then modified to override the
    setup_config method.

    Some other unit tests were slightly adjusted to pass using the default config.

    Closes-Bug: #1346673
    Change-Id: I3724200fa932115c0a8c264640b1a9dbe431a1cc