functional DB tests based on SqlFixture don't actually use non-sqlite DB

Bug #1594898 reported by Mike Bayer
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Mike Bayer

Bug Description

Currently only neutron/tests/functional/db/test_ipam.py seems to use this fixture, however it is not interacting correctly with oslo.db such that it actually uses the engine set up by oslo.

Add a test like this:

diff --git a/neutron/tests/functional/db/test_ipam.py b/neutron/tests/functional/db/test_ipam.py
index 0f28f74..d14bf6e 100644
--- a/neutron/tests/functional/db/test_ipam.py
+++ b/neutron/tests/functional/db/test_ipam.py
@@ -156,8 +156,8 @@ class IpamTestCase(base.BaseTestCase):

 class TestIpamMySql(common_base.MySQLTestCase, IpamTestCase):
- pass
-
+ def test_we_are_on_mysql(self):
+ self.cxt.session.execute("SELECT CURDATE()")

 class TestIpamPsql(common_base.PostgreSQLTestCase, IpamTestCase):
     pass

then run:

[classic@photon2 neutron]$ tox -e functional neutron.tests.functional.db.test_ipam
functional develop-inst-nodeps: /home/classic/dev/redhat/openstack/neutron
functional installed: ( ... output skipped ... )
functional runtests: PYTHONHASHSEED='545881821'
functional runtests: commands[0] | /home/classic/dev/redhat/openstack/neutron/tools/ostestr_compat_shim.sh neutron.tests.functional.db.test_ipam

( ... output skipped ... )

{3} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_fixed_ip [1.510751s] ... ok
{1} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_fixed_ip [1.822431s] ... ok
{2} neutron.tests.functional.db.test_ipam.IpamTestCase.test_allocate_ip_exausted_pool [2.468420s] ... ok
{1} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
{0} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_allocate_ip_exausted_pool [2.873318s] ... ok
{2} neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql [0.993651s] ... FAILED
{0} neutron.tests.functional.db.test_ipam.TestIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
{1} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_fixed_ip [1.133034s] ... ok
{0} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_ip_exausted_pool ... SKIPPED: backend 'postgresql' unavailable
{2} neutron.tests.functional.db.test_ipam.TestPluggableIpamPsql.test_allocate_fixed_ip ... SKIPPED: backend 'postgresql' unavailable
{3} neutron.tests.functional.db.test_ipam.TestPluggableIpamMySql.test_allocate_ip_exausted_pool [2.740086s] ... ok

==============================
Failed 1 tests - output below:
==============================

neutron.tests.functional.db.test_ipam.TestIpamMySql.test_we_are_on_mysql
------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "neutron/tests/functional/db/test_ipam.py", line 160, in test_we_are_on_mysql
        self.cxt.session.execute("SELECT CURDATE()")
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
        bind, close_with_result=True).execute(clause, params or {})
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
        return meth(self, multiparams, params)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
        return connection._execute_clauseelement(self, multiparams, params)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
        compiled_sql, distilled_params
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
        context)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
        util.raise_from_cause(newraise, exc_info)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
        reraise(type(exception), exception, tb=exc_tb, cause=cause)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
        context)
      File "/home/classic/dev/redhat/openstack/neutron/.tox/functional/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
        cursor.execute(statement, parameters)
    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such function: CURDATE [SQL: u'SELECT CURDATE()']

At the end there, that's a SQLite error. You're not supposed to get those in the MySQL test suite :).

The problem is that the SqlFixture is calling upon neutron.db.api.get_engine() but this is in no way associated with the context that oslo.db creates within the MySQLOpportunisticFixture approach. As neutron is using enginefacade now we need to swap in the facade that's specific to oslo_db.sqlalchemy.test_base.DbFixture and make sure everything is linked up.

Note that this problem does not impact the alembic migration tests, as that test suite does its own set up of alembic fixtures.

I'm working on a reorg of the test fixtures here so this can work, as we will need these fixtures to be effective for the upcoming CIDR stored functions / triggers to be tested.

Mike Bayer (zzzeek)
Changed in neutron:
assignee: nobody → Mike Bayer (zzzeek)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/332476

Changed in neutron:
status: New → In Progress
tags: added: db
tags: added: functional-tests
Changed in neutron:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/332476
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=546ba377db81e7ba984b63942f6bbb974a653e3c
Submitter: Jenkins
Branch: master

commit 546ba377db81e7ba984b63942f6bbb974a653e3c
Author: Mike Bayer <email address hidden>
Date: Tue Jun 21 16:52:11 2016 -0400

    Restore MySQL and Postgresql functional testing

    The oslo.db opportunistic test fixtures were not being
    used effectively and the MySQL / PG databases were not
    being used. This patch restores working patterns against
    oslo.db. Additionally, the migration level tests have also
    been updated to make use of oslo.db provisioning functionality
    and unused methods have been removed.

    The current approach makes use of oslo.db provisioning constructs
    directly, as well as some semi-private attribute access within
    oslo.db enginefacade, in order to work around some issues
    that have arisen in oslo.db's test_base.
    A new release of oslo.db will provide
    public API points to resolve pending issues, and to
    allow neutron's use cases here which will
    also be of general applicability to openstack projects.

    Closes-bug: #1594898

    Change-Id: Ie27cf174fa24c2f479af47335d9ae139fb7d159a

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/336663

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/336663
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=92934ddb91c900f7a927bf064f4e9aa60c7cc058
Submitter: Jenkins
Branch: master

commit 92934ddb91c900f7a927bf064f4e9aa60c7cc058
Author: Henry Gessau <email address hidden>
Date: Fri Jul 1 12:50:34 2016 -0400

    Update the template for model sync test docs

    Update the docs to match the changes from
      https://review.openstack.org/332476

    Related-bug: #1594898

    Change-Id: Iefebbac213db043e27b5244b63df1e04ee9cdc4f

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 9.0.0.0b2

This issue was fixed in the openstack/neutron 9.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.