oslo.db should provide a way to reset the engine facade for testing

[Sean Dague]

When attempting to write general purpose tests in the embedded unit / functional test suites for Nova, we run into an issue that once the engine facade is initialized, there is no ability to reset it. That means all future tests are using the sqlite connection built by our main Database fixture.

This prohibits using the MySQLOportunistic test mixin in a generic way (outside the very specific database migration low level test). There are a number of bugs that require MySQL behavior to be reproduced, so we'd like to have a way to reset() engine facade connections in these cases.

[Roman Podoliaka (rpodolyaka)]

Patch proposed: https://review.openstack.org/343126

[Mike Bayer (zzzeek)]

I implemented this in an ad-hoc way for Neutron at https://review.openstack.org/#/c/332476/ which I was planning on working into new fixtures inside of oslo.db itself. Like Roman's patch, the need for a new fixture in test_base is evident. I was hoping the fixture could roll in the needs of Neutron, which does not use the global "enginefacade" but makes their own _TransactionContextManager as well as some other more esoteric setups like that of gnocchi, which uses a _TransactionContextManager-per-API instance (e.g. it refers to a "self"). In order for it to integrate well with the provisioning system, there is flexible setup scheme that allows use of create_all() vs. migrations (SQLite DBs should be built using create_all, mysql/postgresql migrations) as well as a place for Neutron to put their "delete from all tables" teardown scheme.

[Mike Bayer (zzzeek)]

okey doke, so I have something coming along that so far I'm testing a lot towards Neutron, since they have fixture use that hugs closely to what's in oslo.db. I'm developing the APIs here against oslo.db and implementing how neutron can use it at the same time, as a means of optimizing how the API is organized. So now we come to Nova, and I can see Nova has a nice simple system now, which is great, in the form of the nova.tests.fixtures.Database fixture. So I'm probably going to have to get discussion going on the ML about this, but for now I'm assuming the goal is, get Database to also work against mysql, Postgresql, etc., include systems for building up the database schema, and then do the right thing when we plug in MySQLOpportunisticTestCase and PsqlOpportunisticTestCase, right? Because the clever trick that Database is using right now with sqlite_connection.iterdump() isn't going to work w/ the other DBs. oslo.db's fixture has a whole way of doing all this using testresources but safe to assume I can propose changes to Database? Or is there something already in the works and you really really just need the one enginefacade method "enginefacade.please_connect_to_this_other_db_instead()" ? (the patch has that also).

[Mike Bayer (zzzeek)]

per chat with sdague on IRC, the mysql tests they're looking for are themselves not a priority for N, and changing fixtures.Database is also a non-starter for N. But because the tests aren't needed urgently, we don't need a short patch for this and I can hopefully attempt to propose a generalized solution in fixtures.Database that will be targeted for O or later.

[Roman Podoliaka (rpodolyaka)]

https://review.openstack.org/#/c/351411/ is merged

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.db 4.15.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by ChangBo Guo(gcb) (<email address hidden>) on branch: master
Review: https://review.openstack.org/345601
Reason: Please restore it if you want to continue working on this.