test_models_sync may not always detect if a model is not included in head.py

[Henry Gessau]

Change https://review.openstack.org/212213 added some models but they were not added to head.py. This should have been detected by test_models_sync in the functional job, but it is currently not being detected. I tried the test_models_sync locally and it correctly fails.

[Henry Gessau (gessau)]

Ann, can you look into this? I tried looking at the logs briefly but I am not sure there is enough information there.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/271865

[Akihiro Motoki (amotoki)]

VPNaaS test_models_sync in gate-neutron-vpnaas-dsvm-functional is failing due to neutron model mismatch.
For example, http://logs.openstack.org/09/261109/3/check/gate-neutron-vpnaas-dsvm-functional/a26256d/
I think it is related.

bug 1537578 is related. I think VPNaaS test should not fail too. bug 1537578 is filed to tackle this point.

[Akihiro Motoki (amotoki)]

@Henry, could you share a way to reproduce this bug locally?
I tried it but all related test was skipped.... I am missing something.

VPNaaS test is failing due to test_models_sync now. I wonder what is different between neutron functional (constraint) and VPNaaS functional (non-constraint) tests.

----
{2} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_external_tables_not_changed ... SKIPPED: backend 'mysql' unavailable
{3} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_branches ... SKIPPED: backend 'mysql' unavailable
{2} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_has_offline_migrations_pending_contract_scripts ... SKIPPED: backend 'mysql' unavailable
{2} neutron.tests.functional.db.test_migrations.TestModelsMigrationsPsql.test_models_sync ... SKIPPED: backend 'postgresql' unavailable
{0} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_has_offline_migrations_all_heads_upgraded ... SKIPPED: backend 'mysql' unavailable
{3} neutron.tests.functional.db.test_migrations.TestWalkMigrations.test_no_downgrade [0.026441s] ... ok
{1} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_check_mysql_engine ... SKIPPED: backend 'mysql' unavailable
{1} neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_models_sync ... SKIPPED: backend 'mysql' unavailable

[Ann Taraday (akamyshnikova)]

I will take a look into it ASAP.

@amotoki seems that MySQL-python and Psycopg2 are not installed in your test running env.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Akihiro Motoki (<email address hidden>) on branch: master
Review: https://review.openstack.org/271865
Reason: The jobs for https://review.openstack.org/#/c/271773/ have not completed yet, but the functional tests succeeded. This patch is no longer needed.

[Ann Taraday (akamyshnikova)]

Seems that tests passed because we have this module imported in another functional test - http://paste.openstack.org/show/484863/

[Henry Gessau (gessau)]

Thanks Ann. That also explains why it fails when I run test_models_sync individually (not together with all the other functional tests). So this will be fixed by https://review.openstack.org/271918

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/271918
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d9693d787812347133e9ff765428389a2c332cdf
Submitter: Jenkins
Branch: master

commit d9693d787812347133e9ff765428389a2c332cdf
Author: Akihiro Motoki <email address hidden>
Date: Mon Jan 25 16:46:08 2016 +0900

    Add dns_db to models/head.py

    commit 93ac8b3a3318c0b762d399fa78b8ce981d44b3d9 adds dns_db
    but it forgot to add it to models/heads.
    As a result, db migration model sync check is failing.

    Closes-Bug: #1537578
    Closes-Bug: #1537510

    Note that it was not detected in our neutron functional tests.
    It is being investigated in bug 1537510.

    Change-Id: Ica15afb1fd2335e5fe686ec36b05513a46427bcd

[Carl Baldwin (carl-baldwin)]

I'm a little bit confused about why https://review.openstack.org/271918 closed this bug? Isn't this bug to investigate why we needed to file https://review.openstack.org/271918 in the first place and hopefully how the problem could've been avoided with the original merge of https://review.openstack.org/#/c/212213/?

[Henry Gessau (gessau)]

Carl, you are right. That did not fix this "bug". This "bug" is explained by comment #7.

To elaborate...

When the functional tests are run under tox, the test discovery mechanism ends up importing all the test modules to search for test cases. One of the test modules (neutron/tests/unit/extensions/test_l3.py in fact) imports dns_db.py, which causes the test_models_sync test case to succeed because now all models are loaded.

If you tried to run test_models_sync directly (without test discovery) it would fail because dns_db.py was not imported in head.py.

That being said, I don't know how to make this kind of issue detectable. Test discovery importing all test modules has bitten us before. Maybe for this test_models_sync case we could separate it from the main tox run.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/272677

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/272677
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0a96b7a57e3c27d24e13a1b31405ce5eb237415f
Submitter: Jenkins
Branch: master

commit 0a96b7a57e3c27d24e13a1b31405ce5eb237415f
Author: Miguel Lavalle <email address hidden>
Date: Tue Jan 26 11:58:06 2016 -0600

    Update alembic migration documentation

    Alembic migration documentation for developers is updated to explain the need
    to update neutron/db/migration/models/head.py when adding models to the
    database

    Change-Id: I2894058269cbabfd0cbfa83c7f076701e2087a11
    Partial-Bug: #1537510

[Miguel Lavalle (minsel)]

Per conversation with Henry Gessau, lowering importnace of this bug, since now we have documented the need to update neutron/db/migration/models/head.py when adding models

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b3 development milestone.

[Henry Gessau (gessau)]

The importance of this bug is so low I am tempted to mark it as won't fix.