unittest: NoReverseMatch for test panels sometimes happens

[Akihiro Motoki]

NoReverseMatch error occasionally happens in openstack_dashboard unit tests.
For example, NoReverseMatch: u'plugin_panel' is not a registered namespace inside 'horizon:admin'
This seems to depends on the order of tests executed.

Recent examples are:
http://logs.openstack.org/18/498018/34/check/openstack-tox-py27/e1fc800/job-output.txt.gz#_2018-12-28_11_51_09_741094
http://logs.openstack.org/26/627626/1/check/openstack-tox-py27/0f30a56/job-output.txt.gz#_2018-12-28_11_41_14_703315

Captured example is http://paste.openstack.org/show/739018/

In addition, horizon current unit test does not catch errors in template rendering.
This can be enabled by specifying debug=True in OPTIONS parameter in TEMPLATES.
https://docs.djangoproject.com/en/2.1/topics/templates/#module-django.template.backends.django

[Akihiro Motoki (amotoki)]

Test order can be controlled by passing test modules directly to python manage.py test.

It turns out that the following command line reproduces this bug with 100%.

python manage.py test --settings=openstack_dashboard.test.settings --exclude-tag selenium --exclude-tag integration --verbosity 2 openstack_dashboard.test.unit openstack_dashboard.test.test_plugins openstack_dashboard.dashboards.identity.projects.tests

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/627741

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/627640
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=a69ba853a78090d0e23be2dd5f8f2e5d4e564df4
Submitter: Zuul
Branch: master

commit a69ba853a78090d0e23be2dd5f8f2e5d4e564df4
Author: Akihiro Motoki <email address hidden>
Date: Sun Jan 6 15:05:43 2019 +0900

    UT: Run test_plugins as separate process

    This commit changes to run test_plugins in a separate process.

    test_plugins touches urlpatterns and Site registry during setUp(),
    but the cleanup process in tearDown does not recover urlpatterns correctly.
    After tests under test_plugins run, the urlpatterns continues to have
    entries of test panels and this leads to NoReverseMatch error when
    rendering other panels after that.

    I investigated details in setUp/tearDown of PluginTestCase but failed
    to find a good solution to recover urlpatterns and Horizon Site registry.
    As a workaround, runnig test_plugins in a separate process can avoid
    the issue.

    Closes-Bug: #1809983
    Change-Id: I848f6b341b3f93ed055955b2b12d2497811edc5a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/627741
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=28b457bc627d76031317c733f42b4701d53a3f85
Submitter: Zuul
Branch: master

commit 28b457bc627d76031317c733f42b4701d53a3f85
Author: Akihiro Motoki <email address hidden>
Date: Sat Dec 29 05:40:27 2018 +0900

    UT: Detect template rendering errors

    This commit enables template debug option to detect errors
    in template rendering in unit tests.

    Several errors in template rendering are found by enabling the
    debug option and they are fixed in this commit.

    Related-Bug: #1809983
    Change-Id: I1a2d19f2eae62e16f02d3386abd65527bac4e7a0

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/horizon 15.0.0.0b2 development milestone.