Tempest tests fail on the gate with an inconsistent tempest version

[Goutham Pacha Ravi]

Manila pins the version of tempest it tests on the gate. This is antithetical to the tempest plugin mechanism. So, our gate breaks when some other test plugin relies on code from tempest that is not part of the commit manila pins.

Example Failure log:

http://logs.openstack.org/46/465846/8/check/gate-manila-tempest-minimal-dsvm-lvm-ubuntu-xenial/6fbb5e7/

2017-05-26 11:42:19.393559 | 2017-05-26 11:42:19.393 | all-plugin runtests: PYTHONHASHSEED='1357686361'
2017-05-26 11:42:19.395131 | 2017-05-26 11:42:19.394 | all-plugin runtests: commands[0] | find . -type f -name *.pyc -delete
2017-05-26 11:42:19.474173 | 2017-05-26 11:42:19.473 | all-plugin runtests: commands[1] | tempest run --regex (^manila_tempest_tests.tests)(?=.*\[.*\bbackend\b.*\]) --concurrency=8
2017-05-26 11:42:23.700027 | 2017-05-26 11:42:23.699 | running=OS_STDOUT_CAPTUR File "/usr/local/lib/python2.7/dist-packages/testrepository/commands/__init__.py", line 143, in execute
2017-05-26 11:42:23.702289 | 2017-05-26 11:42:23.701 | result = self.run()
2017-05-26 11:42:23.703977 | 2017-05-26 11:42:23.703 | File "/usr/local/lib/python2.7/dist-packages/testrepository/commands/run.py", line 213, in run
2017-05-26 11:42:23.705693 | 2017-05-26 11:42:23.705 | return self._run_tests(cmd)
2017-05-26 11:42:23.707803 | 2017-05-26 11:42:23.707 | File "/usr/local/lib/python2.7/dist-packages/testrepository/commands/run.py", line 345, in _run_tests
2017-05-26 11:42:23.709695 | 2017-05-26 11:42:23.709 | cmd.setUp()
2017-05-26 11:42:23.711661 | 2017-05-26 11:42:23.711 | File "/usr/local/lib/python2.7/dist-packages/testrepository/testcommand.py", line 234, in setUp
2017-05-26 11:42:23.713704 | 2017-05-26 11:42:23.713 | self.test_ids = self.list_tests()
2017-05-26 11:42:23.715344 | 2017-05-26 11:42:23.714 | File "/usr/local/lib/python2.7/dist-packages/testrepository/testcommand.py", line 312, in list_tests
2017-05-26 11:42:23.717175 | 2017-05-26 11:42:23.716 | % (run_proc.returncode))
2017-05-26 11:42:23.719247 | 2017-05-26 11:42:23.718 | E=${
2017-05-26 11:42:23.721149 | 2017-05-26 11:42:23.720 | OSNon-zero exit code (2) from test listing.
2017-05-26 11:42:23.722960 | 2017-05-26 11:42:23.722 | _STDOUT_CAPTURE:-1} \
2017-05-26 11:42:23.725060 | 2017-05-26 11:42:23.724 | OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
2017-05-26 11:42:23.727255 | 2017-05-26 11:42:23.726 | OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
2017-05-26 11:42:23.729350 | 2017-05-26 11:42:23.728 | OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
2017-05-26 11:42:23.731634 | 2017-05-26 11:42:23.730 | ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list
2017-05-26 11:42:23.733818 | 2017-05-26 11:42:23.733 | --- import errors ---
2017-05-26 11:42:23.736747 | 2017-05-26 11:42:23.735 | Failed to import test module: keystone_tempest_plugin.tests.api.identity.v3.test_mapping_rules
2017-05-26 11:42:23.738733 | 2017-05-26 11:42:23.738 | Traceback (most recent call last):
2017-05-26 11:42:23.740666 | 2017-05-26 11:42:23.740 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 456, in _find_test_path
2017-05-26 11:42:23.742369 | 2017-05-26 11:42:23.741 | module = self._get_module_from_name(name)
2017-05-26 11:42:23.744267 | 2017-05-26 11:42:23.743 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 395, in _get_module_from_name
2017-05-26 11:42:23.746005 | 2017-05-26 11:42:23.745 | __import__(name)
2017-05-26 11:42:23.748176 | 2017-05-26 11:42:23.747 | File "/opt/stack/new/keystone/keystone_tempest_plugin/tests/api/identity/v3/test_mapping_rules.py", line 23, in <module>
2017-05-26 11:42:23.750503 | 2017-05-26 11:42:23.750 | class MappingRulesTest(base.BaseIdentityTest):
2017-05-26 11:42:23.752700 | 2017-05-26 11:42:23.752 | File "/opt/stack/new/keystone/keystone_tempest_plugin/tests/api/identity/v3/test_mapping_rules.py", line 50, in MappingRulesTest
2017-05-26 11:42:23.755092 | 2017-05-26 11:42:23.754 | @decorators.attr(type=['negative'])
2017-05-26 11:42:23.757504 | 2017-05-26 11:42:23.757 | AttributeError: 'module' object has no attribute 'attr'
2017-05-26 11:42:23.759812 | 2017-05-26 11:42:23.759 |
2017-05-26 11:42:23.762165 | 2017-05-26 11:42:23.761 | Failed to import test module: keystone_tempest_plugin.tests.api.identity.v3.test_service_providers
2017-05-26 11:42:23.764414 | 2017-05-26 11:42:23.764 | Traceback (most recent call last):
2017-05-26 11:42:23.766586 | 2017-05-26 11:42:23.766 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 456, in _find_test_path
2017-05-26 11:42:23.768972 | 2017-05-26 11:42:23.768 | module = self._get_module_from_name(name)
2017-05-26 11:42:23.772018 | 2017-05-26 11:42:23.771 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 395, in _get_module_from_name
2017-05-26 11:42:23.774663 | 2017-05-26 11:42:23.774 | __import__(name)
2017-05-26 11:42:23.776984 | 2017-05-26 11:42:23.776 | File "/opt/stack/new/keystone/keystone_tempest_plugin/tests/api/identity/v3/test_service_providers.py", line 25, in <module>
2017-05-26 11:42:23.779315 | 2017-05-26 11:42:23.778 | class ServiceProvidersTest(base.BaseIdentityTest):
2017-05-26 11:42:23.781857 | 2017-05-26 11:42:23.781 | File "/opt/stack/new/keystone/keystone_tempest_plugin/tests/api/identity/v3/test_service_providers.py", line 75, in ServiceProvidersTest
2017-05-26 11:42:23.783968 | 2017-05-26 11:42:23.783 | @decorators.attr(type=['negative'])
2017-05-26 11:42:23.786519 | 2017-05-26 11:42:23.786 | AttributeError: 'module' object has no attribute 'attr'
2017-05-26 11:42:23.788634 | 2017-05-26 11:42:23.788 | > /usr/local/lib/python2.7/dist-packages/testrepository/testcommand.py(312)list_tests()
2017-05-26 11:42:23.790679 | 2017-05-26 11:42:23.790 | -> % (run_proc.returncode))
2017-05-26 11:42:23.792797 | 2017-05-26 11:42:23.792 | (Pdb)
2017-05-26 11:42:23.794747 | 2017-05-26 11:42:23.794 | The test run didn't actually run any tests
2017-05-26 11:42:23.796712 | 2017-05-26 11:42:23.796 | ERROR: InvocationError: '/opt/stack/new/tempest/.tox/all-plugin/bin/tempest run --regex (^manila_tempest_tests.tests)(?=.*\\[.*\\bbackend\\b.*\\]) --concurrency=8'
2017-05-26 11:42:23.799040 | 2017-05-26 11:42:23.798 | ___________________________________ summary ____________________________________
2017-05-26 11:42:23.800773 | 2017-05-26 11:42:23.800 | ERROR: all-plugin: commands failed

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/468434
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=76e569a6167c5648864e9cdbf006e274616a8e17
Submitter: Jenkins
Branch: master

commit 76e569a6167c5648864e9cdbf006e274616a8e17
Author: Goutham Pacha Ravi <email address hidden>
Date: Fri May 26 10:20:43 2017 -0400

    CI: Update tempest commit

    Manila pins the version of tempest it tests on the gate.
    This is antithetical to the tempest plugin mechanism. So,
    our gate breaks when some plugin relies on code from tempest
    that is not part of the commit manila pins in the CI.

    While we are working towards a permanent fix to this problem,
    let's update our pin as a workaround.

    Closes-Bug: #1693832

    Change-Id: Ide131229efdea5a48dc389449602f2b0abb699f7

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/468545

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 5.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tom Barron (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/468545
Reason: No activity since June last year; feel free to restore, rebase, etc. if appropriate.