Stable branch gate tests are broken because of wrong microversion in the tests

[Goutham Pacha Ravi]

With the separation of manila-tempest-plugin in the Queens release, we finally have "branchless" tempest running against master and stable branches. This is only for stable/queens; The branchless "manila-tempest-plugin" is not installed in stable/pike or stable/ocata or stable/newton; and the in-tree manila_tempest_tests package is installed instead which provides a stable plugin for those branches.

This means we can no longer have a stable tempest configuration either. We will need to maintain an override in the CI hooks in a stable fashion via manila.

This particular breakage occurred because tempest.share.max_api_microversion parameter is altered, and newer micro versions that are not available in stable branches force the manila API to return HTTP 406, resulting in gratuitous test failures.

Sample failure:

Patch: https://review.openstack.org/579273
Job: manila-tempest-minimal-dsvm-dummy / gate
Log link: http://logs.openstack.org/73/579273/1/gate/manila-tempest-minimal-dsvm-dummy/d66b39f/

Console logs and tempest log from the test failures are attached in case they disappear from the log server.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/582696

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/582697

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/582696
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=2f42c984e5ec0633910156718aeca4634a284cde
Submitter: Zuul
Branch: master

commit 2f42c984e5ec0633910156718aeca4634a284cde
Author: Goutham Pacha Ravi <email address hidden>
Date: Fri Jul 13 15:31:46 2018 -0700

    Allow setting test API microversions in gate tests

    Add MANILA_TEMPEST_MIN_API_MICROVERSION and
    MANILA_TEMPEST_MAX_API_MICROVERSION to the ci
    post_test_hook. These options default to
    the values specified in the api_version_request.py
    file.

    Also add the API Version History to the api-ref.

    Change-Id: I21b3a705986216afb0ce73ca1f92571dda757408
    Closes-Bug: #1781671

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/582697
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=a2e040dc2afc8365a34dd986bf069abe42cd2bef
Submitter: Zuul
Branch: stable/queens

commit a2e040dc2afc8365a34dd986bf069abe42cd2bef
Author: Goutham Pacha Ravi <email address hidden>
Date: Fri Jul 13 15:31:46 2018 -0700

    Allow setting test API microversions in gate tests

    Add MANILA_TEMPEST_MIN_API_MICROVERSION and
    MANILA_TEMPEST_MAX_API_MICROVERSION to the ci
    post_test_hook. These options default to
    the values specified in the api_version_request.py
    file.

    Also add the API Version History to the api-ref.

    Change-Id: I21b3a705986216afb0ce73ca1f92571dda757408
    Closes-Bug: #1781671

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/583649

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/583750

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/583649
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=ed95238da9fca293165a5025f486f37c0be3149c
Submitter: Zuul
Branch: master

commit ed95238da9fca293165a5025f486f37c0be3149c
Author: Goutham Pacha Ravi <email address hidden>
Date: Wed Jul 18 08:43:15 2018 -0700

    [CI] Don't set test config for API microversions if master

    If the version's being bumped on manila,
    the newer API changes cause test failures with
    manila-tempest-plugin.

    We could carefully craft a
    manila-tempest-plugin changeset that fixes
    any incongruous test assumptions and make the
    manila change depend on the tempest changeset.
    However, that may make developing the API harder
    than it is today.

    Change-Id: Ie97fb49b38b565dcc57332db8b42605216372aa8
    Related-Bug: #1781671

[OpenStack Infra (hudson-openstack)]

Change abandoned by Goutham Pacha Ravi (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/583750
Reason: Tom said:
> stable/* branches will never be master so there is no need for this backport.

Agree with Tom, this won't ever affect stable/queens - but we'll have this review for inquisitive minds if they go tinkering with this code path again.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 6.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 7.0.0.0b3 development milestone.