stable/ussuri branch tempest jobs failing

Bug #1928880 reported by Goutham Pacha Ravi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Medium
Goutham Pacha Ravi

Bug Description

The traceback in the test jobs is as follows:

Failed to import test module: tempest.api.compute.admin.test_fixed_ips
Traceback (most recent call last):
  File "/usr/lib/python3.6/unittest/loader.py", line 428, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/opt/stack/new/tempest/tempest/api/compute/admin/test_fixed_ips.py", line 16, in <module>
    from tempest.api.compute import base
  File "/opt/stack/new/tempest/tempest/api/compute/base.py", line 30, in <module>
    import tempest.test
  File "/opt/stack/new/tempest/tempest/test.py", line 26, in <module>
    from tempest.common import credentials_factory as credentials
  File "/opt/stack/new/tempest/tempest/common/credentials_factory.py", line 20, in <module>
    from tempest.lib.common import preprov_creds
  File "/opt/stack/new/tempest/tempest/lib/common/preprov_creds.py", line 19, in <module>
    from oslo_utils.secretutils import md5
ImportError: cannot import name 'md5'

The console log file is attached to the bug

Seems to be related to this recent change in tempest: https://github.com/openstack/tempest/commit/56860ccae71114bfb32138c823ea04a50b916aa2

Revision history for this message
Goutham Pacha Ravi (gouthamr) wrote :
Changed in manila:
importance: Undecided → Medium
assignee: nobody → Goutham Pacha Ravi (gouthamr)
milestone: none → xena-1
status: New → Confirmed
Revision history for this message
Goutham Pacha Ravi (gouthamr) wrote :

oslo.utils change: https://github.com/openstack/oslo.utils/commit/603fa500c1a24ad8753b680b8d75468abbd3dd76

Was released as part of oslo.utils 4.9

The test job in stable/ussuri installs oslo.utils===3.41.6 based on the various constraints since it's a single node devstack job, and we're installing manila-tempest-plugin directly on the host. Perhaps we should switch to installing the tempest plugin within a venv?

Revision history for this message
Goutham Pacha Ravi (gouthamr) wrote :

An ussuri job defined in a native zuulv3 format doesn't hit this issue: https://review.opendev.org/c/openstack/manila-tempest-plugin/+/725692

In this job tempest plugin is installed into a virtualenv, which always uses tempest requirements from master:

https://github.com/openstack/devstack/blob/a64c843052529cf5c8ac8cf8eb46c59b103b4b47/lib/tempest#L731-L743

So options here seem to be:

- constrain tempest on the old legacy jobs we have in stable/ussuri to a tag that doesn't need this latest version of oslo.utils;
- replace all the test jobs in stable/ussuri to the zuulv3 native jobs.
- figure out a way to stop installing/using tempest from the global python namespace, and use a tempest virtualenv.

I expect all third party CI jobs on stable branches to be broken as well.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/manila/+/792475
Committed: https://opendev.org/openstack/manila/commit/ed316557e9b7d33277c2b346a7251ef391f771db
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit ed316557e9b7d33277c2b346a7251ef391f771db
Author: Ghanshyam Mann <email address hidden>
Date: Thu May 20 14:15:14 2021 -0500

    Fix CI issues and adjust test jobs

    It is incorrect to install tempest and manila-tempest-plugin
    globally in devstack jobs. So this commit removes the use
    of the test plugin's devstack installation hook, and uses
    a "tempest-venv" that devstack sets up to run the tests.

    To allow zuulv3 style jobs elsewhere [1] use the right
    api versions when testing against this stable branch,
    we enhance the devstack plugin.sh script to set api
    versions as tempest configuration options.

    Alongside these changes, we can perform a much needed
    test job cleanup by eliminating some jobs we don't have
    the bandwidth to maintain:

    - manila-tempest-dsvm-postgres-generic-singlebackend
      Single backend jobs are not very useful - the job
      sets up a single backend in a multibackend style.
      We've dropped this from future branches, and we
      don't expect this job to catch any regressions that
      other jobs will miss.
    - manila-tempest-dsvm-generic-no-share-servers
      This job's been failing; and wasting CI resources.
      We have other DHSS=False reference drivers (CephFS
      native/nfs, ZFSOnLinux, LVM) where we test the
      driver mode sufficiently.
    - manila-tempest-dsvm-glusterfs-nfs-heketi
      and manila-tempest-dsvm-glusterfs-native-heketi
      these were in the experimental queue, unmaintained
    - manila-tempest-dsvm-container-scenario-custom-image
      and manila-tempest-dsvm-generic-scenario-custom-image
      New zuulv3 style jobs in the manila-image-elements
      project replace these jobs [2].

    [1] https://review.opendev.org/c/openstack/manila-tempest-plugin/+/725692
    [2] https://opendev.org/openstack/manila-image-elements/src/commit/9c00e86d17a3487c9467dbedfe09a0d5623974b5/zuul.d/manila-image-elements-jobs.yaml

    Closes-Bug: #1928880
    Depends-On: I2eceb1d5fe42b1b779e115a0144a5d9639d3753f
    Change-Id: I6c7db2286d544ff8156926e07efd57628cfdc9ff
    Signed-off-by: Goutham Pacha Ravi <email address hidden>

tags: added: in-stable-ussuri
Changed in manila:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers