tox testenv not installing requirements properly

Bug #1852356 reported by Brian Rosmaita on 2019-11-13
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Medium
Brian Rosmaita

Bug Description

I've encountered this recently when building new tox environments:

=========================
Failures during discovery
=========================
--- import errors ---
Failed to import test module: glance.tests.unit.api.test_cmd
Traceback (most recent call last):
  File "/home/brosmait/repos/openstack/glance/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/brosmait/repos/openstack/glance/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "/home/brosmait/repos/openstack/glance/glance/tests/unit/api/test_cmd.py", line 20, in <module>
    import glance.cmd.api
  File "/home/brosmait/repos/openstack/glance/glance/cmd/api.py", line 42, in <module>
    from oslo_reports import guru_meditation_report as gmr
ModuleNotFoundError: No module named 'oslo_reports'

Looking at the current tox.ini, the base testenv sets some env vars and then we have:
install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master} {opts} {packages}
deps = -r{toxinidir}/test-requirements.txt

No mention of the requirements at all.

I think we are hitting a bug that Tony Breeds patched in Cinder, Change-Id: I5b1c285680f7c3256a707ea5973573f1176d4a26

Change the testenv like this:

install_command = pip install {opts} {packages}

deps = -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
       -r{toxinidir}/test-requirements.txt
       -r{toxinidir}/requirements.txt

This makes sure that the u-c file is applied to both the test-req and the requirements, which is what we want.

Fix proposed to branch: master
Review: https://review.opendev.org/693951

Changed in glance:
status: Triaged → In Progress
Brian Rosmaita (brian-rosmaita) wrote :

Some evidence that this change is needed is that with it, the lower-constraints job fails:

Collecting psycopg2==2.6.2
  Downloading http://mirror.regionone.fortnebula.opendev.org/pypifiles/packages/7b/a8/dc2d50a6f37c157459cd18bab381c8e6134b9381b50fbe969997b2ae7dbc/psycopg2-2.6.2.tar.gz (376kB)
    ERROR: Command errored out with exit status 1:
     command: /home/zuul/src/opendev.org/openstack/glance/.tox/lower-constraints/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5w8p_8bb/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5w8p_8bb/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-5w8p_8bb/psycopg2/pip-egg-info
         cwd: /tmp/pip-install-5w8p_8bb/psycopg2/
[snip]
    Error: could not determine PostgreSQL version from '10.10'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

This error happens with versions of psycopg < 2.7 (see, for example, [0]). So the fact that we haven't seen it before illustrates the problem with our current tox configuration where the u-c are added as part of the install_command instead of in the deps.

[0] https://help.heroku.com/BFIHK81D/why-do-i-get-error-could-not-determine-postgresql-version-from-10-x-when-building-psycopg2

Reviewed: https://review.opendev.org/693951
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=d2d67ae3a1e7f4e482adf6b2e050f81cf57ec9e7
Submitter: Zuul
Branch: master

commit d2d67ae3a1e7f4e482adf6b2e050f81cf57ec9e7
Author: Brian Rosmaita <email address hidden>
Date: Tue Nov 12 19:30:46 2019 -0500

    Install requirements during the correct tox phase

    Glance isn't explicitly installing requirements, so add requirements
    as tox deps in such a way that they'll be properly constrained by the
    current openstack upper-constraints.

    This change also bumps the version of psycopg to that of the current
    upper-constraints in test-requirements and lower-constraints. (The
    version in our current lower-constraints is subject to the "could not
    determine PostgreSQL version from '10.x'" error, which causes the
    lower-constraints job to fail.)

    Change-Id: I1bb5759e6b1e3f8545ab417617072a61323b59a7
    Closes-bug: #1852356

Changed in glance:
status: In Progress → Fix Released

This issue was fixed in the openstack/glance 20.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers