tox molecule|linters jobs are timing out on tripleo-quickstart-extras

Bug #2000622 reported by chandan kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned

Bug Description

https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-linters&project=openstack/tripleo-quickstart-extras and https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-molecule&project=openstack/tripleo-quickstart-extras are timing out from 27th Dec.

tox -e linters are finishing successfully but ansible shell is not existing.
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_826/868677/1/check/openstack-tox-molecule/82661ff/job-output.txt
```
 ubuntu-jammy | molecule installed: ansi2html==1.8.0,ansible==4.10.0,ansible-core==2.11.12,arrow==1.2.3,attrs==22.2.0,bcrypt==4.0.1,binaryornot==0.4.4,Cerberus==1.3.4,certifi==2022.12.7,cffi==1.15.1,chardet==5.1.0,charset-normalizer==2.1.1,click==8.1.3,click-completion==0.5.2,click-help-colors==0.9.1,colorama==0.4.6,cookiecutter==2.1.1,coverage==7.0.1,cryptography==38.0.4,distro==1.8.0,docker==6.0.1,exceptiongroup==1.1.0,fasteners==0.18,idna==3.4,iniconfig==1.1.1,Jinja2==3.1.2,jinja2-time==0.2.0,MarkupSafe==2.1.1,mock==4.0.3,molecule==3.0.8,more-itertools==9.0.0,packaging==22.0,paramiko==2.12.0,pathspec==0.10.3,pexpect==4.8.0,pluggy==0.13.1,ptyprocess==0.7.0,py==1.11.0,pycparser==2.21,PyNaCl==1.5.0,pytest==7.2.0,pytest-cov==4.0.0,pytest-html==3.2.0,pytest-metadata==2.0.4,pytest-molecule==1.3.0,pytest-plus==0.4.0,python-dateutil==2.8.2,python-gilt==1.2.3,python-slugify==7.0.0,PyYAML==5.4.1,requests==2.28.1,resolvelib==0.5.4,selinux==0.3.0,sh==1.13.1,shellingham==1.5.0,six==1.16.0,tabulate==0.9.0,text-unidecode==1.3,tomli==2.0.1,tree-format==0.1.2,-e git+file:///dev/null@6d250182278bf686d5120f5c5863d1ee24bccb41#egg=tripleo_quickstart_extras,urllib3==1.26.13,websocket-client==1.4.2,yamllint==1.28.0
2022-12-27 12:10:26.170501 | ubuntu-jammy | ___________________________________ summary ____________________________________
2022-12-27 12:10:26.170508 | ubuntu-jammy | molecule: skipped tests
2022-12-27 12:10:26.170514 | ubuntu-jammy | congratulations :)
2022-12-27 12:31:44.230200 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/zuul/zuul-jobs/playbooks/tox/run.yaml@master]
2022-12-27 12:31:44.236639 | POST-RUN START: [untrusted : opendev.org/openstack/openstack-zuul-jobs/playbooks/tox-molecule/post.yaml@master]
```
from the logs, mol finished at 2022-12-27 12:10:26 and job time out at 2022-12-27 12:31:44.

It seems the tox job is hanging out.
The last passing job has runned on 22nd Dec.[1]
[1]. https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_824/868210/6/check/openstack-tox-molecule/82411c1/job-output.txt

In this month, tox-4 landed. Not sure it might be linked with that.
It is blocking the TQE patches.

Tags: alert ci
Revision history for this message
chandan kumar (chkumar246) wrote :
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-quickstart-extras (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/868741
Committed: https://opendev.org/openstack/tripleo-quickstart-extras/commit/b4669d9f7492a7a9ecf7ea54a328ed9fe146de30
Submitter: "Zuul (22348)"
Branch: master

commit b4669d9f7492a7a9ecf7ea54a328ed9fe146de30
Author: Chandan Kumar <email address hidden>
Date: Wed Dec 28 13:03:19 2022 +0530

    Use tox-4 for linters and Molecules

    It removes following things:
    - skipdist option
    - updates whitelist_externals to allowlist_externals
    - Drops tox-bindep as it does not support tox-4
    - Tox-4 also includes fix for hanging issue:
      https://github.com/tox-dev/tox/issues/2595

    bindep is quite stable and tox-4 is around from long
    time and tox-bindep last got released on jan 2021. [1]
    https://github.com/tox-dev/tox-bindep/releases/tag/0.2.1

    So, We are dropping it.

    Closes-Bug: #2000622

    Signed-off-by: Chandan Kumar <email address hidden>
    Change-Id: Ia7f8a4dd970ac76322adadc0698ac1c14aa87180

Changed in tripleo:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.