High rate of TIMED_OUT in neutron-functional-python27

Bug #1848944 reported by Rodolfo Alonso
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Won't Fix
Critical
Rodolfo Alonso

Bug Description

High rate of TIMED_OUT in neutron-functional-python27, from 18/10/2019 23:48 to 20/20/2019 14:06

http://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?orgId=1&from=1571424870181&to=1571595902672

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/689658

Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)
tags: added: functional-tests gate-failure
Changed in neutron:
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.opendev.org/689957

Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/689957
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=471dc98707795a770114de7fc0d0e2ee704b682d
Submitter: Zuul
Branch: master

commit 471dc98707795a770114de7fc0d0e2ee704b682d
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Oct 22 06:03:16 2019 +0000

    Temporary disable CI job neutron-functional-python27

    Due to the high rate of TIME_OUT errors in the CI and the gate.

    Change-Id: I2f584542efc4bac743e34d6ee28a94aa87c3c4f6
    Related-Bug: #1848944

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/689658
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4f59db2fab33e39f3d101885263e5f6483ad2b94
Submitter: Zuul
Branch: master

commit 4f59db2fab33e39f3d101885263e5f6483ad2b94
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Oct 20 19:16:48 2019 +0000

    Reset timeout exception in DietTestCase when retrying

    In case of fixtures.TimeoutException, DietTestCase retries the
    execution of the test case (introduced in [1]). But the timeout
    fixture was not reset. This can lead to a general execution timeout
    if a test do not exit in a proper time.

    [1] If865c4683645f9bd11f5e1b528bade0547505bfd

    Related-Bug: #1843478
    Partial-Bug: #1848944
    Change-Id: Ib6ef58b5259ea4bf151e866f6244ff87b195b139

tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/705758

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/705759

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/train)

Reviewed: https://review.opendev.org/705758
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9db46c5794493bcd7e768c97458aba105242c75f
Submitter: Zuul
Branch: stable/train

commit 9db46c5794493bcd7e768c97458aba105242c75f
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Oct 20 19:16:48 2019 +0000

    Reset timeout exception in DietTestCase when retrying

    In case of fixtures.TimeoutException, DietTestCase retries the
    execution of the test case (introduced in [1]). But the timeout
    fixture was not reset. This can lead to a general execution timeout
    if a test do not exit in a proper time.

    [1] If865c4683645f9bd11f5e1b528bade0547505bfd

    Related-Bug: #1843478
    Partial-Bug: #1848944
    Change-Id: Ib6ef58b5259ea4bf151e866f6244ff87b195b139
    (cherry picked from commit 4f59db2fab33e39f3d101885263e5f6483ad2b94)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/705759
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4ff4ff4864fb62a0b581a1821e02dec24581d09a
Submitter: Zuul
Branch: stable/stein

commit 4ff4ff4864fb62a0b581a1821e02dec24581d09a
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Oct 20 19:16:48 2019 +0000

    Reset timeout exception in DietTestCase when retrying

    In case of fixtures.TimeoutException, DietTestCase retries the
    execution of the test case (introduced in [1]). But the timeout
    fixture was not reset. This can lead to a general execution timeout
    if a test do not exit in a proper time.

    [1] If865c4683645f9bd11f5e1b528bade0547505bfd

    Related-Bug: #1843478
    Partial-Bug: #1848944
    Change-Id: Ib6ef58b5259ea4bf151e866f6244ff87b195b139
    (cherry picked from commit 4f59db2fab33e39f3d101885263e5f6483ad2b94)

tags: added: in-stable-stein
Revision history for this message
Slawek Kaplonski (slaweq) wrote :

The neutron-functional-python27 job is now gone and we don't have this issue anymore, so I will close this bug now.

Changed in neutron:
status: In Progress → Won't Fix
tags: removed: neutron-proactive-backport-potential
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.