"TestNeutronServer" related tests failing frequently

Bug #1930367 reported by Rodolfo Alonso
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
High
Unassigned

Bug Description

"TestNeutronServer" and those inheriting from this parent class ("TestWsgiServer", "TestRPCServer", "TestPluginWorker") are failing frequently.

Error: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d0d/791365/7/gate/neutron-functional-with-uwsgi/d0d293b/testr_results.html

Snippet: http://paste.openstack.org/show/806208/

Hongbin Lu (hongbin.lu)
Changed in neutron:
importance: Undecided → High
status: New → Confirmed
tags: added: gate-failure
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/793899

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :
Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/791365
Committed: https://opendev.org/openstack/neutron/commit/4de350a54a3e3e59371391a7a022447b49ae508e
Submitter: "Zuul (22348)"
Branch: master

commit 4de350a54a3e3e59371391a7a022447b49ae508e
Author: Slawek Kaplonski <email address hidden>
Date: Fri May 14 08:31:32 2021 +0200

    Switch neutron-rally-task job to ML2/OVS and skip "test_models_sync"

    After switch of the default Neutron backend to the ML2/OVN
    in the Devstack we had 2 rally jobs running actually with the
    same Neutron backend.

    This patch configures explicitly ML2/OVS as a backend in the
    'neutron-rally-task' job so it is like it was before the change in
    the Devstack repo.

    In order to fix the CI, this patch also skips "test_models_sync" test
    cases. Once [1] is fixed and released, we'll be able to create SQL
    tables specifying the "_constraints_included" flag. When creating a
    table, this flag should be set to "False" in order to be able to set
    the "index" and "unique" parameters for each column in the table
    definition, as we do now.

    "test_update_router_admin_state" is excluded from
    "neutron-ovn-tempest-slow" until LP#1890445 is fixed.

    Set "test_restart_wsgi_on_sighup_multiple_workers" as unstable.

    CI job "neutron-ovn-tempest-slow" has been set to non-voting until
    LP#1930402 is fixed.

    Related-Bug: #1929518
    Related-Bug: #1890445
    Related-Bug: #1930402
    Related-Bug: #1930367

    [1]https://github.com/sqlalchemy/alembic/issues/848

    Change-Id: I38ed7931169434fecbbefa3762f51a522ef3c509

tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/793899
Committed: https://opendev.org/openstack/neutron/commit/da760973be09a98a1fc07c519e74a11899cb4aa5
Submitter: "Zuul (22348)"
Branch: master

commit da760973be09a98a1fc07c519e74a11899cb4aa5
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Jun 1 07:44:33 2021 +0000

    Use "multiprocessing.Queue" for "TestNeutronServer" related tests

    Instead of using a file to log the processes status and actions,
    "TestNeutronServer" now uses a "multiprocessing.Queue" that is
    safer than writting a single file accross multiple processes.

    Change-Id: I6d04df180cd9b2d593bb99c8d22a60a3534f22a0
    Closes-Bug: #1930367

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/799149

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/799150

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/799151

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/799191

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/799150
Committed: https://opendev.org/openstack/neutron/commit/a254cdaf0295f159088745793a9bb1c891a43bbc
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit a254cdaf0295f159088745793a9bb1c891a43bbc
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Jun 1 07:44:33 2021 +0000

    Use "multiprocessing.Queue" for "TestNeutronServer" related tests

    Instead of using a file to log the processes status and actions,
    "TestNeutronServer" now uses a "multiprocessing.Queue" that is
    safer than writting a single file accross multiple processes.

    Change-Id: I6d04df180cd9b2d593bb99c8d22a60a3534f22a0
    Closes-Bug: #1930367
    (cherry picked from commit da760973be09a98a1fc07c519e74a11899cb4aa5)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/799149
Committed: https://opendev.org/openstack/neutron/commit/3534dc7aed125e9c42b805e891b12ab63ea751bd
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 3534dc7aed125e9c42b805e891b12ab63ea751bd
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Jun 1 07:44:33 2021 +0000

    Use "multiprocessing.Queue" for "TestNeutronServer" related tests

    Instead of using a file to log the processes status and actions,
    "TestNeutronServer" now uses a "multiprocessing.Queue" that is
    safer than writting a single file accross multiple processes.

    Change-Id: I6d04df180cd9b2d593bb99c8d22a60a3534f22a0
    Closes-Bug: #1930367
    (cherry picked from commit da760973be09a98a1fc07c519e74a11899cb4aa5)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/799151
Committed: https://opendev.org/openstack/neutron/commit/b5097c9fb233753b912ffb405bea96cf10e37ea3
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit b5097c9fb233753b912ffb405bea96cf10e37ea3
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Jun 1 07:44:33 2021 +0000

    Use "multiprocessing.Queue" for "TestNeutronServer" related tests

    Instead of using a file to log the processes status and actions,
    "TestNeutronServer" now uses a "multiprocessing.Queue" that is
    safer than writting a single file accross multiple processes.

    Change-Id: I6d04df180cd9b2d593bb99c8d22a60a3534f22a0
    Closes-Bug: #1930367
    (cherry picked from commit da760973be09a98a1fc07c519e74a11899cb4aa5)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 16.4.0

This issue was fixed in the openstack/neutron 16.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.2.0

This issue was fixed in the openstack/neutron 17.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.1.0

This issue was fixed in the openstack/neutron 18.1.0 release.

Revision history for this message
Slawek Kaplonski (slaweq) wrote : auto-abandon-script

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: Rodolfo Alonso (rodolfo-alonso-hernandez) → nobody
tags: added: timeout-abandon
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/train)

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/799191
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.0.0.0rc1

This issue was fixed in the openstack/neutron 19.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.