[FT] Test "test_show" failing randomly

Bug #1962608 reported by Rodolfo Alonso
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Lajos Katona
Changed in neutron:
importance: Undecided → Medium
status: New → Confirmed
Changed in neutron:
assignee: nobody → Lajos Katona (lajos-katona)
Revision history for this message
Lajos Katona (lajos-katona) wrote :

Seems it happens quite often:
$ logsearch log --project openstack/neutron --branch master --job neutron-functional-with-uwsgi --result FAILURE --limit 50 "takes 1 positional argument but 3 were given"
...
...
Builds with matching logs 9/50:
+----------------------------------+---------------------+----------+-----------------------------------+
| uuid | finished | pipeline | review |
+----------------------------------+---------------------+----------+-----------------------------------+
| b6f69beafaa34f3eb49ea2b159fcfc71 | 2022-03-03T18:16:48 | check | https://review.opendev.org/831830 |
| cdcc0e246f1941dba5e1123570838b92 | 2022-03-02T16:27:06 | check | https://review.opendev.org/831214 |
| 18db40500ad4415a8d2092516d49686f | 2022-03-01T18:01:10 | check | https://review.opendev.org/831409 |
| 5cb1c021879e4e96bd83c288bd07a7cb | 2022-02-28T07:40:21 | check | https://review.opendev.org/829161 |
| 52dbd0b330b1496bbc4c01384c8389b2 | 2022-02-27T04:10:00 | check | https://review.opendev.org/823000 |
| 3d2563909dd74b5b9698a6d4f3f1cedd | 2022-02-24T12:38:19 | gate | https://review.opendev.org/827302 |
| efe0312d2fac4cec85afa1190f54dc32 | 2022-02-23T12:06:29 | gate | https://review.opendev.org/830509 |
| ab4e8bead0a9401ca1448bcff768f5c8 | 2022-02-23T02:04:26 | check | https://review.opendev.org/816802 |
| f2021614b5214207ac5b101999d1786e | 2022-02-22T11:51:12 | gate | https://review.opendev.org/828022 |
+----------------------------------+---------------------+----------+-----------------------------------+

Revision history for this message
Lajos Katona (lajos-katona) wrote :

The exception raised (TypeError: __init__() takes 1 positional argument but 3 were given ) coming from here: https://github.com/svinota/pyroute2/blob/0.6.7/pyroute2.core/pr2modules/netlink/exceptions.py#L61-L67

The exception NetlinkDumpInterrupted was introduced by [1] & [2].

The exception NetlinkDumpInterrupted can be handled in [3], in my env it is quite frequent, so have to think about how to handle it.

[1]: https://github.com/svinota/pyroute2/commit/4a0c42f8a6b4668957bc231ff90a5016c94f3aa4
[2]: https://github.com/svinota/pyroute2/commit/25266ab1689488d96c6697c4dbe2cca1038c559d
[3]: https://opendev.org/openstack/neutron/src/branch/master/neutron/privileged/agent/linux/ip_lib.py#L564-L575

Revision history for this message
Lajos Katona (lajos-katona) wrote :

I commented on original github issue:
https://github.com/svinota/pyroute2/issues/874#issuecomment-1062815225 let's see if this can work in pyroute2

Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Very good catch!

I didn't reproduce that locally and I was trying to dig into this code.

"requirements" was bumped to pyroute v0.6.7 on 2022-02-25 [1].

[1]https://review.opendev.org/c/openstack/requirements/+/830301

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/833015

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833015
Committed: https://opendev.org/openstack/neutron/commit/74a9e832d790f659eb10b22ba0428b60733d384e
Submitter: "Zuul (22348)"
Branch: master

commit 74a9e832d790f659eb10b22ba0428b60733d384e
Author: elajkat <email address hidden>
Date: Tue Mar 8 17:44:11 2022 +0100

    Add retry for privsep get_link_devices

    pyroute 0.6.6 introduced a new exception NetlinkDumpInterrupted which
    is raised when NLM_F_DUMP_INTR is set in the flags during dump of
    devices.
    The suggestion from pyroute developers is to retry in case of this
    exception (see [1]).

    [1]: https://github.com/svinota/pyroute2/issues/874#issuecomment-1063139555

    Closes-Bug: #1962608

    Change-Id: Ie195ad596fd148708fc30946bde964d52444afee

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/837681

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/837681
Committed: https://opendev.org/openstack/neutron/commit/541d1fc9e62df9f15fe003735c63f98a0a0432c3
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 541d1fc9e62df9f15fe003735c63f98a0a0432c3
Author: elajkat <email address hidden>
Date: Tue Mar 8 17:44:11 2022 +0100

    Add retry for privsep get_link_devices

    pyroute 0.6.6 introduced a new exception NetlinkDumpInterrupted which
    is raised when NLM_F_DUMP_INTR is set in the flags during dump of
    devices.
    The suggestion from pyroute developers is to retry in case of this
    exception (see [1]).

    [1]: https://github.com/svinota/pyroute2/issues/874#issuecomment-1063139555

    Closes-Bug: #1962608

    Change-Id: Ie195ad596fd148708fc30946bde964d52444afee
    (cherry picked from commit 74a9e832d790f659eb10b22ba0428b60733d384e)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/844364

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/844366

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/844364
Committed: https://opendev.org/openstack/neutron/commit/ea85053a87196c3d6477277b111eee66f57a4ea9
Submitter: "Zuul (22348)"
Branch: stable/xena

commit ea85053a87196c3d6477277b111eee66f57a4ea9
Author: elajkat <email address hidden>
Date: Tue Mar 8 17:44:11 2022 +0100

    Add retry for privsep get_link_devices

    pyroute 0.6.6 introduced a new exception NetlinkDumpInterrupted which
    is raised when NLM_F_DUMP_INTR is set in the flags during dump of
    devices.
    The suggestion from pyroute developers is to retry in case of this
    exception (see [1]).

    [1]: https://github.com/svinota/pyroute2/issues/874#issuecomment-1063139555

    Closes-Bug: #1962608

    Depends-On: https://review.opendev.org/c/openstack/requirements/+/845268
    Change-Id: Ie195ad596fd148708fc30946bde964d52444afee
    (cherry picked from commit 74a9e832d790f659eb10b22ba0428b60733d384e)
    (cherry picked from commit 541d1fc9e62df9f15fe003735c63f98a0a0432c3)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/844366
Committed: https://opendev.org/openstack/neutron/commit/7773f7c2c1d6cf6703a2afe97139c5dfd1fd959e
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 7773f7c2c1d6cf6703a2afe97139c5dfd1fd959e
Author: elajkat <email address hidden>
Date: Tue Mar 8 17:44:11 2022 +0100

    Add retry for privsep get_link_devices

    pyroute 0.6.6 introduced a new exception NetlinkDumpInterrupted which
    is raised when NLM_F_DUMP_INTR is set in the flags during dump of
    devices.
    The suggestion from pyroute developers is to retry in case of this
    exception (see [1]).

    [1]: https://github.com/svinota/pyroute2/issues/874#issuecomment-1063139555

    Closes-Bug: #1962608

    Depends-On: https://review.opendev.org/c/openstack/requirements/+/845268
    Change-Id: Ie195ad596fd148708fc30946bde964d52444afee
    (cherry picked from commit 74a9e832d790f659eb10b22ba0428b60733d384e)
    (cherry picked from commit 541d1fc9e62df9f15fe003735c63f98a0a0432c3)
    (cherry picked from commit ea85053a87196c3d6477277b111eee66f57a4ea9)
    Conflicts: neutron/privileged/agent/linux/ip_lib.py
            requirements.txt

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.5.0

This issue was fixed in the openstack/neutron 18.5.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.4.0

This issue was fixed in the openstack/neutron 19.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.2.0

This issue was fixed in the openstack/neutron 20.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 21.0.0.0rc1

This issue was fixed in the openstack/neutron 21.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.