ovn-agent: exception when deleing agent

Bug #2091071 reported by Kien Nguyen
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
New
High
Kien Nguyen

Bug Description

Call neutron API to delete ovn agent, it returns 204 but the agent isn't deleted. Check neutron-server.log find out there is an exception when deleting agent:

```
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: TypeError: unhashable type: 'list'
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event Traceback (most recent call last):
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/ovsdbapp/event.py", line 177, in notify_loop
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event match.run(event, row, updates)
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 330, in run
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event n_agent.AgentCache().delete([row.external_ids['delete_agent']])
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py", line 282, in delete
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event del self.agents[agent_id]
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event TypeError: unhashable type: 'list'
2024-12-04 09:21:15.303 838 ERROR ovsdbapp.event
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: TypeError: unhashable type: 'list'
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event Traceback (most recent call last):
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/ovsdbapp/event.py", line 177, in notify_loop
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event match.run(event, row, updates)
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py", line 330, in run
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event n_agent.AgentCache().delete([row.external_ids['delete_agent']])
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event File "/var/lib/kolla/venv/lib/python3.10/site-packages/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py", line 282, in delete
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event del self.agents[agent_id]
2024-12-04 09:21:15.303 799 ERROR ovsdbapp.event TypeError: unhashable type: 'list'
```

Tags: ovn
Revision history for this message
Brian Haley (brian-haley) wrote :

An fyi that this was broken in 2023.2 with https://review.opendev.org/c/openstack/neutron/+/883607 so any fix will need to be cherry-picked back to there one it merges on master branch.

tags: added: ovn
Changed in neutron:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Jimin Shin (jimin3-shin) wrote :

I'm the reporter of https://bugs.launchpad.net/neutron/+bug/2091055 , can I fix this bug?

Revision history for this message
Brian Haley (brian-haley) wrote :

There was already a patch submitted, but I had asked Kien to submit a bug to track it since we need to backport.

https://review.opendev.org/c/openstack/neutron/+/936988

Miguel Lavalle (minsel)
Changed in neutron:
assignee: nobody → Kien Nguyen (kiennt2609)
Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/936988
Committed: https://opendev.org/openstack/neutron/commit/ce6e2d87c52a11ec76a0f6fb7deb25a3dc26cf86
Submitter: "Zuul (22348)"
Branch: master

commit ce6e2d87c52a11ec76a0f6fb7deb25a3dc26cf86
Author: Kien Nguyen Tuan <email address hidden>
Date: Wed Dec 4 11:10:47 2024 +0700

    Use the correct input for OVN agent deletion

    The key [1] is a list which is unhashable, therefore,
    it can not be passed as dict key [2]. It causes
    TypeError: unhashable type: 'list' exception.

    [1] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L332
    [2] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L281

    Closes-bug: #2091071
    Change-Id: I064d9b3e6cb72562a16030e1b31de45ddc8f487c

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Patch needs testing to prevent regressions.

Changed in neutron:
status: Fix Released → New
Revision history for this message
Jimin Shin (jimin3-shin) wrote :

Is this bug fix going to be cherry-picked to 2024.1 branch?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/2024.2)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/937093
Committed: https://opendev.org/openstack/neutron/commit/9527821e124db50f93f9a9c4ffcb303a72b659a4
Submitter: "Zuul (22348)"
Branch: stable/2024.2

commit 9527821e124db50f93f9a9c4ffcb303a72b659a4
Author: Kien Nguyen Tuan <email address hidden>
Date: Wed Dec 4 11:10:47 2024 +0700

    Use the correct input for OVN agent deletion

    The key [1] is a list which is unhashable, therefore,
    it can not be passed as dict key [2]. It causes
    TypeError: unhashable type: 'list' exception.

    [1] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L332
    [2] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L281

    Closes-bug: #2091071
    Change-Id: I064d9b3e6cb72562a16030e1b31de45ddc8f487c
    (cherry picked from commit ce6e2d87c52a11ec76a0f6fb7deb25a3dc26cf86)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/2023.2)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/937091
Committed: https://opendev.org/openstack/neutron/commit/81e38bc8972c7dbc7f15957aed91c5581fde4079
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 81e38bc8972c7dbc7f15957aed91c5581fde4079
Author: Kien Nguyen Tuan <email address hidden>
Date: Wed Dec 4 11:10:47 2024 +0700

    Use the correct input for OVN agent deletion

    The key [1] is a list which is unhashable, therefore,
    it can not be passed as dict key [2]. It causes
    TypeError: unhashable type: 'list' exception.

    [1] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L332
    [2] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L281

    Closes-bug: #2091071
    Change-Id: I064d9b3e6cb72562a16030e1b31de45ddc8f487c
    (cherry picked from commit ce6e2d87c52a11ec76a0f6fb7deb25a3dc26cf86)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/2024.1)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/937092
Committed: https://opendev.org/openstack/neutron/commit/75c5ef31915449cc478a884c6a1331dc2ba4e6c0
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 75c5ef31915449cc478a884c6a1331dc2ba4e6c0
Author: Kien Nguyen Tuan <email address hidden>
Date: Wed Dec 4 11:10:47 2024 +0700

    Use the correct input for OVN agent deletion

    The key [1] is a list which is unhashable, therefore,
    it can not be passed as dict key [2]. It causes
    TypeError: unhashable type: 'list' exception.

    [1] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L332
    [2] https://github.com/openstack/neutron/blob/24e70ea166b517df78ca720b279e2520720de1e7/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py#L281

    Closes-bug: #2091071
    Change-Id: I064d9b3e6cb72562a16030e1b31de45ddc8f487c
    (cherry picked from commit ce6e2d87c52a11ec76a0f6fb7deb25a3dc26cf86)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.