octavia

interface name collision in the amphora

Bug #2017894 reported by Gregory Thiemonge
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
octavia
Fix Released
High
Gregory Thiemonge

Bug Description

On octavia master (also on all stable branches):

after adding/deleting members from different networks, adding a new member may trigger an interface name collision in the amphora.

Steps to reproduce:
- create a LB, a listener and a pool
- add member1 (subnet1/network1) -> eth2 is created in the ns
- add member2 (subnet2/network2) -> eth3 is created
- delete member1 (subnet1/network1) -> eth2 is deleted
- add member3 (subnet3/network3) -> Plugged interface ens8 will become eth3 in the namespace amphora-haproxy

it fails because eth3 already exists

Revision history for this message
Gregory Thiemonge (gthiemonge) wrote :
Download full text (11.7 KiB)

Apr 27 07:34:35 amphora-bc294a34-e0fe-4fd1-a5ad-d3930b4e71b5 gunicorn.gunicorn.error: [659] POST /1.0/plug/network
Apr 27 07:34:35 amphora-bc294a34-e0fe-4fd1-a5ad-d3930b4e71b5 amphora-agent: 2023-04-27 07:34:35.853 659 INFO octavia.amphorae.backends.agent.api_server.plug [-] Plugged interface ens8 will become eth3 in the namespace amphora-haproxy
Apr 27 07:34:35 amphora-bc294a34-e0fe-4fd1-a5ad-d3930b4e71b5 amphora-agent: 2023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server [-] Exception on /1.0/plug/network [POST]: pyroute2.netlink.exceptions.NetlinkError: (17, 'File exists')#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server Traceback (most recent call last):#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server response = self.full_dispatch_request()#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server rv = self.handle_user_exception(e)#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server rv = self.dispatch_request()#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/octavia/amphorae/backends/agent/api_server/server.py", line 216, in plug_network#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server return self._plug.plug_network(port_info['mac_address'],#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/octavia/amphorae/backends/agent/api_server/plug.py", line 237, in plug_network#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server ipr.link('set', index=idx,#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server File "/opt/amphora-agent-venv/lib64/python3.9/site-packages/pyroute2/iproute/linux.py", line 1672, in link#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends.agent.api_server.server ret = self.nlm_request(msg, msg_type=msg_type, msg_flags=msg_flags)#0122023-04-27 07:34:35.882 659 ERROR octavia.amphorae.backends...

Revision history for this message
Gregory Thiemonge (gthiemonge) wrote :

Adding reproducer

Changed in octavia:
assignee: nobody → Gregory Thiemonge (gthiemonge)
importance: Undecided → High
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/octavia/+/881719

Changed in octavia:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (master)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/881719
Committed: https://opendev.org/openstack/octavia/commit/c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2
Submitter: "Zuul (22348)"
Branch: master

commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1

Changed in octavia:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/2023.1)

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/octavia/+/885572

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/zed)

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/octavia/+/885573

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/octavia/+/885574

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/octavia/+/885715

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/octavia/+/885716

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/octavia/+/885724

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/octavia/+/885726

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/octavia/+/885727

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885572
Committed: https://opendev.org/openstack/octavia/commit/1b17529b134456980e302239225514f8f876aa36
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 1b17529b134456980e302239225514f8f876aa36
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885724
Committed: https://opendev.org/openstack/octavia/commit/fe0b3fa4a9517f8772ec8e6bd6591555c5bd3ded
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit fe0b3fa4a9517f8772ec8e6bd6591555c5bd3ded
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Conflicts:
        octavia/amphorae/backends/agent/api_server/plug.py
        octavia/tests/functional/amphorae/backend/agent/api_server/test_server.py
        octavia/tests/unit/amphorae/backends/agent/api_server/test_plug.py

    Note: Victoria and older
        Added a missing constant in octavia/common/constants.py

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)
    (cherry picked from commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042)
    (cherry picked from commit 202d1dc4002f9996eb8047613757a572bc4366ed)
    (cherry picked from commit 4260a127045b01fa58cf49b33883fae9106daccb)
    (cherry picked from commit f204b6f2c5e7ea6958123cd1f01f5e26271335c4)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/train)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885727
Committed: https://opendev.org/openstack/octavia/commit/355f6b1c1bf2e524cde403011b882dc79f91039a
Submitter: "Zuul (22348)"
Branch: stable/train

commit 355f6b1c1bf2e524cde403011b882dc79f91039a
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Conflicts:
        octavia/amphorae/backends/agent/api_server/plug.py
        octavia/tests/functional/amphorae/backend/agent/api_server/test_server.py
        octavia/tests/unit/amphorae/backends/agent/api_server/test_plug.py
        octavia/common/constants.py

    Note: Victoria and older
        Added a missing constant in octavia/common/constants.py
    Note: Train
        Removed f-strings

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)
    (cherry picked from commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042)
    (cherry picked from commit 202d1dc4002f9996eb8047613757a572bc4366ed)
    (cherry picked from commit 4260a127045b01fa58cf49b33883fae9106daccb)
    (cherry picked from commit f204b6f2c5e7ea6958123cd1f01f5e26271335c4)
    (cherry picked from commit fe0b3fa4a9517f8772ec8e6bd6591555c5bd3ded)
    (cherry picked from commit 7d9faf452c6375b9539fb71eb3ee628368d6e5a4)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/zed)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885573
Committed: https://opendev.org/openstack/octavia/commit/05b3343473fa3d2df14eb6d4e44a01c9e83d3042
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)

tags: added: in-stable-zed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885715
Committed: https://opendev.org/openstack/octavia/commit/4260a127045b01fa58cf49b33883fae9106daccb
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 4260a127045b01fa58cf49b33883fae9106daccb
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)
    (cherry picked from commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042)
    (cherry picked from commit 202d1dc4002f9996eb8047613757a572bc4366ed)

tags: added: in-stable-xena
tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885726
Committed: https://opendev.org/openstack/octavia/commit/7d9faf452c6375b9539fb71eb3ee628368d6e5a4
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 7d9faf452c6375b9539fb71eb3ee628368d6e5a4
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Conflicts:
        octavia/amphorae/backends/agent/api_server/plug.py
        octavia/tests/functional/amphorae/backend/agent/api_server/test_server.py
        octavia/tests/unit/amphorae/backends/agent/api_server/test_plug.py
        octavia/common/constants.py

    Note: Victoria and older
        Added a missing constant in octavia/common/constants.py

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)
    (cherry picked from commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042)
    (cherry picked from commit 202d1dc4002f9996eb8047613757a572bc4366ed)
    (cherry picked from commit 4260a127045b01fa58cf49b33883fae9106daccb)
    (cherry picked from commit f204b6f2c5e7ea6958123cd1f01f5e26271335c4)
    (cherry picked from commit fe0b3fa4a9517f8772ec8e6bd6591555c5bd3ded)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/885716
Committed: https://opendev.org/openstack/octavia/commit/f204b6f2c5e7ea6958123cd1f01f5e26271335c4
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit f204b6f2c5e7ea6958123cd1f01f5e26271335c4
Author: Gregory Thiemonge <email address hidden>
Date: Thu Apr 27 06:00:52 2023 -0400

    Avoid interface name collisions in the amphora

    After unpluging a network, the octavia-agent may try to plug a new
    network to a new interface while another interface with the same name
    already exists.

    Closes-Bug: #2017894

    Change-Id: I7f150c2c1c3da706055aba5e02b5ad7c032806f1
    (cherry picked from commit c4dd87f9b119ccaef2d01ae35c9fa0ffbe5f23b2)
    (cherry picked from commit 1b17529b134456980e302239225514f8f876aa36)
    (cherry picked from commit 05b3343473fa3d2df14eb6d4e44a01c9e83d3042)
    (cherry picked from commit 202d1dc4002f9996eb8047613757a572bc4366ed)
    (cherry picked from commit 4260a127045b01fa58cf49b33883fae9106daccb)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 10.1.0

This issue was fixed in the openstack/octavia 10.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 11.0.1

This issue was fixed in the openstack/octavia 11.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 13.0.0.0rc1

This issue was fixed in the openstack/octavia 13.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia train-eol

This issue was fixed in the openstack/octavia train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia ussuri-eol

This issue was fixed in the openstack/octavia ussuri-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia victoria-eom

This issue was fixed in the openstack/octavia victoria-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia wallaby-eom

This issue was fixed in the openstack/octavia wallaby-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia xena-eom

This issue was fixed in the openstack/octavia xena-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.