When two ovs ports contain same external_ids:face-id field, ovs agent might fail finding correct port.

Bug #1370914 reported by John Schwarz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Undecided
John Schwarz

Bug Description

As the title says, if there are 2 different ovs ports with the same external_ids:iface-id field (which is the port_id), when at least one of them is managed by the ovs agent, it might fail finding the correct one if they are not connected to the same bridge.

Steps to reproduce:
1. Create a router with an internal port to some Neutron network
2. Find the port in 'ovs-vsctl show'
3. Use the following command to find the port_id in ovs: sudo ovs-vsctl --columns=external_ids list Interface <port_name>
4. Use the following commands to create a new port with the same field in a new bridge:
 sudo ovs-vsctl add-br a
 sudo ip link add dummy12312312 type dummy
 sudo ovs-vsctl add-port br-a dummy12312312
 sudo ovs-vsctl set Interface dummy12312312 external_ids:iface-id="<port_id>" # port_id was obtained in point 3.
5. Restart the ovs agent.

At this point the ovs agent's log should show "Port: dummy12312312 is on br-a, not on br-int".

Expected result: ovs agent should know to iterate though the options and find the correct port in the correct bridge.

John Schwarz (jschwarz)
Changed in neutron:
assignee: nobody → John Schwarz (jschwarz)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/122357

Changed in neutron:
status: New → In Progress
Revision history for this message
Assaf Muller (amuller) wrote :

It should be noted that John isn't reproducing this bug for fun. This was discovered during the addition of integrating testing for L3 HA, where we simulate multi-node testing on a single box by bringing up multiple L3 agents. HA routers create the same port on multiple L3 / OVS agents pairs. Thus, we have the same port on different bridges managed by different OVS agents. This patch attempts to solve an OVS agent interacting with ports on a bridge it doesn't manage.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/122357
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e90d6d6c833db51964037afd1c4aa1f521ee380a
Submitter: Jenkins
Branch: master

commit e90d6d6c833db51964037afd1c4aa1f521ee380a
Author: John Schwarz <email address hidden>
Date: Thu Sep 18 11:24:53 2014 +0300

    Iterate over same port_id if more than one exists

    In certain cases where multiple ports can have the same
    external_ids:iface_id property, the ovs agent will arbitrarily choose
    one and ignore the rest. In case the chosen port isn't on the
    integration bridge the ovs agent is managing, an error is returned to
    the calling function. This is faulty since one of the other ports may
    belong to the correct bridge and it should be chosen instead.

    This is interesting for future L3 HA integration tests, where 2
    different instances of l3 agents are needed to run on the same machine.
    In this case, both agents will register ports which have the same
    iface_id property, but obviously only one of the ports is relevant for
    each agent.

    Closes-bug: #1370914
    Related-bug: #1374947
    Change-Id: I05d70417e0f78ae51a9ce377fc93a3f12046b313

Changed in neutron:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (feature/lbaasv2)

Fix proposed to branch: feature/lbaasv2
Review: https://review.openstack.org/130864

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (feature/lbaasv2)
Download full text (72.6 KiB)

Reviewed: https://review.openstack.org/130864
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c089154a94e5872efc95eab33d3d0c9de8619fe4
Submitter: Jenkins
Branch: feature/lbaasv2

commit 62588957fbeccfb4f80eaa72bef2b86b6f08dcf8
Author: Kevin Benton <email address hidden>
Date: Wed Oct 22 13:04:03 2014 -0700

    Big Switch: Switch to TLSv1 in server manager

    Switch to TLSv1 for the connections to the backend
    controllers. The default SSLv3 is no longer considered
    secure.

    TLSv1 was chosen over .1 or .2 because the .1 and .2 weren't
    added until python 2.7.9 so TLSv1 is the only compatible option
    for py26.

    Closes-Bug: #1384487
    Change-Id: I68bd72fc4d90a102003d9ce48c47a4a6a3dd6e03

commit 17204e8f02fdad046dabdb8b31397289d72c877b
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Oct 22 06:20:15 2014 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I58db0476c810aa901463b07c42182eef0adb5114

commit d712663b99520e6d26269b0ca193527603178742
Author: Carl Baldwin <email address hidden>
Date: Mon Oct 20 21:48:42 2014 +0000

    Move disabling of metadata and ipv6_ra to _destroy_router_namespace

    I noticed that disable_ipv6_ra is called from the wrong place and that
    in some cases it was called with a bogus router_id because the code
    made an incorrect assumption about the context. In other case, it was
    never called because _destroy_router_namespace was being called
    directly. This patch moves the disabling of metadata and ipv6_ra in
    to _destroy_router_namespace to ensure they get called correctly and
    avoid duplication.

    Change-Id: Ia76a5ff4200df072b60481f2ee49286b78ece6c4
    Closes-Bug: #1383495

commit f82a5117f6f484a649eadff4b0e6be9a5a4d18bb
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Oct 21 12:11:19 2014 +0000

    Updated from global requirements

    Change-Id: Idcbd730f5c781d21ea75e7bfb15959c8f517980f

commit be6bd82d43fbcb8d1512d8eb5b7a106332364c31
Author: Angus Lees <email address hidden>
Date: Mon Aug 25 12:14:29 2014 +1000

    Remove duplicate import of constants module

    .. and enable corresponding pylint check now the only offending instance
    is fixed.

    Change-Id: I35a12ace46c872446b8c87d0aacce45e94d71bae

commit 9902400039018d77aa3034147cfb24ca4b2353f6
Author: rajeev <email address hidden>
Date: Mon Oct 13 16:25:36 2014 -0400

    Fix race condition on processing DVR floating IPs

    Fip namespace and agent gateway port can be shared by multiple dvr routers.
    This change uses a set as the control variable for these shared resources
    and ensures that Test and Set operation on the control variable are
    performed atomically so that race conditions do not occur among
    multiple threads processing floating IPs.
    Limitation: The scope of this change is limited to addressing the race
    condition described in the bug report. It may not address other issues
    such as pre-existing issue wit...

Thierry Carrez (ttx)
Changed in neutron:
milestone: none → kilo-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in neutron:
milestone: kilo-1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.