OVS agent not reconfigure ext-bridge in case when it was removed and created again by administrator

Bug #1768990 reported by Slawek Kaplonski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Slawek Kaplonski

Bug Description

Neutron OVS agent will configure phys_bridges only during agent start process.
If later someone remove and create such bridge again it will clean all flows from it.
Because such bridges are configured with "fail_mode=secure" that might cause to lost of connection in Neutron control plane as it is quite common deployment practice to use same external bridge for data plane and for control plane.

Changed in neutron:
assignee: nobody → Slawek Kaplonski (slaweq)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/566178

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
Andreas Karis (akaris) wrote :

I'd like to add to the issue description that I don't thinkg that the fail_mode is necessarily the culprit. But if somebody or something (e.g. tripleo) deletes and recreates the bridge, the bridge will lose all settings, such as:
* flows
* virtual patch cable into br-int
* fail_mode settings

Revision history for this message
Slawek Kaplonski (slaweq) wrote :

@Andreas. Bridge with fail_mode="secure" will not set up flows on its own when the controller connection fails and without any flow rules for bridge packets will be dropped. That will cause problems with control plane connections if control plane also uses such external bridge.
Problem here is that "fail_mode" is stored in ovsdb so it is persistent but open flow rules are not persistent.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/566178
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=85b46cd51ebec67d5018d8bef4f796ee5ef4fedf
Submitter: Zuul
Branch: master

commit 85b46cd51ebec67d5018d8bef4f796ee5ef4fedf
Author: Sławek Kapłoński <email address hidden>
Date: Fri May 4 00:12:03 2018 +0200

    Monitor phys_bridges to reconfigured it if created again

    In case when external bridge configured in OVS agent's bridge_mappings
    will be destroyed and created again (for example by running ifup-ovs
    script on Centos) bridge wasn't configured by OVS agent.
    That might cause broken connectivity for OpenStack's dataplane if
    dataplane network also uses same bridge.

    This patch adds additional ovsdb-monitor to monitor if any
    of physical bridges configured in bridge_mappings was created.
    If so, agent will reconfigure it to restore proper openflow rules
    on it.

    Change-Id: I9c0dc587e70327e03be5a64522d0c679665f79bd
    Closes-Bug: #1768990

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/567145

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/567691

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/queens)

Reviewed: https://review.openstack.org/567145
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=62240c10971274c03632ee25b92848f0f138b28e
Submitter: Zuul
Branch: stable/queens

commit 62240c10971274c03632ee25b92848f0f138b28e
Author: Sławek Kapłoński <email address hidden>
Date: Fri May 4 00:12:03 2018 +0200

    Monitor phys_bridges to reconfigured it if created again

    In case when external bridge configured in OVS agent's bridge_mappings
    will be destroyed and created again (for example by running ifup-ovs
    script on Centos) bridge wasn't configured by OVS agent.
    That might cause broken connectivity for OpenStack's dataplane if
    dataplane network also uses same bridge.

    This patch adds additional ovsdb-monitor to monitor if any
    of physical bridges configured in bridge_mappings was created.
    If so, agent will reconfigure it to restore proper openflow rules
    on it.

    Change-Id: I9c0dc587e70327e03be5a64522d0c679665f79bd
    Closes-Bug: #1768990
    (cherry picked from commit 85b46cd51ebec67d5018d8bef4f796ee5ef4fedf)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/pike)

Reviewed: https://review.openstack.org/567691
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b557d4ac6cd51fc3e3bd424b14fb32013a41c557
Submitter: Zuul
Branch: stable/pike

commit b557d4ac6cd51fc3e3bd424b14fb32013a41c557
Author: Sławek Kapłoński <email address hidden>
Date: Fri May 4 00:12:03 2018 +0200

    Monitor phys_bridges to reconfigured it if created again

    In case when external bridge configured in OVS agent's bridge_mappings
    will be destroyed and created again (for example by running ifup-ovs
    script on Centos) bridge wasn't configured by OVS agent.
    That might cause broken connectivity for OpenStack's dataplane if
    dataplane network also uses same bridge.

    This patch adds additional ovsdb-monitor to monitor if any
    of physical bridges configured in bridge_mappings was created.
    If so, agent will reconfigure it to restore proper openflow rules
    on it.

    Change-Id: I9c0dc587e70327e03be5a64522d0c679665f79bd
    Closes-Bug: #1768990
    (cherry picked from commit 85b46cd51ebec67d5018d8bef4f796ee5ef4fedf)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 13.0.0.0b2

This issue was fixed in the openstack/neutron 13.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.0.3

This issue was fixed in the openstack/neutron 12.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 11.0.5

This issue was fixed in the openstack/neutron 11.0.5 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.