Bridges keep controller settings after reboot

Bug #1712517 reported by Jakub Libosvar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-net-config
Fix Released
High
Jakub Libosvar

Bug Description

Bridge configuration settings consume a fail_mode that is set when node boots. Sometimes, bridge may be used for control plane traffic/management network, such bridges are put to standalone mode. neutron-openvswitch-agent manages such bridges and once it's up and ready, it sets a controller to itself and puts the bridge into secure mode.

From man ovs-vsctl(8): "If the value is standalone, or if neither of these settings is set, ovs-vswitchd will take over responsibility for setting up flows when no message has been received from the controller for three times the inactivity probe interval."

In case node is rebooted, bridge is on boot set to standalone but controller address is persistently stored in ovsdb and once openvswitch-agent starts, it will attempt to communicate via bridge with server. As bridge in standalone mode has access to the controller (ovs-agent), it won't be managed by ovs-vswitchd and won't let ovs-agent communicate with server. Hence agent won't get information about bridges settings and that creates a chicken-egg problem.

The solution would be to delete controller along with setting bridge to standalone.

Revision history for this message
Jakub Libosvar (libosvar) wrote :
Changed in os-net-config:
status: New → In Progress
assignee: nobody → Jakub Libosvar (libosvar)
Revision history for this message
Jakub Libosvar (libosvar) wrote :

Backport to Pike requested: https://review.openstack.org/#/c/496707/

Changed in os-net-config:
status: In Progress → Fix Released
Revision history for this message
Ben Nemec (bnemec) wrote :

Apparently we don't have stable releases defined on os-net-config, so all of the backports will have to be tracked against this bug and we'll need to ignore the bug status.

Changed in os-net-config:
status: Fix Released → Triaged
importance: Undecided → High
Brent Eagles (beagles)
Changed in os-net-config:
status: Triaged → Fix Committed
tags: added: ocata-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-net-config (stable/pike)

Reviewed: https://review.openstack.org/496707
Committed: https://git.openstack.org/cgit/openstack/os-net-config/commit/?id=77fe5922bd7ca5c2a199d383b03537c08edfcc1c
Submitter: Jenkins
Branch: stable/pike

commit 77fe5922bd7ca5c2a199d383b03537c08edfcc1c
Author: Jakub Libosvar <email address hidden>
Date: Mon Aug 21 16:21:12 2017 +0000

    Delete controller for standalone OVS bridges

    The patch adds an OVS extra parameter to delete controller for bridges
    configured with standalone fail mode. By default, bridges are created
    without having an openflow controllers. If node is restarted, the bridge
    is set to standalone mode but if a service managing the bridge sets a
    controller, it will remain in the ovsdb.

    As ovs-vswitchd sets the bridge behavior to normal MAC learning switch
    only if bridge in standalone mode can't communicate with its controller,
    leaving controller defined can cause node outage when bridge is used as
    management network. In such case controller service, like
    neutron-openvswitch-agent, would need to communicate over management
    network but given that bridge is in standalone mode but communicates
    with controller, management network won't be reachable. This creates a
    chicken-egg problem.

    By removing controller by default, ovs-vswitchd implements a normal
    action rule to the standalone bridge and service can use the bridge as
    management network and eventually set the brdige to secure and set the
    flows manually.

    See opened Bugzilla for more information:
    https://bugzilla.redhat.com/show_bug.cgi?id=1473763

    Closes-bug: #1712517

    Change-Id: Iad48312667834ea8f5c7145595ae89cb5159b36d
    (cherry picked from commit f8d76d2cdebfa0d06233a59a8f6539207c5b5a4e)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-net-config 7.3.0

This issue was fixed in the openstack/os-net-config 7.3.0 release.

Changed in os-net-config:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.