Upgrading neutron-openvswitch charm from 16.07 to 16.10 unnecessarily restarts openvswitch-switch

Bug #1712444 reported by Billy Olsen
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Neutron Open vSwitch Charm
Fix Released
Critical
Billy Olsen

Bug Description

Upgrading the neutron-openvswitch charm from 16.07 to 16.10 or higher results in a restart of the openvswitch-switch service, which disrupts the datapath.

The problem is caused by commit 4463c334 [0] in which new lines to the header of /etc/default/openvswitch-switch were introduced. These new lines added the restart_trigger capability but also introduce additional text which will cause the file contents to change, causing a restart (since restarts are guarded by the changed contents of a file).

[0] https://github.com/openstack/charm-neutron-openvswitch/commit/4463c334ca8aaef5aef68175e0caabe516859ec8

Tags: sts
Changed in charm-neutron-openvswitch:
milestone: none → 17.08
assignee: nobody → Billy Olsen (billy-olsen)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-neutron-openvswitch (master)

Fix proposed to branch: master
Review: https://review.openstack.org/496464

Changed in charm-neutron-openvswitch:
status: Triaged → In Progress
Revision history for this message
Billy Olsen (billy-olsen) wrote :

The problem is actually a bit different than previously described. The template change is a problem that will cause a restart of the service, however another other problem is that the 16.10 charms changed from writing the /etc/default/openvswitch-switch problem only when dpdk was enabled, to always writing the /etc/default/openvswitch-switch (regardless of dpdk).

In order to prevent the data plane outage from the openvswitch-switch restart, the charms need to have a similar strategy for when to manage the openvswitch-switch file.

Changed in charm-neutron-openvswitch:
importance: High → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-neutron-openvswitch (master)

Reviewed: https://review.openstack.org/496464
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-openvswitch/commit/?id=e3ec31c91df72cf64f79f14e0223ec61382c0553
Submitter: Jenkins
Branch: master

commit e3ec31c91df72cf64f79f14e0223ec61382c0553
Author: Billy Olsen <email address hidden>
Date: Tue Aug 22 19:19:06 2017 -0700

    Migrate openvswitch-switch file to avoid restarts

    The 16.10 release of the neutron-openvswitch charm changed the
    file management strategy of /etc/default/openvswitch-switch
    config file from managing the file when dpdk is enabled to always
    managing the file. Additionally, the template file was changed
    in the 16.10 release to modify the file header (commit 4463c334).
    These two changes guarantee that the contents of the file will
    change when upgrading the charm.

    The changing file contents causes the openvswitch-switch service
    to be restarted, which in turn causes a data plane outage. This
    commit fixes that by migrating the /etc/default/openvswitch-switch
    to be charm managed without restarting the openvswitch-switch
    service.

    The change will only attempt to migrate versions of the file
    which were created before 16.10 by searching for a marker in the
    rendered version of the file which was added in 16.10.

    Change-Id: Icc0f326991be239b88a57292740473f501181ebb
    Closes-Bug: #1712444

Changed in charm-neutron-openvswitch:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-neutron-openvswitch (stable/17.02)

Fix proposed to branch: stable/17.02
Review: https://review.openstack.org/497695

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-neutron-openvswitch (stable/17.02)

Reviewed: https://review.openstack.org/497695
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-openvswitch/commit/?id=69e4eb8b00c9792d06f636724a2369b8bf7af092
Submitter: Jenkins
Branch: stable/17.02

commit 69e4eb8b00c9792d06f636724a2369b8bf7af092
Author: Billy Olsen <email address hidden>
Date: Tue Aug 22 19:19:06 2017 -0700

    Migrate openvswitch-switch file to avoid restarts

    The 16.10 release of the neutron-openvswitch charm changed the
    file management strategy of /etc/default/openvswitch-switch
    config file from managing the file when dpdk is enabled to always
    managing the file. Additionally, the template file was changed
    in the 16.10 release to modify the file header (commit 4463c334).
    These two changes guarantee that the contents of the file will
    change when upgrading the charm.

    The changing file contents causes the openvswitch-switch service
    to be restarted, which in turn causes a data plane outage. This
    commit fixes that by migrating the /etc/default/openvswitch-switch
    to be charm managed without restarting the openvswitch-switch
    service.

    The change will only attempt to migrate versions of the file
    which were created before 16.10 by searching for a marker in the
    rendered version of the file which was added in 16.10.

    Change-Id: Icc0f326991be239b88a57292740473f501181ebb
    Closes-Bug: #1712444
    (cherry picked from commit e3ec31c91df72cf64f79f14e0223ec61382c0553)

James Page (james-page)
Changed in charm-neutron-openvswitch:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.