IVS restart causes network disruption during update

Bug #1757130 reported by Bob Fournier
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-net-config
Fix Released
Undecided
Bob Fournier

Bug Description

Instances network are intermittently disconnected during stack update when using BCF.
ivs service will be restarted several times during which the instance network get disconnected

Feb 26 12:26:48 XXX.localnet os-collect-config[7070]: [2018/02/26 12:26:48 PM] [INFO] Restart ivs
Feb 26 12:48:34 XXX.localnet os-collect-config[7070]: [2018/02/26 12:48:34 PM] [INFO] Restart ivs
Feb 26 12:49:56 XXX.localnet os-collect-config[7070]: [2018/02/26 12:49:56 PM] [INFO] Restart ivs
Feb 26 12:52:24 XXX.localnet os-collect-config[7070]: [2018/02/26 12:52:24 PM] [INFO] Restart ivs
Feb 26 12:58:22 XXX.localnet os-collect-config[7070]: [2018/02/26 12:58:22 PM] [INFO] Restart ivs
Feb 26 13:04:20 XXX.localnet os-collect-config[7070]: [2018/02/26 01:04:20 PM] [INFO] Restart ivs
Feb 26 13:10:18 XXX.localnet os-collect-config[7070]: [2018/02/26 01:10:18 PM] [INFO] Restart ivs
Feb 26 13:16:25 XXX.localnet os-collect-config[7070]: [2018/02/26 01:16:25 PM] [INFO] Restart ivs
Feb 26 13:22:35 XXX.localnet os-collect-config[7070]: [2018/02/26 01:22:35 PM] [INFO] Restart ivs

The restart is in the code below in os_net_config/impl_ifcfg.py:

           if ivs_uplinks or ivs_interfaces:
                logger.info("Attach to ivs with "
                            "uplinks: %s, "
                            "interfaces: %s" %
                            (ivs_uplinks, ivs_interfaces))
                for ivs_uplink in ivs_uplinks:
                    self.ifup(ivs_uplink)
                for ivs_interface in ivs_interfaces:
                    self.ifup(ivs_interface)
                msg = "Restart ivs"
                self.execute(msg, '/usr/bin/systemctl',
                             'restart', 'ivs')

Is this restart necessary? Can it be changed to something less intrusive?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-net-config (master)

Fix proposed to branch: master
Review: https://review.openstack.org/555369

Changed in os-net-config:
assignee: nobody → Bob Fournier (bfournie)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-net-config (master)

Reviewed: https://review.openstack.org/555369
Committed: https://git.openstack.org/cgit/openstack/os-net-config/commit/?id=db081a75c5d2b7a99b4b796f9de7a4aadd7ab522
Submitter: Zuul
Branch: master

commit db081a75c5d2b7a99b4b796f9de7a4aadd7ab522
Author: Bob Fournier <email address hidden>
Date: Thu Mar 22 12:34:59 2018 -0400

    Don't restart ivs/nfvswitch in os-net-config

    Restarting ivs during ugrades causes network disruptions. Removing both
    the ivs and nvfswitch restarts as they are not required.

    Change-Id: I3d26cbc8484c72d6bfc133c59c8fc065add226c9
    Closes-Bug: 1757130

Changed in os-net-config:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-net-config (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/556998

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-net-config (stable/queens)

Reviewed: https://review.openstack.org/556998
Committed: https://git.openstack.org/cgit/openstack/os-net-config/commit/?id=08a0f081bf627ef09b934b2cc1dc66e172a1dbd4
Submitter: Zuul
Branch: stable/queens

commit 08a0f081bf627ef09b934b2cc1dc66e172a1dbd4
Author: Bob Fournier <email address hidden>
Date: Thu Mar 22 12:34:59 2018 -0400

    Don't restart ivs/nfvswitch in os-net-config

    Restarting ivs during ugrades causes network disruptions. Removing both
    the ivs and nvfswitch restarts as they are not required.

    Change-Id: I3d26cbc8484c72d6bfc133c59c8fc065add226c9
    Closes-Bug: 1757130
    (cherry picked from commit db081a75c5d2b7a99b4b796f9de7a4aadd7ab522)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-net-config (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/561608

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-net-config (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/561609

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-net-config (stable/pike)

Reviewed: https://review.openstack.org/561608
Committed: https://git.openstack.org/cgit/openstack/os-net-config/commit/?id=137abf0138f538d79b79c4aefef03e290cba808e
Submitter: Zuul
Branch: stable/pike

commit 137abf0138f538d79b79c4aefef03e290cba808e
Author: Bob Fournier <email address hidden>
Date: Thu Mar 22 12:34:59 2018 -0400

    Don't restart ivs/nfvswitch in os-net-config

    Restarting ivs during ugrades causes network disruptions. Removing both
    the ivs and nvfswitch restarts as they are not required.

    Change-Id: I3d26cbc8484c72d6bfc133c59c8fc065add226c9
    Closes-Bug: 1757130
    (cherry picked from commit db081a75c5d2b7a99b4b796f9de7a4aadd7ab522)
    (cherry picked from commit 08a0f081bf627ef09b934b2cc1dc66e172a1dbd4)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-net-config 9.0.0

This issue was fixed in the openstack/os-net-config 9.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-net-config 8.4.1

This issue was fixed in the openstack/os-net-config 8.4.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-net-config 7.3.5

This issue was fixed in the openstack/os-net-config 7.3.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-net-config (stable/ocata)

Reviewed: https://review.openstack.org/561609
Committed: https://git.openstack.org/cgit/openstack/os-net-config/commit/?id=5e97b5c3643c12897aa6f3d1f4870378881f582d
Submitter: Zuul
Branch: stable/ocata

commit 5e97b5c3643c12897aa6f3d1f4870378881f582d
Author: Bob Fournier <email address hidden>
Date: Thu Mar 22 12:34:59 2018 -0400

    Don't restart ivs/nfvswitch in os-net-config

    Restarting ivs during ugrades causes network disruptions. Removing both
    the ivs and nvfswitch restarts as they are not required.

    Change-Id: I3d26cbc8484c72d6bfc133c59c8fc065add226c9
    Closes-Bug: 1757130
    (cherry picked from commit db081a75c5d2b7a99b4b796f9de7a4aadd7ab522)
    (cherry picked from commit 08a0f081bf627ef09b934b2cc1dc66e172a1dbd4)

tags: added: in-stable-ocata
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/os-net-config 6.1.5

This issue was fixed in the openstack/os-net-config 6.1.5 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.