In previous releases, and upgrade to the openvswitch packages which results in a restart of the userspace daemons is detected by the neutron-openvswitch-agent, which then tidies and restores the required state resulting in a minimal outage to instances on the hypervisor being upgrades.
Whilst testing updates to ovs 2.8.1, I noted that instance connectivity dropped but was never restored; restarting the neutron-openvswitch-agent resolved the issue.
2017-10-11 22:12:43.481 28598 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: send error: Connection refused
2017-10-11 22:12:43.482 28598 WARNING ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused)
2017-10-11 22:12:45.332 28598 WARNING neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] OVS is restarted. OVSNeutronAgent will reset bridges and recover ports.
2017-10-11 22:12:45.647 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Mapping physical network physnet1 to bridge br-data
2017-10-11 22:12:45.700 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Bridge br-data has datapath-ID 00003e24776a3b44
2017-10-11 22:12:45.886 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Port b08484e1-5e7d-45e9-86d8-7078ded39d9f updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': 'f1b6715f-9d79-457f-8272-6b54d0f82332', 'segmentation_id': 5, 'fixed_ips': [{'subnet_id': 'b069aeff-8879-4007-9799-bd350c3d1e5c', 'ip_address': '192.168.21.3'}], 'device_owner': u'compute:nova', 'physical_network': None, 'mac_address': 'fa:16:3e:69:89:92', 'device': u'b08484e1-5e7d-45e9-86d8-7078ded39d9f', 'port_security_enabled': True, 'port_id': 'b08484e1-5e7d-45e9-86d8-7078ded39d9f', 'network_type': u'gre', 'security_groups': ['84afead6-fcc4-45f7-94f1-017f263ca080']}
2017-10-11 22:12:45.887 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Assigning 1 as local vlan for net-id=f1b6715f-9d79-457f-8272-6b54d0f82332
2017-10-11 22:12:45.892 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Port 8aedca8c-fd85-42f0-97fa-eacb43a3364b updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': 'f1b6715f-9d79-457f-8272-6b54d0f82332', 'segmentation_id': 5, 'fixed_ips': [{'subnet_id': 'b069aeff-8879-4007-9799-bd350c3d1e5c', 'ip_address': '192.168.21.10'}], 'device_owner': u'compute:nova', 'physical_network': None, 'mac_address': 'fa:16:3e:50:b7:1a', 'device': u'8aedca8c-fd85-42f0-97fa-eacb43a3364b', 'port_security_enabled': True, 'port_id': '8aedca8c-fd85-42f0-97fa-eacb43a3364b', 'network_type': u'gre', 'security_groups': ['84afead6-fcc4-45f7-94f1-017f263ca080']}
2017-10-11 22:12:45.893 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Assigning 1 as local vlan for net-id=f1b6715f-9d79-457f-8272-6b54d0f82332
2017-10-11 22:12:45.902 28598 INFO neutron.agent.securitygroups_rpc [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Preparing filters for devices set([u'b08484e1-5e7d-45e9-86d8-7078ded39d9f', u'8aedca8c-fd85-42f0-97fa-eacb43a3364b'])
2017-10-11 22:12:47.813 28598 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-d616236d-5a5f-4f55-a721-83ff5079e5f0 - - - - -] Configuration for devices up [u'b08484e1-5e7d-45e9-86d8-7078ded39d9f', u'8aedca8c-fd85-42f0-97fa-eacb43a3364b'] and devices down [] completed.
In previous releases, and upgrade to the openvswitch packages which results in a restart of the userspace daemons is detected by the neutron- openvswitch- agent, which then tidies and restores the required state resulting in a minimal outage to instances on the hypervisor being upgrades.
Whilst testing updates to ovs 2.8.1, I noted that instance connectivity dropped but was never restored; restarting the neutron- openvswitch- agent resolved the issue.
2017-10-11 22:12:43.481 28598 WARNING ovsdbapp. backend. ovs_idl. vlog [-] tcp:127.0.0.1:6640: send error: Connection refused backend. ovs_idl. vlog [-] tcp:127.0.0.1:6640: connection dropped (Connection refused) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] OVS is restarted. OVSNeutronAgent will reset bridges and recover ports. plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Mapping physical network physnet1 to bridge br-data plugins. ml2.drivers. openvswitch. agent.openflow. native. ovs_bridge [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Bridge br-data has datapath-ID 00003e24776a3b44 plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Port b08484e1- 5e7d-45e9- 86d8-7078ded39d 9f updated. Details: {'profile': {}, 'network_ qos_policy_ id': None, 'qos_policy_id': None, 'allowed_ address_ pairs': [], 'admin_state_up': True, 'network_id': 'f1b6715f- 9d79-457f- 8272-6b54d0f823 32', 'segmentation_id': 5, 'fixed_ips': [{'subnet_id': 'b069aeff- 8879-4007- 9799-bd350c3d1e 5c', 'ip_address': '192.168.21.3'}], 'device_owner': u'compute:nova', 'physical_network': None, 'mac_address': 'fa:16: 3e:69:89: 92', 'device': u'b08484e1- 5e7d-45e9- 86d8-7078ded39d 9f', 'port_security_ enabled' : True, 'port_id': 'b08484e1- 5e7d-45e9- 86d8-7078ded39d 9f', 'network_type': u'gre', 'security_groups': ['84afead6- fcc4-45f7- 94f1-017f263ca0 80']} plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Assigning 1 as local vlan for net-id= f1b6715f- 9d79-457f- 8272-6b54d0f823 32 plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Port 8aedca8c- fd85-42f0- 97fa-eacb43a336 4b updated. Details: {'profile': {}, 'network_ qos_policy_ id': None, 'qos_policy_id': None, 'allowed_ address_ pairs': [], 'admin_state_up': True, 'network_id': 'f1b6715f- 9d79-457f- 8272-6b54d0f823 32', 'segmentation_id': 5, 'fixed_ips': [{'subnet_id': 'b069aeff- 8879-4007- 9799-bd350c3d1e 5c', 'ip_address': '192.168.21.10'}], 'device_owner': u'compute:nova', 'physical_network': None, 'mac_address': 'fa:16: 3e:50:b7: 1a', 'device': u'8aedca8c- fd85-42f0- 97fa-eacb43a336 4b', 'port_security_ enabled' : True, 'port_id': '8aedca8c- fd85-42f0- 97fa-eacb43a336 4b', 'network_type': u'gre', 'security_groups': ['84afead6- fcc4-45f7- 94f1-017f263ca0 80']} plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Assigning 1 as local vlan for net-id= f1b6715f- 9d79-457f- 8272-6b54d0f823 32 agent.securityg roups_rpc [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Preparing filters for devices set([u' b08484e1- 5e7d-45e9- 86d8-7078ded39d 9f', u'8aedca8c- fd85-42f0- 97fa-eacb43a336 4b']) plugins. ml2.drivers. openvswitch. agent.ovs_ neutron_ agent [req-d616236d- 5a5f-4f55- a721-83ff5079e5 f0 - - - - -] Configuration for devices up [u'b08484e1- 5e7d-45e9- 86d8-7078ded39d 9f', u'8aedca8c- fd85-42f0- 97fa-eacb43a336 4b'] and devices down [] completed.
2017-10-11 22:12:43.482 28598 WARNING ovsdbapp.
2017-10-11 22:12:45.332 28598 WARNING neutron.
2017-10-11 22:12:45.647 28598 INFO neutron.
2017-10-11 22:12:45.700 28598 INFO neutron.
2017-10-11 22:12:45.886 28598 INFO neutron.
2017-10-11 22:12:45.887 28598 INFO neutron.
2017-10-11 22:12:45.892 28598 INFO neutron.
2017-10-11 22:12:45.893 28598 INFO neutron.
2017-10-11 22:12:45.902 28598 INFO neutron.
2017-10-11 22:12:47.813 28598 INFO neutron.
ProblemType: Bug openvswitch- agent 2:11.0. 1-0ubuntu1~ cloud0 [origin: Canonical] ature: Ubuntu 4.4.0-96. 119-generic 4.4.83
"impl" : "launchpad",
"project" : "cloud-archive",
"bug_ pattern_ url": "http:// people. canonical. com/~ubuntu- archive/ bugpatterns/ bugpatterns. xml", Zone: nova ture: all 256color- bce DIR=<set> .etc.neutron. plugins. ml2.openvswitch _agent. ini: 2017-10- 11T19:36: 21.675410
DistroRelease: Ubuntu 16.04
Package: neutron-
ProcVersionSign
Uname: Linux 4.4.0-96-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.10
Architecture: amd64
CrashDB:
{
}
Date: Wed Oct 11 22:17:32 2017
Ec2AMI: ami-0000022c
Ec2AMIManifest: FIXME
Ec2Availability
Ec2InstanceType: m1.medium
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
PackageArchitec
ProcEnviron:
TERM=screen-
PATH=(custom, no user)
XDG_RUNTIME_
LANG=en_US.UTF-8
SHELL=/bin/bash
SourcePackage: neutron
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile.