Comment 1 for bug 1885264

Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Some additional information regarding how the bug was found:

During the upgrade the ovs runs, deleting the ports. They don't come back, even after restarting the services, which suggests an issue within the neutron openvswitch agent in terms of re-creating ports that have been deleted.

from the ovsdb-tool -mm show-log tool on the neutron-gateway AFTER the upgrade.

```
record 44: 2020-06-25 19:10:48.024
  table Interface row "tap03282320-fb" (74bf6e22):
    delete row
  table Interface row "tap7e15b3c6-e3" (d331a9b6):
    delete row
  table Interface row "tap8e76995d-a6" (f8ca1e54):
    delete row
  table Port row "tap03282320-fb" (d891c2a5):
    delete row
  table Port row "tap7e15b3c6-e3" (8f105882):
    delete row
  table Port row "tap8e76995d-a6" (b5f389e0):
    delete row
  table Bridge row "br-int" (56555638):
    ports=[704ecf79-ff42-44bd-a72c-c586f59f433b, dcfd1c64-2e07-4bf7-8815-a96950384705, f38673c2-87c9-4ce1-b160-236637caf8d2]
  table Open_vSwitch row 70200fb1 (70200fb1):
    next_cfg=111
```

i.e. it was asked to be deleted.

from syslog (during upgrade):

```
Jun 25 19:10:44 juju-ed722c-mojo-14 systemd[1]: Stopping Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:44 juju-ed722c-mojo-14 ovsdb-client: ovs|00001|fatal_signal|WARN|terminating with signal 15 (Terminated)
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopped Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopped OpenStack Neutron OVS cleanup.
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron OVS cleanup...
Jun 25 19:10:46 juju-ed722c-mojo-14 systemd[1]: Starting OpenStack Neutron OVS cleanup...
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.589055] device tap03282320-fb left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.611240] device tap7e15b3c6-e3 left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 kernel: [ 6499.612612] device tap8e76995d-a6 left promiscuous mode
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Started OpenStack Neutron OVS cleanup.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Starting Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Started Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Reloading.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopping Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopped Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopped OpenStack Neutron OVS cleanup.
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron OVS cleanup...
Jun 25 19:10:48 juju-ed722c-mojo-14 systemd[1]: Starting OpenStack Neutron OVS cleanup...
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Started OpenStack Neutron OVS cleanup.
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Starting Openstack Neutron Open vSwitch Plugin Agent...
Jun 25 19:10:50 juju-ed722c-mojo-14 systemd[1]: Started Openstack Neutron Open vSwitch Plugin Agent.
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: Reloading.
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: message repeated 2 times: [ Reloading.]
Jun 25 19:10:51 juju-ed722c-mojo-14 systemd[1]: Stopping OpenStack Neutron Metering Agent...
```