Comment 42 for bug 1869808

Revision history for this message
Trent Lloyd (lathiat) wrote :

Ubuntu SRU Justification

[Impact]

- When there is a RabbitMQ or neutron-api outage, the neutron-openvswitch-agent undergoes a "resync" process and temporarily blocks all VM traffic. This always happens for a short time period (maybe <1 second) but in some high scale environments this lasts for minutes. If RabbitMQ is down again during the re-sync, traffic will also be blocked until it can connect which may be for a long period. This also affects situations where neutron-openvswitch-agent is intentionally restarted while RabbitMQ is down. Bug #1869808 addresses this issue and Bug #1887148 is a fix for that fix to prevent network loops during DVR startup.

- In the same situation, the neutron-l3-agent can delete the L3 router (Bug #1871850)

[Test Case]

(1) Deploy Openstack Bionic-Queens with DVR and a *VLAN* tenant network (VXLAN or FLAT will not reproduce the issue). With a standard deployment, simply enabling DHCP on the ext_net subnet will allow VMs to be booted directly on the ext_net provider network. "openstack subnet set --dhcp ext_net and then deploy the VM directly to ext_net"

(2) Deploy a VM to the VLAN network

(3) Start pinging the VM from an external network

(4) Stop all RabbitMQ servers

(5) Restart neutron-openvswitch-agent

(6) Ping traffic should cease and not recover

(7) Start all RabbitMQ servers

(8) Ping traffic will recover after 30-60 seconds

[Where problems could occur]

These patches are all cherry-picked from the upstream stable branches, and have existed upstream including the stable/queens branch for many months and in Ubuntu all supported subsequent releases (Stein onwards) have also had these patches for many months with the exception of Queens.

There is a chance that not installing these drop flows during startup could have traffic go somewhere that's not expected when the network is in a partially setup case, this was the case for DVR and in setups where more than 1 DVR external network port existed a network loop was possibly temporarily created. This was already addressed with the included patch for Bug #1869808. Checked and could not locate any other merged changes to this drop_port logic that also need to be backported.

[Other Info]