Comment 24 for bug 1927868

Our order of operations went as such:

- All packages get updated to latest available Ussuri (16.3.1) for Bionic via apt-get dist-upgrade
- Stop services (systemctl stop neutron-dhcp-agent; systemctl stop neutron-metadata-agent; systemctl stop neutron-ovs-cleanup; systemctl stop neutron-l3-agent; systemctl stop neutron-openvswitch-agent; systemctl stop neutron-server)
- Upgrade database (neutron-db-manage current; neutron-db-manage upgrade heads)
- Start services back up (systemctl start neutron-dhcp-agent; systemctl start neutron-metadata-agent; systemctl start neutron-ovs-cleanup; systemctl start neutron-l3-agent; systemctl start neutron-openvswitch-agent; systemctl start neutron-server)
- All L3 agents for all routers elect to be down
- Rebooted control plane several times while troubleshooting, L3 agents stay down
- Found that 16.3.2 was available on staging repository, installed it, all L3 agents go to standby
- Install 16.0.0 for Focal (all I could find during my scramble to fix the outage), L3 agents start to elect a master for each router
- Later that week, re-attempt 16.3.1, all L3 agents go to standby
- Find 16.2.0 for Bionic and downgrade to it, all L3 agents begin to elect masters