Removing router interface causes router to stop routing between all
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Cloud Archive |
Fix Released
|
High
|
Unassigned | ||
Stein |
Fix Released
|
Undecided
|
Unassigned | ||
Train |
Fix Released
|
Undecided
|
Unassigned | ||
Ussuri |
Fix Released
|
High
|
Unassigned | ||
Victoria |
Fix Released
|
High
|
Unassigned | ||
neutron |
Fix Released
|
High
|
Hemanth Nakkina | ||
neutron (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Focal |
Fix Released
|
High
|
Unassigned | ||
Groovy |
Fix Released
|
High
|
Unassigned | ||
Hirsute |
Fix Released
|
High
|
Unassigned |
Bug Description
[Impact]
Stumbled upon an issue where removing a DVR HA router interface renders all other subnets connected to that router to stop routing. VMs can't reach the HA port (IP) of the router (ping).
Worked around this by:
openstack router set --disabled <ROUTER>
openstack router set --enable <ROUTER>
This has happened more than once in the current deployment
- cloud:bionic-stein
- neutron 2:14.0.
[Test Case]
1. Reproducing the issue
1a. Deploy openstack using stsstack-bundles
https:/
1b. Run the test script lp1892405_
The script does the following (Detailed steps in comment #4)
- Create 3 projects P1, P2, P3
- Create a router and network in each project, say R1,R2,R3 and
N1,N2,N3
- Cross-connect networks by adding ports to router.
- Launch VMs on N1, N2 (Ensure VMs are landed on 2 different compute
nodes)
- ping from VM1 -> VM2 should be successful
- Detach leg from N1 -> N3
- Check for any packet loss during ping from VM1 -> VM2
The script output shows the ping output from VM1 -> VM2 and there will
be packet loss
2. Install the package with fixed code
3. Confirm bug have been fixed
3a. Cleanup of projects P1,P2,P3 and associated resources created in 1b
Re-enable the hypervisor which is disabled as part of 1b script.
Commands for the cleanup:
openstack server list --all-projects -c ID -f value | xargs openstack server delete
openstack router remove port P2-router to-n2
openstack router remove port P1-router from-n2
openstack router remove port P1-router from-n3
for i in P1 P2 P3; do openstack subnet list --project $i -c ID -f value | xargs openstack router remove subnet $i-router; done
for i in P1 P2 P3; do openstack router delete $i-router; done
for i in P1 P2 P3; do openstack network list --project $i -c ID -f value | xargs openstack network delete; done
openstack floating ip list -c ID -f value | xargs openstack floating ip delete
for i in P1 P2 P3; do openstack project delete $i; done
openstack compute service list --service nova-compute | grep disabled | awk '{print $6}' | xargs -I {} openstack compute service set --enable {} nova-compute
3b. Re-run the script 1b
The script output shows the ping output from VM1 -> VM2 and there
should not be any packet loss
[Where problems could occur]
Upstream CI ran all the functional and tempest test cases that involves deletion of DVR port connected to router which should cover the scenarios involving the code change.
Installation of new package will result in restart of neutron-openvswitch service and will take a few milliseconds to repopulate all the OVS flows.
tags: | added: l3-dvr-backlog |
Changed in neutron: | |
importance: | Undecided → High |
Changed in neutron: | |
assignee: | nobody → Hemanth Nakkina (hemanth-n) |
description: | updated |
description: | updated |
description: | updated |
tags: | added: neutron-proactive-backport-potential |
Subscribing field-high, as core functionality is being impacted