[ovn-octavia-provider] Deleting or updating Load Balancer members in on pool can effect other pools or same pool if the same IP is used
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
New
|
Undecided
|
Unassigned |
Bug Description
* High level description:
We have discovered that when using Octavia OVN Load Balancers, if you have 2 or more pools with the same member ips (but different ports) if you remove the member from one pool it will remove the ip configuration from the "ip_port_mappings" in the OVN Load Balancer even if another pool is still using this IP. Additionally, if you update the pool members to point to the some member IP's but a different port the "ip_port_mappings" will also be cleared for those IP's. The update behavior is because the additions are done by the provider followed by the deletions, so if the IP addresses are the same, the final deletions will clear out the IP's.
We experienced this problem when using Magnum with OVN load balancers, as Magnum in some cases would update the load balancers to ensure proper configuration, and this would result in the load balancers not function even though the final Octavia configuration was correct (but since the ip_port_mapping table was cleared the LB were not operational)
This behavior was likely introduced by the fix for bug:
https:/
Which resulted in commit:
https:/
This commit updates the OVS / OVN databases in a more efficient way (only adding or deleting what is needed). However, since it is not recomputing all ip_port_mappings for every update, its possible to hit this scenario.
The "ovn-octavia-
However, it should be noted that the logic in the "_clean_
Therefore, it may be optimal to update the checks to be more thorough. If the maintainers of this project are in agreement with the logic of my proposed updates, I could work to submit an oficial patch for this issue.
* Version:
** OpenStack version
Openstack Antelope (2023.1)
** Linux distro, kernel.
Ubuntu 22.04
** DevStack or other _deployment_ mechanism?
Kolla Anisable
Patch for fixing issue:
```
--- helper.py.new_orig 2024-01-21 03:05:27.000000000 +0000
+++ helper.py.proposed1 2024-01-24 19:12:17.843028336 +0000
@@ -2441,7 +2441,7 @@
return True
- def _update_
+ def _update_
# ip_port_
@@ -2449,10 +2449,42 @@
# MEMBER_IP: IP of member_lsp
# LSP_NAME_MEMBER: Logical switch port
# HEALTH_SRC: source IP of hm_port
-
if delete:
- self.ovn_
- backend_
+ # NOTE(jrhine): This is basically the same code as in the _clean_
+ # function, but with a single member ip. Just like in that function,
+ # before removing a member from the ip_port_mappings
+ # list, we need to ensure that the member is not being used
+ # by an other pool to prevent accidentally removing the
+ # member we can use the neutron:
+ # other members with the same address
+ other_members = []
+ for k, v in ovn_lb.
+ if ovn_const.
+ other_members.
+ ovn_lb.
+
+ member_statuses = ovn_lb.
+ ovn_const.
+
+ try:
+ member_statuses = jsonutils.
+ except TypeError:
+ LOG.debug("no member status on external_ids: %s",
+ str(member_
+ member_statuses = {}
+
+ execute_delete = True
+ for member_id in [item[3] for item in other_members
+ if item[0] == backend_ip]:
+ if member_
+ member_id, '') != constants.
+ # same address being monitorized by another HM
+ execute_delete = False
+ LOG.debug(f"Rumble: member {member_id} using address {backend_ip}, so not deleting it ")
+ if execute_delete:
+ LOG.debug(f"Rumble: verified address {backend_ip} is not being used both other memebers, so deleting ip_port_mapping")
+ self.ovn_
+ backend_
else:
@@ -2542,7 +2574,7 @@
- member_lsp.name, hm_source_ip,
+ member_lsp.name, hm_source_ip, pool_key,
```