HAProxy sends request to unhealthy servers
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
kolla-ansible |
In Progress
|
Undecided
|
Aravindh |
Bug Description
What happened: HA Proxy keeps sending traffic to some unhealthy containers.
What I expect to happen: HA Proxy should detect the unavailability and remove then from loadbalancing pool.
How to reproduce it:
- In rare scenarios, TCP connection is healthy for a container, but the web server within the containers are not responding properly.
In my case keystone-api on one of the servers encountered this scenario, and I noticed many of my auth request failed intermittently. Further troubleshooting shed light of this mentioned scenario.
Environment:
- OS = Ubuntu Jammy 22.04
- Kernel = 5.15.0-105-generic #115-Ubuntu SMP
- Docker Version = 26
- Kolla Ansible = stable/2023.1
- Docker Image Distribution = Ubuntu
- Using self built images, only added Neutron FWaaS unrelated to this scenario
Changed in kolla-ansible: | |
assignee: | nobody → Aravindh (highlow7) |
Changed in kolla-ansible: | |
status: | New → In Progress |
Add L7 Checks like this (https:/ /review. opendev. org/c/openstack /kolla- ansible/ +/918437) fixes the issue.