looking at haproxy logs, we can see mysql server oscillation:

https://logserver.rdoproject.org/32/40932/9/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/52ef39f/logs/overcloud-controller-1/var/log/containers/haproxy/haproxy.log.txt.gz

Jul 12 03:53:09 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is DOWN, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 15ms. 0 active and 2 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
Jul 12 03:53:29 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 15ms. 0 active and 3 backup servers online. Running on backup. 0 sessions requeued, 0 total in queue.
Jul 12 03:54:10 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is DOWN, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 17ms. 0 active and 2 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
Jul 12 03:54:12 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 16ms. 0 active and 3 backup servers online. Running on backup. 0 sessions requeued, 0 total in queue.
Jul 12 03:54:39 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is DOWN, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 16ms. 0 active and 2 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
Jul 12 03:54:42 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-1.internalapi.localdomain is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 14ms. 0 active and 3 backup servers online. Running on backup. 0 sessions requeued, 0 total in queue.
Jul 12 03:55:41 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-2.internalapi.localdomain is DOWN, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 14ms. 0 active and 2 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
Jul 12 03:55:43 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-2.internalapi.localdomain is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 14ms. 0 active and 3 backup servers online. Running on backup. 0 sessions requeued, 0 total in queue.
Jul 12 03:57:36 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-2.internalapi.localdomain is DOWN, reason: Layer4 timeout, check duration: 1000ms. 0 active and 2 backup servers left. Running on backup. 0 sessions active, 0 requeued, 0 remaining in queue.
Jul 12 03:57:37 overcloud-controller-1 haproxy[12]: Backup Server mysql/overcloud-controller-2.internalapi.localdomain is UP, reason: Layer7 check passed, code: 200, info: "OK", check duration: 14ms. 0 active and 3 backup servers online. Running on backup. 0 sessions requeued, 0 total in queue.