Activity log for bug #2056300

Date Who What changed Old value New value Message
2024-03-06 09:15:23 Dmitriy Rabotyagov bug added bug
2024-03-06 09:16:37 Dmitriy Rabotyagov description I do have a basic OpenStack-Ansible All-in-One deployment of the current master branch. Horizon SHA: a77818d745f25ed1e5b576760d3481f0f62ce042 Deployment was done following way: 1. git clone https://opendev.org/openstack/openstack-ansible 2. cd openstack-ansible 3. ./scripts/gate-check-commit.sh aio_lxc Once it's done, horizon is working properly. However, from time to time I do catch "429 Too many requests" which is potentially legit behavior by LB configuration, that tries to implement bot protection: http-request deny deny_status 429 if { sc_http_err_rate(0) gt 20 }: https://opendev.org/openstack/openstack-ansible/src/commit/e72984ca956d44d8056ca4ea8ea7e263bc3c8881/inventory/group_vars/horizon_all/haproxy_service.yml#L24 Violation of this rule is triggered by invalid path to roboto fontface which returns quite some 404 per each request: [06/Mar/2024:08:58:13.851] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/1/1/2 404 340 - - ---- 32/1/0/0/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Regular.woff2 HTTP/2.0" [06/Mar/2024:08:58:13.856] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/1/1 404 340 - - ---- 32/1/0/0/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2 HTTP/2.0" On the horizon container this path is indeed not valid: root@aio1-horizon-container-f7cace3d:/# ls -l /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2ls: cannot access '/openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2': No such file or directory root@aio1-horizon-container-f7cace3d:/# However, expected path is slightly different: root@aio1-horizon-container-f7cace3d:/# ls -l /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Bold.woff2 -rwxr-xr-x 1 root root 63596 Mar 6 08:37 /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Bold.woff2 root@aio1-horizon-container-f7cace3d:/# So doing smth like `cp -r /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/ /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/` does the trick. And the issue is basically in `roboto` vs `Roboto` in path. If I would try to `mv roboto Roboto` - then some dashboard will fail, like heat-dashboard, as they do expect `roboto`: [06/Mar/2024:09:10:30.452] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/0/0 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://94.101.116.61/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.woff2 HTTP/2.0" [06/Mar/2024:09:10:30.560] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/3/3 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://94.101.116.61/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.woff HTTP/2.0" [06/Mar/2024:09:10:30.604] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/1/1 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://94.101.116.61/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.ttf HTTP/2.0" The /roboto directory is being populated automatically during `horizon-manage.py collectstatic --noinput` execution. I do have a basic OpenStack-Ansible All-in-One deployment of the current master branch. Horizon SHA: a77818d745f25ed1e5b576760d3481f0f62ce042 Deployment was done following way: 1. git clone https://opendev.org/openstack/openstack-ansible 2. cd openstack-ansible 3. ./scripts/gate-check-commit.sh aio_lxc Once it's done, horizon is working properly. However, from time to time I do catch "429 Too many requests" which is potentially legit behavior by LB configuration, that tries to implement bot protection: http-request deny deny_status 429 if { sc_http_err_rate(0) gt 20 }: https://opendev.org/openstack/openstack-ansible/src/commit/e72984ca956d44d8056ca4ea8ea7e263bc3c8881/inventory/group_vars/horizon_all/haproxy_service.yml#L24 Violation of this rule is triggered by invalid path to roboto fontface which returns quite some 404 per each request: [06/Mar/2024:08:58:13.851] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/1/1/2 404 340 - - ---- 32/1/0/0/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Regular.woff2 HTTP/2.0" [06/Mar/2024:08:58:13.856] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/1/1 404 340 - - ---- 32/1/0/0/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2 HTTP/2.0" On the horizon container this path is indeed not valid: root@aio1-horizon-container-f7cace3d:/# ls -l /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2ls: cannot access '/openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/Roboto-Bold.woff2': No such file or directory root@aio1-horizon-container-f7cace3d:/# However, expected path is slightly different: root@aio1-horizon-container-f7cace3d:/# ls -l /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Bold.woff2 -rwxr-xr-x 1 root root 63596 Mar 6 08:37 /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Bold.woff2 root@aio1-horizon-container-f7cace3d:/# So doing smth like `cp -r /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/roboto/ /openstack/venvs/horizon-28.1.0.dev67/lib/python3.10/site-packages/static/horizon/lib/roboto_fontface/fonts/Roboto/` does the trick. And the issue is basically in `roboto` vs `Roboto` in path. If I would try to `mv roboto Roboto` - then some dashboard will fail, like heat-dashboard, as they do expect `roboto`: [06/Mar/2024:09:10:30.452] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/0/0 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.woff2 HTTP/2.0" [06/Mar/2024:09:10:30.560] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/3/3 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.woff HTTP/2.0" [06/Mar/2024:09:10:30.604] base-front-1~ horizon-back/aio1_horizon_container-f7cace3d 0/0/0/1/1 404 340 - - ---- 38/1/2/2/0 0/0 "GET https://ip.add.re.ss/static/horizon/lib/roboto_fontface/fonts/roboto/Roboto-Regular.ttf HTTP/2.0" The /roboto directory is being populated automatically during `horizon-manage.py collectstatic --noinput` execution.