Need to monitor availability of wsgi.py content

[Drew Freiberger]

It was found during triage of lp#1849864, that two of the three horizon units were returning 404 errors for /auth/login URI.

Logs:

/var/log/apache2/ssl_access.log
10.x.x.x - - [25/Oct/2019:19:14:15 +0000] "GET /auth/login/ HTTP/1.1" 404 706 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36"

/var/log/apache2/error.log
[Fri Oct 25 19:16:34.772793 2019] [wsgi:error] [pid 1575811:tid 140498858981120] [client 10.55.196.109:60258] Target WSGI script not found or unable to stat: /usr/share/openstack-dashboard/openstack_dashboard/wsgi.py

Having a simple check against the local apache instance on each dashboard unit that exercises the wsgi.py script and ensures that return code is not 404 would help to detect issues with dashboard installs/upgrades.

[Drew Freiberger (afreiberger)]

Two of three units were exhibiting issues, while the third unit was returning wsgi.py content properly. When the engineer was performing a manual check, it was pseudo-random due to haproxy high-availability whether you received a working Horizon dashboard or received 404 errors due to the broken path to wsgi.py.