I did several no-op stack updates where puppet was re-applied on a nonha stack, and it seems that httpd fails to start after a couple of iterations, possibly related to this aodh error:
# systemctl status httpd.service
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/httpd.service.d
└─openstack-dashboard.conf
Active: failed (Result: exit-code) since Thu 2016-06-09 11:04:56 UTC; 22s ago
Docs: man:httpd(8)
man:apachectl(8)
Process: 29445 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=1/FAILURE)
Process: 29442 ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND (code=exited, status=1/FAILURE)
Process: 29377 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py compress --force (code=exited, status=0/SUCCESS)
Process: 29357 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear (code=exited, status=0/SUCCESS)
Main PID: 29442 (code=exited, status=1/FAILURE)
Jun 09 11:04:45 overcloud-controller-0 python[29357]: Copying '/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/l...help.html'
Jun 09 11:04:45 overcloud-controller-0 python[29357]: Copying '/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/l...ints.html'
Jun 09 11:04:45 overcloud-controller-0 python[29357]: Copying '/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/l...roller.js'
Jun 09 11:04:45 overcloud-controller-0 python[29357]: Copying '/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/l...s.spec.js'
Jun 09 11:04:56 overcloud-controller-0 systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Jun 09 11:04:56 overcloud-controller-0 kill[29445]: kill: cannot find process ""
Jun 09 11:04:56 overcloud-controller-0 systemd[1]: httpd.service: control process exited, code=exited status=1
Jun 09 11:04:56 overcloud-controller-0 systemd[1]: Failed to start The Apache HTTP Server.
Jun 09 11:04:56 overcloud-controller-0 systemd[1]: Unit httpd.service entered failed state.
Jun 09 11:04:56 overcloud-controller-0 systemd[1]: httpd.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
[root@overcloud-controller-0 ~]# journalctl -xe
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination File "/usr/lib/python2.7/site-packages/tooz/drivers/redis.py",
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination cause=e)
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination File "/usr/lib/python2.7/site-packages/tooz/coordination.py", l
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination excutils.raise_with_cause(exc_cls, message, *args, **kwargs)
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py",
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination six.raise_from(exc_cls(message, *args, **kwargs), kwargs.get(
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination File "/usr/lib/python2.7/site-packages/six.py", line 692, in ra
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination raise value
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination ToozConnectionError: Error 113 connecting to 192.0.2.6:6379. No r
Jun 09 11:05:18 overcloud-controller-0 aodh-evaluator[20324]: 2016-06-09 11:05:18.768 20324 ERROR aodh.coordination
/var/log/aodh/evaluator.log is full of similar errors.
Not seen this in a while so marking invalid as I think it was fixed via another bug