openstack services resource agents

[Andrea Ieri]

The principal openstack charms are currently only leveraging the hacluster charm for managing VIPs and running haproxy. The assumption is that haproxy should run whenever possible, and that haproxy health checks should take care of not routing traffic to backends that aren't functional; the openstack services themselves are not under pacemaker's management and will thus never be stopped. This is desirable under healthy conditions, but in some cases we might want to avoid routing traffic to a backend that isn't actually able to respond, for example because it's under heavy memory pressure or burdened by excessive cpu load. Bringing the openstack services under pacemaker's management would let the cluster disable backends that aren't functional.

I propose for openstack-specific, ocf-compliant resource agents to be created. The resource agents should include a health check, which will allow pacemaker to detect failures and react accordingly.

The health check doesn't need to address system-level state (e.g. ram usage and cpu load - this is covered by the system health resource agent in bug 1880613), just the health of the openstack service itself. Even a basic process check would be useful, as the presence of the resource agent would then allow pacemaker to include it in placement rules.

The new resource agents should then be upstreamed to https://github.com/ClusterLabs/resource-agents

Note: this proposal is not in contrast with requesting l7 haproxy backend checks in bug 1880610

[Billy Olsen (billy-olsen)]

Note that there exists https://opendev.org/x/openstack-resource-agents, however it appears to be a dead project. There are some packages available in older Ubuntu releases. It's possible these could be used to help influence, but upstream has yet to agree on health checks.

Additionally, the community has been discussing health checks in general but have not yet come to a consensus on what makes good health checks.