deferred restarts provide incorrect charm status after machine reboots

[Adam Dyess]

The use of charm specific files in /etc/policy-rc.d/ to prevent services from restarting during a package upgrade (or other charm services) also prevents those from starting up when the machine is rebooted.

Combined with LP#1934406, the charm presents a very friendly "active/idle" when in reality there are necessary services not running requiring the restart actions to be run.

Can you conceive of some way to monitor or signal that there are vital services not running after a reboot, or even better ensure services are running after a machine reboot?

[Adam Dyess (addyess)]

After an analysis of the logs at /var/log/policy-rc.d.log, we believe that the charm status just didn't reflect that the services were running after reboot while the 'start' actions seem to have been permitted.

The normal 'service' nagios checks were showing the service up and running, but the charms indicated that there were actions needing to be run on the charm.

is this status still correct even if the machine reboots?
"Hooks skipped due to disabled auto restarts: configure_ovs, install"

[Drew Freiberger (afreiberger)]

It would be nice if there were a configurable grace-period after reboot of a machine that hooks could be non-deferred to address the hooks not firing at boot time. Otherwise, there is a lot of operations work to restore services after a host reboot.