[K8s] Install hook is not retriggered when Kubernetes Pod containing charm is recreated

[Claudiu Belu]

In Kubernetes, the Pods may be recreated due to various reasons (e.g.: Node restarted and the Pods have been recreated on other Nodes, Pods are evacuated from a Node, etc.). In the case of Kubernetes charms (using the operator framework), if a Pod is getting recreated, the install hook is not ran in the newly created Pod, which can cause some failures (e.g.: some dependencies may be installed in the install hook). After a bit of investigation, it seems that the upgrade hook is still triggered when recreating the Pod however.

To avoid this issue, ideally, the install hook would still be executed in case a Pod is recreated as it would be the most intuitive from a charm development perspective.

In the case in which dependencies are being installed in the install hook, a way to have them preinstalled would be useful (via charmcraft or similar), and it would speed up the charm deployment process.

How to reproduce this issue:

1. Have a charm with an install hook, in which an apt package is installed.
2. Assert that the apt package is installed in the container after deploying the charm.
3. Crash the charm's Kubernetes Pod, causing it to be recreated (the easiest way to do it is to delete the Pod: kubectl delete -n juju-model pod/charm-name-0
4. After the Pod is recreated, assert that the apt package no longer exists in the container.

Juju version: 3.2.4