after stopping and restarting containerized services, swift_proxy healthcheck doesn't work anymore

[Damien Ciabrini]

Seen on a HA controller after I did the following:

# stop all paunch-managed services
systemctl stop tripleo_\*

# once they are all stopped, restart them
systemctl start -a tripleo_\*

after a couple of minutes, all healthcheck are rescheduled as expected, but swift_proxy healthcheck failed and is stopped. From systemctl logs:

[root@controller-2 ~]# systemctl status tripleo_swift_proxy_healthcheck.service
● tripleo_swift_proxy_healthcheck.service - swift_proxy healthcheck
   Loaded: loaded (/etc/systemd/system/tripleo_swift_proxy_healthcheck.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2019-06-26 19:58:42 UTC; 13s ago
  Process: 515701 ExecStart=/usr/bin/podman exec swift_proxy /openstack/healthcheck (code=exited, status=1/FAILURE)
 Main PID: 515701 (code=exited, status=1/FAILURE)

Jun 26 19:58:40 controller-2 systemd[1]: Starting swift_proxy healthcheck...
Jun 26 19:58:42 controller-2 podman[515701]: 2019-06-26 19:58:42.354028122 +0000 UTC m=+2.279254025 container exec 97bc6ad79999d86f380494c517e1f578888b67c1d96a70a2ddbdd7fd6931a3c9 (image=192.168.24.1:8787/rhosp15/openstack-swift-proxy-server:20190621.1, name=swift_proxy)
Jun 26 19:58:42 controller-2 podman[515701]: /var/cache/swift/object.recon does not exist for file modification check
Jun 26 19:58:42 controller-2 podman[515701]: Cache file /var/cache/swift/object.recon was not updated within interval of 300 seconds.
Jun 26 19:58:42 controller-2 podman[515701]: Error: exit status 1
Jun 26 19:58:42 controller-2 systemd[1]: tripleo_swift_proxy_healthcheck.service: Main process exited, code=exited, status=1/FAILURE
Jun 26 19:58:42 controller-2 systemd[1]: tripleo_swift_proxy_healthcheck.service: Failed with result 'exit-code'.
Jun 26 19:58:42 controller-2 systemd[1]: Failed to start swift_proxy healthcheck.

[Damien Ciabrini (dciabrin)]

Forgot to mentionned: this is based on Stein (OSP15 downstream)