config-changed hook fails with enable_livestatus enabled

[Jason Hobbs]

http://paste.ubuntu.com/p/qQn2CZSPvY/

2019-03-11 20:20:47 INFO juju-log Fixing perms on the socket
2019-03-11 20:20:47 DEBUG config-changed Traceback (most recent call last):
2019-03-11 20:20:47 DEBUG config-changed File "/var/lib/juju/agents/unit-nagios-0/charm/hooks/config-changed", line 350, in <module>
2019-03-11 20:20:47 DEBUG config-changed enable_livestatus_config()
2019-03-11 20:20:47 DEBUG config-changed File "/var/lib/juju/agents/unit-nagios-0/charm/hooks/config-changed", line 106, in enable_livestatus_config
2019-03-11 20:20:47 DEBUG config-changed os.chown(livestatus_path, uid, gid)
2019-03-11 20:20:47 DEBUG config-changed OSError: [Errno 2] No such file or directory: '/var/lib/nagios3/livestatus/socket'

In enable_livestatus_config, it is creating this directory and then trying to chown livestatus_path, but it doesn't look like anything has made a socket there. I don't know what is supposed to.

This results in a hook failure. The application eventually gets to Ready, but this is enough to trip juju wait.

[Xav Paice (xavpaice)]

I'm unable to find a reproducer for this, it's likely either been fixed by an update since the bug was reported, or I need to get some more info on how to demonstrate this issue.

[Launchpad Janitor (janitor)]

[Expired for Nagios Charm because there has been no activity for 60 days.]

[Xav Paice (xavpaice)]

Reproducer:

juju deploy cs:nagios-38

wait for hooks to settle

juju config nagios enable_livestatus=true

2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 Traceback (most recent call last):
2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 File "/var/lib/juju/agents/unit-nagios-0/charm/hooks/config-changed", line 623, in <module>
2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 enable_livestatus_config()
2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 File "/var/lib/juju/agents/unit-nagios-0/charm/hooks/config-changed", line 185, in enable_livestatus_config
2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 os.chown(livestatus_path, uid, gid)
2020-09-30 07:32:28 WARNING unit.nagios/0.config-changed logger.go:60 OSError: [Errno 2] No such file or directory: '/var/lib/nagios3/livestatus/socket'
2020-09-30 07:32:28 ERROR juju.worker.uniter.operation runhook.go:136 hook "config-changed" (via explicit, bespoke hook script) failed: exit status 1

This is clearly some kind of race condition because retrying the hook is successful.

[Drew Freiberger (afreiberger)]

merged the fix for this into cs:~llama-charmers-next/nagios.

@jhobbs, can you test whether this new revision resolves the issue so we can enable livestatus in the fce bundles? If it resolves the issue, I can work on releasing this to cs:nagios on top of stable/20.10.