Livepatch charm "server not running" after machine restart

[Yamen Hatahet]

After setting up the canonical-livepatch-server charm in juju and attaching the token to it, then running the `juju run-action livepatch/leader get-resource-token --wait` command, the status is set to "✓ Livepatch server running.". However, when restarting the machine that is hosting juju/charms, the status is reverted back to "✗ Livepatch server not running."

Is it possible to remove the need to manually have to fetch the token every time the host machine restarts?

Running on Channel "latest/stable", with revision "30".

- output of `juju config livepatch`: https://pastebin.canonical.com/p/Dm7Bnh9vPT/

[Kian Parvin (kian-parvin)]

Hi,

Thanks for the report. It's likely not the case that the get-resource-token action needs to be run again. A few things,

- You can try `juju run-action livepatch/<unit> restart` to restart the server and see if that addresses the issue. In any case though the charm should be starting the server when the machine is restarted so that still needs some investigating.
- Looking at the logs from `juju debug-log` and inside the machine via `juju ssh livepatch/leader` then `journalctl` is there anything indicating why Livepatch can't/was unable to start after the machine was rebooted?
- The charm revision seems out of date, the latest revision in "latest/stable" is "39" as seen here https://charmhub.io/canonical-livepatch-server

[Yamen Hatahet (yhatahet)]

Hi Kian,
Thank you, will update it soon and fetch you updated logs :)

[Adam Vest (foxmulder2004)]

Hello,

Going to throw my thoughts in on this because I've been noticing the same behavior (namely my test livepatch deployments that I power off for the day and bring back up the next day don't start of their own volition). I think the issue is because the livepatch service file itself isn't enabled, and can't be enabled (normally) because it has no [Install] section:
---
:~# systemctl cat livepatch-server.service
# /etc/systemd/system/livepatch-server.service
[Unit]
Description=Livepatch server

[Service]
Type=simple
ExecStart=/usr/local/bin/livepatchd /etc/livepatchd.yaml
Restart=always
RestartSec=10
StartLimitBurst=5
StartLimitInterval=0
---

I think a simple solution here (and one I've taken to implementing myself for now) is to just add an [Install] section to the service file and enable it:
---
echo -e "[Install]\nWantedBy=multi-user.target" >> /etc/systemd/system/livepatch-server.service
systemctl daemon-reload
systemctl enable --now livepatch-server
---

Resulting in:
---
:~# systemctl cat livepatch-server.service
# /etc/systemd/system/livepatch-server.service
[Unit]
Description=Livepatch server

[Service]
Type=simple
ExecStart=/usr/local/bin/livepatchd /etc/livepatchd.yaml
Restart=always
RestartSec=10
StartLimitBurst=5
StartLimitInterval=0

[Install]
WantedBy=multi-user.target

:~# systemctl status livepatch-server
● livepatch-server.service - Livepatch server
     Loaded: loaded (/etc/systemd/system/livepatch-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-11-17 15:55:40 UTC; 2min 19s ago
---

...and a service that will start automatically on boot. Which, unless I'm missing some other functionality that should be negating the need for this modification, is desirable.

[Kian Parvin (kian-parvin)]

Aha, the behaviour that the server doesn't automatically start after the machine restarts is something I have noted before. I do agree that the behaviour seems unfriendly to the user so I don't see any reason why we can't add the [Install] section to the service file.

On another note, we have an updated charm that uses a Snap internally instead of just a binary (so that would automatically start on startup) and the charm uses the Juju operator framework instead of the reactive framework used by the current charm. I'll post a link to the guide for that when I update this bug report as the guide should be up in a week or two.