Missing workload=active records in repeatable 'show-status-log'

[Alex Lutay]

Hi,

I am using juju 2.9.38 with juju-agent 2.9.29. It might be no longer topical for juju 3+.
microk8s: MicroK8s v1.26.1 revision 4595 (stable channel).

Issue: during the bootstrap of mysql-k8s charm I was monitoring `juju show-status-log` and noticed missing/jumping records `workload=active` for the every next call.
It looks like juju shown the last statement only hiding the previous, which makes complex charm/test troubleshooting.

Here the example of two calls, but detailed log is also available in https://pastebin.canonical.com/p/D242yjSRgZ/

```ubuntu@taurus-dev:~$ juju show-status-log mysql-k8s/0 -n 100
Time Type Status Message
17 Feb 2023 13:39:33+01:00 juju-unit allocating
17 Feb 2023 13:39:54+01:00 juju-unit allocating Pulling image "jujusolutions/charm-base:ubuntu-22.04"
17 Feb 2023 13:39:54+01:00 workload waiting installing agent
17 Feb 2023 13:40:00+01:00 workload waiting agent initializing
17 Feb 2023 13:40:11+01:00 workload maintenance installing charm software
17 Feb 2023 13:40:11+01:00 juju-unit executing running install hook
17 Feb 2023 13:40:13+01:00 juju-unit executing running restart-relation-created hook
17 Feb 2023 13:40:13+01:00 juju-unit executing running database-peers-relation-created hook
17 Feb 2023 13:40:14+01:00 juju-unit executing running leader-elected hook
17 Feb 2023 13:40:15+01:00 juju-unit executing running database-storage-attached hook
17 Feb 2023 13:40:16+01:00 juju-unit executing running config-changed hook
17 Feb 2023 13:40:16+01:00 juju-unit executing running start hook
17 Feb 2023 13:40:17+01:00 workload unknown
17 Feb 2023 13:40:17+01:00 juju-unit executing running mysql-pebble-ready hook
17 Feb 2023 13:40:19+01:00 juju-unit idle
17 Feb 2023 13:40:19+01:00 workload maintenance Initialising mysqld
17 Feb 2023 13:40:28+01:00 workload active Unit is ready: Mode: RW <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<!!!!!!!!!!
17 Feb 2023 13:40:29+01:00 juju-unit executing running database-peers-relation-changed hook
17 Feb 2023 13:40:31+01:00 juju-unit executing running database-peers-relation-joined hook for mysql-k8s/1
17 Feb 2023 13:40:47+01:00 juju-unit executing running restart-relation-joined hook for mysql-k8s/1
17 Feb 2023 13:40:48+01:00 juju-unit executing running database-peers-relation-changed hook for mysql-k8s/1
17 Feb 2023 13:40:48+01:00 juju-unit executing running restart-relation-changed hook for mysql-k8s/1
17 Feb 2023 13:40:49+01:00 juju-unit executing running restart-relation-joined hook for mysql-k8s/2
17 Feb 2023 13:40:50+01:00 juju-unit executing running restart-relation-changed hook for mysql-k8s/2
17 Feb 2023 13:40:50+01:00 juju-unit executing running database-peers-relation-changed hook
17 Feb 2023 13:40:51+01:00 juju-unit executing running database-peers-relation-joined hook for mysql-k8s/2

```

The following line is missing in a the next call:

> 17 Feb 2023 13:40:28+01:00 workload active Unit is ready: Mode: RW

```
ubuntu@taurus-dev:~$ juju show-status-log mysql-k8s/0 -n 100
Time Type Status Message
17 Feb 2023 13:39:33+01:00 juju-unit allocating
17 Feb 2023 13:39:54+01:00 juju-unit allocating Pulling image "jujusolutions/charm-base:ubuntu-22.04"
17 Feb 2023 13:39:54+01:00 workload waiting installing agent
17 Feb 2023 13:40:00+01:00 workload waiting agent initializing
17 Feb 2023 13:40:11+01:00 workload maintenance installing charm software
17 Feb 2023 13:40:11+01:00 juju-unit executing running install hook
17 Feb 2023 13:40:13+01:00 juju-unit executing running restart-relation-created hook
17 Feb 2023 13:40:13+01:00 juju-unit executing running database-peers-relation-created hook
17 Feb 2023 13:40:14+01:00 juju-unit executing running leader-elected hook
17 Feb 2023 13:40:15+01:00 juju-unit executing running database-storage-attached hook
17 Feb 2023 13:40:16+01:00 juju-unit executing running config-changed hook
17 Feb 2023 13:40:16+01:00 juju-unit executing running start hook
17 Feb 2023 13:40:17+01:00 workload unknown
17 Feb 2023 13:40:17+01:00 juju-unit executing running mysql-pebble-ready hook
17 Feb 2023 13:40:19+01:00 juju-unit idle
17 Feb 2023 13:40:19+01:00 workload maintenance Initialising mysqld
17 Feb 2023 13:40:29+01:00 juju-unit executing running database-peers-relation-changed hook
17 Feb 2023 13:40:31+01:00 juju-unit executing running database-peers-relation-joined hook for mysql-k8s/1
17 Feb 2023 13:40:47+01:00 juju-unit executing running restart-relation-joined hook for mysql-k8s/1
17 Feb 2023 13:40:48+01:00 juju-unit executing running database-peers-relation-changed hook for mysql-k8s/1
17 Feb 2023 13:40:48+01:00 juju-unit executing running restart-relation-changed hook for mysql-k8s/1
17 Feb 2023 13:40:49+01:00 juju-unit executing running restart-relation-joined hook for mysql-k8s/2
17 Feb 2023 13:40:50+01:00 juju-unit executing running restart-relation-changed hook for mysql-k8s/2
17 Feb 2023 13:40:50+01:00 juju-unit executing running database-peers-relation-changed hook
17 Feb 2023 13:40:51+01:00 juju-unit executing running database-peers-relation-joined hook for mysql-k8s/2
17 Feb 2023 13:41:07+01:00 juju-unit executing running database-peers-relation-changed hook for mysql-k8s/2
17 Feb 2023 13:41:08+01:00 juju-unit executing running database-peers-relation-changed hook for mysql-k8s/1
17 Feb 2023 13:41:08+01:00 juju-unit executing running database-peers-relation-changed hook
17 Feb 2023 13:41:10+01:00 juju-unit executing running database-peers-relation-changed hook for mysql-k8s/2
17 Feb 2023 13:41:09+01:00 workload active Unit is ready: Mode: RW <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<!!!!!!!!!!
```

The only last workload=active record is shown (13:41:09) but the previous one is missing (13:40:28). Is it expected? Tnx!

[Juan M. Tirado (tiradojm)]

Is this something you can easily reproduce? I mean, if you rerun this from time to time you see this behaviour?

[Joseph Phillips (manadart)]

Looks to be by design. We don't add a new record if the status has not changed. We just update the "since" field.

https://github.com/juju/juju/blob/5132716900d95187e8f7e90809f53876e6e52a8b/state/status.go#L322