Regression in `goal-state` output, individual related unit status no longer listed

[Frode Nordahl]

When juju 2.4.0 was released the output from the `goal-state` hook tool looked something like this:
$ juju run --unit keystone/0 goal-state
units:
  keystone/0:
    status: active
    since: 2018-09-27 11:26:39Z
relations:
  identity-service:
    glance/0:
      status: active
      since: 2018-09-27 11:27:19Z
    glance/1:
      status: active
      since: 2018-09-27 11:27:34Z
    glance/2:
      status: active
      since: 2018-09-27 11:27:55Z
    glance/3:
      status: active
      since: 2018-09-27 11:27:28Z
    glance/4:
      status: waiting
      since: 2018-09-27 11:29:42Z
    keystone/0:
      status: active
      since: 2018-09-27 11:26:39Z
  shared-db:
    keystone/0:
      status: active
      since: 2018-09-27 11:26:39Z
    mysql/0:
      status: active
      since: 2018-09-27 11:13:29Z

In recent versions of juju the same deployment looks like this:
$ juju run --unit keystone/0 goal-state
units:
  keystone/0:
    status: active
    since: 2018-09-27 11:38:28Z
relations:
  identity-service:
    glance:
      status: joined
      since: 2018-09-27 11:37:16Z
    keystone:
      status: joined
      since: 2018-09-27 11:37:16Z
  shared-db:
    keystone:
      status: joined
      since: 2018-09-27 11:34:26Z
    mysql:
      status: joined
      since: 2018-09-27 11:34:26Z

There are a few data points we have lost in this regression:
- We no longer know the number of units that will join a given relation.
- The status of `joined` is printed regardless of all units having completed joining the relation or not where as previously we could quickly query each individual units status.

For charms that are both clustered and are related to many units, such as the `keystone`, `percona-cluster` and `rabbitmq-server` charms having the above mentioned data is invaluable. These charms have to do a fair bit of per-unit relation housekeeping based on changes to their cluster relation and based on changes to their dependencies (for example full completion of the shared-db relation).

What is the reason for the change and is there a way for us to get the old behavior back?

[Richard Harding (rharding)]

When you say "recent versions" can you be more specific to help us chase down the changes? Is this the edge snap of 2.5 in dev or from 2.4.0 to recent 2.4.3?

[Frode Nordahl (fnordahl)]

The original behavior was in juju 2.4.0, which was where we started preparations for development. I confirmed this for this bug report by using juju from the stable PPA on Artful.

(I am well aware of that being EOL, but I had to make sense of why our initial work for this no longer worked as expected :) )

The changed behavior is observed in the juju 2.4.3 stable snap. The changed behavior is also present in the 2.5 edge snap.

Unfortunately I do not have juju binaries for the versions in between readily available, so I have not made an attempt at bisecting any further.

[Richard Harding (rharding)]

Awesome, thanks for the details. I'll see what we can chase down. I can confirm we shouldn't have changed the behavior.

[Ian Booth (wallyworld)]

The change in output was because the original implementation did not adhere to the specification of the work. The specification called for the status of each *relation* to be displayed in the "relations" block. The original output showed instead the status of each *unit*. Since there's a need for additional information, we can add it back in. We'll need to consider where to add it since the "relations" block is for relation status.

[Ian Booth (wallyworld)]

We'll add back in the individual unit status items to the relation block, so you'll get to see both the relation status as originally intended, plus the unit status as requested. It will look something like the below. Note also we were incorrectly showing peer units in the relations section so this has been fixed also.

units:
  keystone/0:
    status: active
    since: 2018-09-27 11:38:28Z
relations:
  identity-service:
    glance:
      status: joined
      since: 2018-09-27 11:37:16Z
    glance/0:
      status: active
      since: 2018-09-27 11:27:19Z
    glance/1:
      status: active
      since: 2018-09-27 11:27:34Z
    glance/2:
      status: active
      since: 2018-09-27 11:27:55Z
    glance/3:
      status: active
      since: 2018-09-27 11:27:28Z
    glance/4:
      status: waiting
      since: 2018-09-27 11:29:42Z
  shared-db:
    mysql:
      status: joined
      since: 2018-09-27 11:34:26Z
    mysql/0:
      status: active
      since: 2018-09-27 11:13:29Z

[Frode Nordahl (fnordahl)]

Perfect, thank you Ian.

We're now unblocked and we'll continue charm development based on your input. We'd be happy to test any changes as they land in edge so let us know.