list-controllers shows HA: none for a HA enabled controller environment

[Gareth Woolridge]

On a freshly bootstrapped controller environment I run juju ensure-ha, then verify I have 3 controller machines in a healthy state using "juju show-controller".

"juju list-controllers" still shows HA as "none" for this controller.

Should we expect this field to show the controller is in fact HA enabled, or am I misinterpreting this field?

The following shows the output I am seeing:

juju --version
2.0.0-xenial-amd64

dpkg -l | grep juju
ii juju 1:2.0.0-0ubuntu1~16.04.2~juju1 all next generation service orchestration system
ii juju-2.0 1:2.0.0-0ubuntu1~16.04.2~juju1 amd64 Juju is devops distilled - client

juju list-controllers
Use --refresh to see the latest information.

Controller Model User Access Cloud/Region Models Machines HA Version
juju2-test* controller admin superuser localhost/localhost 2 1 none 2.0.0

juju show-controller
juju2-test:
  details:
    uuid: 1120e961-44bd-4c11-84ec-9bea83048f49
    api-endpoints: ['10.91.207.234:17070', '10.91.207.119:17070', '10.91.207.28:17070']
    ca-cert: |
      -----BEGIN CERTIFICATE-----
       <cert redacted>
      -----END CERTIFICATE-----
    cloud: localhost
    region: localhost
    agent-version: 2.0.0
  controller-machines:
    "0":
      instance-id: juju-60691e-0
      ha-status: ha-enabled
    "1":
      instance-id: juju-60691e-1
      ha-status: ha-enabled
    "2":
      instance-id: juju-60691e-2
      ha-status: ha-enabled
  models:
    controller:
      uuid: d67d7d7f-6e1a-45b4-8d79-511e2060691e
      machine-count: 3
    default:
      uuid: a72d1217-ae7f-4fb5-8225-dcfd7b345843
  current-model: admin/controller
  account:
    user: admin
    access: superuser

[Ian Booth (wallyworld)]

Controller information is cached locally and when list controllers is run, no API call is made to the controller to avoid a stuck system preventing the command from completing. But it does prompt you to --refresh to see the latest information.

$ juju list-controllers
Use --refresh to see the latest information.
...

[James Troup (elmo)]

Sorry, but it's not OK to require --refresh to get current & useful information.

[Tom Haddon (mthaddon)]

Erm, so the data is out of date by design in case refreshing it causes a problem in the system? That doesn't seem like an ideal situation to me, and certainly doesn't seem like an invalid bug if that's the case.

[Ian Booth (wallyworld)]

This design was agreed to with Mark.
We must not allow list-controllers to block because of a stuck controller.
The addition of the --refresh option seemed like a fair compromise.

[Tom Haddon (mthaddon)]

Surely you know when someone has run "juju enable-ha" so why not update the status at that point?

[James Troup (elmo)]

Sorry, but I think you're conflating two things. I agree
list-controllers shouldn't block. I don't agree it logically follows
that the only path forward is to have a --refresh argument. It could,
for example, try a refresh and fail gracefully if it can't talk to the
controller in a reasonable amount of time.

Also, I don't believe Mark signing off on a design is, in of itself, a
reason to mark a bug as 'invalid'.

[Richard Harding (rharding)]

I think there are improvements here. I think we can look into also getting updated information if a call is made to an api server. If you're running status, etc then it's probably an ok time to refresh any known status information. It's true that the controller can't call back with updated info in a true async way, especially for something like HA which might take a very long time to bring up new machines.

Does running show-controller hit the API server at all? It seems like that might be a legit time to also refresh information for the default controller output?

[Ian Booth (wallyworld)]

We can refresh the local controller info when other API calls are made.
But there can be no guarantee that the list-controllers information won't ever be out of date unless list-controllers also makes API calls to each controller, which we were avoiding. Hence there will always be a need for a --refresh argument if we accept that approach. show-controller etc can pull down the info required and update the local controllers file, but bear in mind it will be for that one controller only, not all of them. If we make list-controllers make an API call to each controller, and it blocks, how long to we wait? 0.5 seconds? 1 second? 10 seconds? We wanted to avoid all that and provided an explicit refresh. If the user really wants list-controllers to refresh, despite the limitations, could an alias be used?

[Anastasia (anastasia-macmood)]

Marking as 'Incomplete" until we reach consensus on how to tackle this \o/

[James Troup (elmo)]

Err, sorry, I really don't mean to be difficult, but 'Incomplete' means 'waiting for information from the submitter' which is not the case here (AFAICS). Additionally, bugs in 'Incomplete' will be auto-expired after 60 days, which doesn't seem appropriate here.

[Tim Penhey (thumper)]

Removing the milestone to stop punting this each time we release.

Adding a card for next work though. The controller info should be updated at the very least when 'status' is called.

[Witold Krecicki (wpk)]

I've added refreshing of the local store for the controller when doing show-controller, that should be 'best of both worlds' in this case.
https://github.com/juju/juju/pull/7573