No way to re-query tags (machine, storage) from MAAS after model creation - have to create a new model

[Dmitrii Shcherbakov]

This is not a regression from 2.1.2 as I experienced the same behavior on that version.

ubuntu-q87:/tmp$ juju --version
2.2-beta4-xenial-amd64

ubuntu@maas:~$ maas maas version read
Success.
Machine-readable output follows:
{"subversion": "bzr6054-0ubuntu1~16.04.1", "version": "2.2.0", "capabilities": ["networks-management", "static-ipaddresses", "ipv6-deployment-ubuntu", "devices-management", "storage-deployment-ubuntu", "network-deployment-ubuntu", "bridging-interface-ubuntu", "bridging-automatic-ubuntu", "authenticate-api"]}

juju deploy ./gluster-charm -n 3 --storage brick='glusterpool'

# forgot to add a tag to one of the disks hence juju complains about not being able to find a suitable machine

ubuntu@maas:~$ maas maas machines read | jq -r '.[] | select( .blockdevice_set[].tags[] | contains("jstorage")) | .hostname'
maas-xenial5
maas-xenial4

ubuntu-q87:/tmp$ juju status
Model Controller Cloud/Region Version SLA
default vmaas vmaas 2.2-beta4 unsupported

App Version Status Scale Charm Store Rev OS Notes
gluster waiting 0/3 gluster local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message
gluster/0 waiting allocating 0 10.10.101.58 waiting for machine
gluster/1 waiting allocating 1 10.10.101.59 waiting for machine
gluster/2 waiting allocating 2 waiting for machine

Machine State DNS Inst id Series AZ Message
0 pending 10.10.101.58 rynftc xenial default Deploying: ubuntu/amd64/ga-16.04/xenial/daily/boot-initrd
1 pending 10.10.101.59 rw7fq3 xenial default Deploying: ubuntu/amd64/ga-16.04/xenial/daily/boot-initrd
2 down pending xenial cannot run instances: cannot run instance: No available machine matches constraints: [('zone', ['default']), ('agent_name', ['0408803e-067a-492f-83c2-cf08f19815fc']), ('storage', ['root:0,2:1(jstorage)'])] (resolved to "storage=root:0,2:1(jstorage) zone=default")

# added a jstorage tag to one of the disks on maas-xenial6

ubuntu@maas:~$ maas maas machines read | jq -r '.[] | select( .blockdevice_set[].tags[] | contains("jstorage")) | .hostname'
maas-xenial5
maas-xenial4
maas-xenial6

ubuntu-q87:/tmp$ juju retry-provisioning 2
ubuntu-q87:/tmp$ juju status
Model Controller Cloud/Region Version SLA
default vmaas vmaas 2.2-beta4 unsupported

App Version Status Scale Charm Store Rev OS Notes
gluster 3.10.2 blocked 2/3 gluster local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message
gluster/0* blocked idle 0 10.10.101.58 No bricks found
gluster/1 blocked idle 1 10.10.101.59 No bricks found
gluster/2 waiting allocating 2 waiting for machine

Machine State DNS Inst id Series AZ Message
0 started 10.10.101.58 rynftc xenial default Deployed
1 started 10.10.101.59 rw7fq3 xenial default Deployed
2 down pending xenial cannot run instances: cannot run instance: No available machine matches constraints: [('zone', ['default']), ('agent_name', ['0408803e-067a-492f-83c2-cf08f19815fc']), ('storage', ['root:0,2:1(jstorage)'])] (resolved to "storage=root:0,2:1(jstorage) zone=default")

Relation Provides Consumes Type
server gluster gluster peer

Retry-provisioning is useless in this case and I have to take down the whole model in order to change this.

I've tried to restart a controller machine to check if there's a code path to do a re-sync on controller reboot but it does not seem to be the case.

I realize that it might be not trivial to re-query the state from MAAS automatically, therefore, I think a manual state sync is also an option.

[John A Meinel (jameinel)]

We just added something like this for "reload-spaces", we may need to consider something similar for refreshing storage information.
Most tags that we use are just passed through to maas (in the case of something like provisioning), which means Juju isn't keeping cached information that can get stale. Probably we are caching some of the storage information so we're failing to notice that things have been updated in MAAS.

[Dmitrii Shcherbakov (dmitriis)]

Something tells me that it may have to be even more generic.

In general, we might query information about provider-specific information like network spaces, storage tags (or other storage-related information), availability zones, security groups (OpenStack) and possibly other information that we haven't thought about yet:

https://bugs.launchpad.net/juju/+bug/1691825

---

This is a case of a distributed metadata storage and I can think of at least three ways of doing that:

1) manual sync - give a user manual control over that. Trade-off: manual work vs complexity of code;
2) polling - requery automatically in Juju with a subjective pre-defined policy. Trade-off: assumptions and policy maintenance;
3) watchers (event subscription) - subscribe to provider-specific events and update as soon as the new information is available. Trade-off: asynchronous code, providers do not support emission of events and client subscription to those in general (they would have to expose an interface like etcd, Zookeeper, consul etc.).

Realistically, I think it is right that we will go with the first approach. Could be something worth documenting though as people may have different expectations from Juju.

[Canonical Juju QA Bot (juju-qa-bot)]

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.