[2.3-beta3] juju cannot bootstrap a controller with a tag constraint if no machine with that tag is present in a first AZ it uses

[Dmitrii Shcherbakov]

A controller machine is in the "default" AZ in maas with a "controller" tag.

availability_zones_in_maas = { default, az1, az2, az3 }

 juju bootstrap vmaas --bootstrap-constraints 'tags=controller'
Creating Juju controller "vmaas" on vmaas
Looking for packaged Juju agent version 2.3-beta3 for amd64
Launching controller instance(s) on vmaas...
ERROR failed to bootstrap model: cannot start bootstrap instance: cannot run instances: cannot run instance: No available machine matches constraints: [('agent_name', ['8fced0c1-b0f1-4baf-8aae-72b941372978']), ('tags', ['controller']), ('mem', ['3584']), ('zone', ['az1'])] (resolved to "mem=3584.0 tags=controller zone=az1")

OTOH, if a zone placement directive is specified then it progresses forward:

juju bootstrap vmaas --bootstrap-constraints 'tags=controller' --to zone=default
Creating Juju controller "vmaas" on vmaas
Looking for packaged Juju agent version 2.3-beta3 for amd64
Launching controller instance(s) on vmaas...
 - cmfcxx (arch=amd64 mem=4G cores=2)
Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.10.2

It would seem that the node selection algorithm should find a zone where a machine with a given tag is available.

So, for a more generic case of machine allocation (not just bootstrap):

- if no az is specified
- find an AZ for a given set of constraints (doesn't have to be just tags)
- error out if no such machine can be found

Not sure about AZ distribution fairness here - a search discipline might be a bit different but more expensive to calculate:

* find all matching nodes in all AZs (or n random AZs, where n is the number of machines to allocate if n <= num_AZs);
* iterate over AZs an pick a random node per AZ.

[Andrew Wilkins (axwalk)]

https://github.com/juju/juju/pull/8082