juju fails to select machines if zones constraints is used

[Marian Gasparovic]

MaaS provider with six machines in three zones (default, zone2, zone3). Plus several KVMs which are pods on Maas nodes, those are created and enlisted before juju deploy

MAAS
Success.
Machine-readable output follows:
[
    {
        "name": "default",
        "description": "",
        "id": 1,
        "resource_uri": "/MAAS/api/2.0/zones/default/"
    },
    {
        "name": "zone2",
        "description": "",
        "id": 2,
        "resource_uri": "/MAAS/api/2.0/zones/zone2/"
    },
    {
        "name": "zone3",
        "description": "",
        "id": 3,
        "resource_uri": "/MAAS/api/2.0/zones/zone3/"
    }
]

bundle
machines:
  # KVMs
  "0":
    constraints: tags=nagios zones=default
    series: bionic
  "1":
    constraints: tags=grafana zones=default
    series: bionic
  "5":
    constraints: tags=elastic zones=default
  "9":
    constraints: tags=prometheus zones=zone3
    series: bionic
  "10":
    constraints: tags=graylog zones=default
    series: bionic
  "13":
    constraints: tags=elastic zones=zone2
  "15":
    constraints: tags=vault zones=default
  "16":
    constraints: tags=vault zones=zone2
  "17":
    constraints: tags=vault zones=zone3
  "18":
    constraints: tags=elastic zones=zone3
  "19":
    constraints: tags=graylog zones=zone2
    series: bionic
  "20":
    constraints: tags=graylog zones=zone3
    series: bionic

  # Baremetals
  "1000":
    constraints: tags=foundation-nodes zones=default
  "1001":
    constraints: tags=foundation-nodes zones=zone2
  "1002":
    constraints: tags=foundation-nodes zones=zone3
  "1003":
    constraints: tags=foundation-nodes zones=default
  "1004":
    constraints: tags=foundation-nodes zones=zone2
  "1005":
    constraints: tags=foundation-nodes zones=zone3

Note: when not using zones constraints, deploy works

Machine State DNS Inst id Series AZ Message
0 started 10.244.41.68 nagios-1 bionic default Deployed
1 started 10.244.41.7 grafana-1 bionic default Deployed
2 started 10.244.41.8 elastic-1 bionic default Deployed
3 started 10.244.41.74 prometheus-3 bionic zone3 Deployed
4 down pending bionic suitable availability zone for machine 4 not found
5 started 10.244.41.72 elastic-2 bionic zone2 Deployed
6 down pending bionic suitable availability zone for machine 6 not found
7 started 10.244.41.78 vault-2 bionic zone2 Deployed
8 started 10.244.41.76 vault-3 bionic zone3 Deployed
9 started 10.244.41.71 elastic-3 bionic zone3 Deployed
10 started 10.244.41.73 graylog-2 bionic zone2 Deployed
11 started 10.244.41.75 graylog-3 bionic zone3 Deployed
12 down pending bionic suitable availability zone for machine 12 not found
13 pending 10.244.41.81 spearow bionic zone2 Deploying: Rebooting
14 started 10.244.41.80 elgyem bionic zone3 Deployed
15 down pending bionic suitable availability zone for machine 15 not found
16 pending 10.244.41.82 ralts bionic zone2 Deploying: Rebooting

Note: when I redeploy several times I get different zones failing, not the same every time

If I remove KVM definition altogether, deploy works.

After discussion with Rick who pointed me to https://bugs.launchpad.net/juju/+bug/1860083 I changed the order of KVM machines to group zones, it did not help.

Another interesting esult was when I kept KVMs as before, I only removed 3 x graylog. And deployment worked. graylog KVMs are the only ones using lxd in KVM, it is the only difference I can see, no idea if it is relevant.

Juju 2.6.10, MaaS maas_2.6.2-7841-ga10625be3-0ubuntu1~18.04.1

[Ian Booth (wallyworld)]

Can you reproduce the issue on juju 2.7.2?
Unless a critical / security issue, we don't plan an any more 2.6 releases.

[Harry Pidcock (hpidcock)]

https://github.com/juju/juju/pull/11296 should fix this issue. It is a duplicate although the workaround described in lp1860083 only works in very specific configurations.

[Marian Gasparovic (marosg)]

I just tested it with 2.7.5+2.7-624eb99 and placement works as expected