Juju sets incorrect architecture for charms when deploying on arm64

[Simon Fels]

I am deploying to a local LXD on an arm64 machine. Deploying from a bundle works fine and I get everything up and running:

ubuntu@hayzee:~/anbox-cloud_1.12.5-alpha.1884+git1820ea6$ juju status
Model Controller Cloud/Region Version SLA Timestamp
default test-controller localhost/localhost 2.9.22 unsupported 08:53:45Z

App Version Status Scale Charm Store Channel Rev OS Message
ams 1.12.5 active 3 ams local 0 ubuntu
ams-node-controller 1.12.5 active 1 ams-node-controller local 0 ubuntu
etcd 3.4.5 active 3 etcd charmstore stable 634 ubuntu Healthy with 3 known peers
etcd-ca 3.0.1 active 1 easyrsa charmstore stable 420 ubuntu Certificate Authority connected.
lxd 4.0.8 active 1 lxd local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message
ams/0 active idle 0 10.93.207.156 8444/tcp,9104/tcp
ams/1* active idle 1 10.93.207.125 8444/tcp,9104/tcp
ams/2 active idle 2 10.93.207.71 8444/tcp,9104/tcp
etcd-ca/0* active idle 0 10.93.207.156 Certificate Authority connected.
etcd/0 active idle 0 10.93.207.156 2379/tcp Healthy with 3 known peers
etcd/1* active idle 1 10.93.207.125 2379/tcp Healthy with 3 known peers
etcd/2 active idle 2 10.93.207.71 2379/tcp Healthy with 3 known peers
lxd/2* active idle 4 10.93.207.196 8443/tcp
  ams-node-controller/0* active idle 10.93.207.196 10000-11000/tcp

Machine State DNS Inst id Series AZ Message
0 started 10.93.207.156 juju-7ab6e7-0 focal Running
1 started 10.93.207.125 juju-7ab6e7-1 focal Running
2 started 10.93.207.71 juju-7ab6e7-2 focal Running
4 started 10.93.207.196 juju-7ab6e7-4 focal Running

However when I now want to add additional units for LXD I get the following error:

ubuntu@hayzee:~/anbox-cloud_1.12.5-alpha.1884+git1820ea6$ juju add-unit lxd
ERROR cannot assign unit "lxd/3" to machine: cannot assign unit "lxd/3" to new machine or container: cannot assign unit "lxd/3" to new machine: invalid constraint value: arch=amd64
valid values are: [arm64 armhf]

Changing the app constraints is not allowed either

ubuntu@hayzee:~/anbox-cloud_1.12.5-alpha.1884+git1820ea6$ juju set-constraints lxd arch=arm64
ERROR changing architecture not supported (not supported)

The only workaround is to add a machine and then map a new unit to it

ubuntu@hayzee:~/anbox-cloud_1.12.5-alpha.1884+git1820ea6$ juju add-machine
created machine 5
ubuntu@hayzee:~/anbox-cloud_1.12.5-alpha.1884+git1820ea6$ juju add-unit lxd --to 5

I understand that charms with Juju 2.9 are now architecture specific but it would be great if we can correctly autodetect what architecture the application is deployed on initially so that subsequent calls to `juju add-unit` just work.

This is with Juju 2.9.22 and LXD 4.0.9

[Simon Richardson (simonrichardson)]

So this happens because the default model has the constraint of amd64. If you bootstrap the controller with an arch constraint, we don't inherit that when adding a model without specifying a constraint (maybe we should?).

You can't change a constraint of a given model, as we may have existing charmhub applications with amd64, so that violates our homogeneous topology for architecture constraints. One possible solution here would be to download a charm with the new arch when a constraint is employed, assuming it exists. This sounds easy but would require some changes to applications i.e. the charmURL as the opaque identifier.

From my experimentation the following works:

 juju bootstrap aws/eu-west-1 test3 --bootstrap-constraints="arch=arm64" --constraints="arch=arm64"
 juju deploy ubuntu
 juju add-unit ubuntu -n 1

The following all works, because we set the default model constraint to arm64. Everything correctly inherits from the application if it's set or via the model.

As a workaround, I would suggest:

 juju add-model other --constraints="arch=arm64"

I'll keep investigating to validate that the LXD provider does correctly return the architectures.

[Simon Richardson (simonrichardson)]

LXD does correctly set the vocabulary for arches in the following PR[1]

1. https://github.com/juju/juju/pull/12383

[Simon Richardson (simonrichardson)]

I think the fix here, is to allow adding constraints when adding a model. If the constraints for arch is empty, then use the os.HostArch to set it as default. We won't have broken anything for homogeneous deployments and we'll have a more sensible architecture. Heterogeneous deployments are still in the same position they're always have been.

If the user wants to change it they can either specify it with add-model command or if nothing is deployed, you can always use juju set-model-constraints.

[Simon Fels (morphis)]

Thanks Simon for looking into this!

I can confirm that working around with settings the model constraints helps

[Ian Booth (wallyworld)]

Closing this bug (since there's a work around for the reported issue) but I've opened a new one for the add-model constraints work.

https://bugs.launchpad.net/juju/+bug/1969402