cannot assign unit "jenkins/0" to new machine: invalid constraint value: arch=amd64 valid values are: [arm64 armhf]

[Patricia Domingues]

Not able to deploy new charms on my juju 2.9.5 localhost cloud (LXD):
```
~$ juju status
Model Controller Cloud/Region Version SLA Timestamp
default localhost-localhost localhost/localhost 2.9.5 unsupported 14:42:55Z

App Version Status Scale Charm Store Channel Rev OS Message
jenkins error 0/1 jenkins charmhub stable 34 ubuntu cannot assign unit "jenkins/0" to machine: cannot assign unit "jenkins/0" to new machine or container: cannot assign unit "jenkins/0" to new machine: invalid constraint value: arch=amd64
valid values are: [arm64 armhf]

Unit Workload Agent Machine Public address Ports Message
jenkins/0 error lost cannot assign unit "jenkins/0" to machine: cannot assign unit "jenkins/0" to new machine or container: cannot assign unit "jenkins/0" to new machine: invalid constraint value: arch=amd64
valid values are: [arm64 armhf]
```

I have juju on AARCH64/arm64 server, juju recognizes it for installing its controller:

```
Creating Juju controller "localhost-localhost" on localhost/localhost
Looking for packaged Juju agent version 2.9.5 for arm64
Located Juju agent version 2.9.5-ubuntu-arm64
```

```
~$ juju status
Model Controller Cloud/Region Version SLA Timestamp
controller localhost-localhost localhost/localhost 2.9.5 unsupported 14:44:27Z

Machine State DNS Inst id Series AZ Message
0 started 10.166.202.54 juju-31e9e2-0 focal Running
```

but when I try to deploy I get the `invalid constraint value: arch=amd64`

[John A Meinel (jameinel)]

For non amd64 workloads you generally have to specify an arch constraint. "juju deploy myapp --constraints arch=arm64"

This is required because with Charmhub there are different charm packages to download based on whether it is amd64 or arm64, and rather than trying to guess we just mandate that an architecture be set, but we might as well have a default for the common amd64 case.

[Patricia Domingues (patriciasd)]

it does work if I run `juju deploy jenkins --constraints="arch=arm64"`, but my point is it did work in the past without adding the architecture constraint (juju version 2.8.11), and also, couldn't juju recognize it is in a arm64 (as it worked for bootstrapping) and overwriting the architecture?

[Simon Richardson (simonrichardson)]

This was a known breaking change between 2.8.x and 2.9.x.

I think we can be a bit more clever with this, maybe consider the controller as well, but in reality, this is known and documented clearly when 2.9.x was released.

See "Architecture and series constraint selection for charms" from https://discourse.charmhub.io/t/juju-2-9-0-release-notes/4525 and "Architecture" from https://discourse.charmhub.io/t/charmhub/3991 (there are more documentation that I've added).

So I believe we've let people know here.

[Harry Pidcock (hpidcock)]

Regardless if this is a known limitation, I believe it is still a valid concern. With the rise of arm in the server space and dominance in edge/IoT, I think we should aim to fix this behavior where possible and remove the amd64 guessing (deferring to the user to set a model default for example).

[Ian Booth (wallyworld)]

I think Juju can do better here. Juju knows the arch of the machine on which the charm is to be deployed. It knows because either a placement directive is used to assign the charm to a pre-existing machine, or Juju will spin up a new machine with either a user specified or a default arch. Yes, charmhub now needs Juju to ask for a charm with the arch specified, but Juju should be able to figure out what that is. And if now, it should fail early and not wait till later to surface the error in status.
Yes, I know that the deploy workflow has Juju downloading the charm early, and only later assigning the charm to a machine, but that's not the user's problem to deal with.

[Ian Booth (wallyworld)]

To clarify a bit more, consider

Case 1

$ juju add-machine --constraints "arch=arm64"
Machine 2 added.
$ juju deploy foo --to 2

Juju knows that it is an arm64 machine so it should ask charmhub for an arm64 charm and fail early if one doesn't exist.

Case 2

$ juju set-model-constraints "arch=arm64"
$ juju deploy foo

Juju will use the model constraints set by the user to spin up arm64 machines whenever a new machine is needed, so again, Juju knows what arch should be used.

Case 3

$ juju deploy foo

Here, Juju will spin up a machine but doesn't know ahead of time what the arch will be. But once the arch is known, Juju can then ask for the relevant charm. The fact that Juju defaults up front to "amd64" and doesn't take account of the ultimate target machine arch is a bug IMO. Just because charms used to be arch agnostic and now they are now is not the user's problem to solve, when Juju does ultimately have all the info needed to make the required decision.

[Ian Booth (wallyworld)]

Looks like Harry and I were both editing the bug at the same time - I didn't see his comment before posting mine but we are both in broad agreement it seems :-)

[Harry Pidcock (hpidcock)]

Regarding Case #3 this could affect the possible introduction of default constraints into charm metadata/manifest. Possibly this means we either force a charm to specify default constraints on a per channel basis or defer to the user when default constraints are ambiguous.

[Ian Booth (wallyworld)]

This is related to bug 1969402 which talks about using model constraints as a partial workaround.
At this stage, any fix will not be in 2.9 so I'll remove the milestone.