charmed k8s deploys incorrect cni-plugins on arm64

[Alan Baghumian]

While deploying a test Charmed Kubernetes 1.29/stable cluster using the attached bundle on arm64 compute hosts, I noticed x86_64 cni-plugins were deployed causing kube-system pods to get stuck with /opt/cni/bin/loopback: cannot execute binary file: Exec format error

Looking deeper, realized the deployed binaries are infact incorrect:

ubuntu@juju-1feed7-amd-k8s-7:~$ file /opt/cni/bin/loopback
/opt/cni/bin/loopback: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=_LVCeNytSGB6n6GzjOV8/JGYRTZG3RFhfr4EGSA96/17j9GVUdDgfu2n5MPXIt/mfARfVVAaw_OETvMtcQ4, not stripped

The destination architecture is something we need to check and consider before unpacking cni-plugins package.

Thank you,
Alan

[Alan Baghumian (alanbach)]

The workaround for the issue was to deploy the cni-plugins manually from upstream on all worker nodes:

$ mkdir tmp; cd tmp
$ wget https://github.com/containernetworking/plugins/releases/download/v1.4.1/cni-plugins-linux-arm64-v1.4.1.tgz
$ tar -xf cni-plugins-linux-arm64-v1.4.1.tgz
$ sudo cp -rP /opt/cni/ /opt/cni.orig
$ sudo cp -v bandwidth bridge dhcp dummy firewall host-* ipvlan loopback macvlan portmap ptp sbr static tap tuning vlan vrf /opt/cni/bin/ -v
$ cd; rm -r tmp

[Alan Baghumian (alanbach)]

This turns out to be an issue with the bundle (revision: 440 line):

 kubernetes-control-plane:
    charm: kubernetes-control-plane
    channel: 1.29/stable
    revision: 440 # this is the amd64 version which will pull the amd64 cni resource
    num_units: 2
    to:
    - "5"
    - "6"
    options:
      channel: 1.29/stable

Most charms are arch independent, however in this occasion "revision: 440" pulls x86_64 artifacts.

[Alan Baghumian (alanbach)]

I am switching this bug back into "New". Redeployed the environment w/o specifying the revision landed me into the same state. Re-tried with adding revision 441, and it still deploys amd64 binaries on arm64.

I'm attaching the updated bundle.

Thanks much,
Alan

[Alan Baghumian (alanbach)]

OK, going over this again, seems like the issue was due to missing correct arm64 constraints at the bundle level. Example:

  kubernetes-worker:
    charm: kubernetes-worker
    channel: 1.29/stable
    constraints: "arch=arm64"
    num_units: 3

Switching this back to invalid.