fail to upgrade juju controller on top of charmed k8s
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Canonical Juju |
Fix Committed
|
Critical
|
Ian Booth |
Bug Description
SolQA deployed Charmed Kubernetes 1.30 and bootstrap a juju controller on top of it, with juju 3.4.4, latest 3.4/stable. And there's no other models on this controller.
with "juju --debug upgrade-controller" I got below error:
# juju --debug upgrade-controller -c foundations-
20:30:08 INFO juju.cmd supercommand.go:56 running juju [3.4.4 5abe6dd51d9e7f8
20:30:08 DEBUG juju.cmd supercommand.go:57 args: []string{
20:30:08 INFO juju.juju api.go:86 connecting to API addresses: [10.152.
20:30:08 INFO juju.kubernetes
20:30:08 DEBUG juju.api apiclient.go:645 starting proxier for connection
20:30:08 DEBUG juju.api apiclient.go:649 tunnel proxy in use at localhost on port 42159
20:30:08 DEBUG juju.api apiclient.go:825 looked up localhost -> [127.0.0.1]
20:30:08 DEBUG juju.api apiclient.go:1172 successfully dialed "wss://
20:30:08 INFO juju.api apiclient.go:707 connection established to "wss://
20:30:08 INFO juju.juju api.go:86 connecting to API addresses: [10.152.
20:30:08 DEBUG juju.api apiclient.go:645 starting proxier for connection
20:30:08 DEBUG juju.api apiclient.go:649 tunnel proxy in use at localhost on port 40341
20:30:08 DEBUG juju.api apiclient.go:1172 successfully dialed "wss://
20:30:08 INFO juju.api apiclient.go:1074 cannot resolve "localhost": lookup localhost: operation was canceled
20:30:08 INFO juju.api apiclient.go:707 connection established to "wss://
20:30:08 INFO juju.juju api.go:86 connecting to API addresses: [10.152.
20:30:08 DEBUG juju.api apiclient.go:645 starting proxier for connection
20:30:08 DEBUG juju.api apiclient.go:649 tunnel proxy in use at localhost on port 36031
20:30:08 DEBUG juju.api apiclient.go:1172 successfully dialed "wss://
20:30:08 INFO juju.api apiclient.go:1074 cannot resolve "localhost": lookup localhost: operation was canceled
20:30:08 INFO juju.api apiclient.go:707 connection established to "wss://
20:30:10 DEBUG juju.api monitor.go:35 RPC connection died
20:30:10 DEBUG juju.api monitor.go:35 RPC connection died
20:30:10 DEBUG juju.api monitor.go:35 RPC connection died
20:30:10 DEBUG juju.cmd.
ERROR cannot get architecture for jujud-operator:
20:30:10 INFO cmd supercommand.go:556 command finished
# juju show-controller foundations-
foundations-
details:
controller-
api-endpoints: ['10.152.
cloud: kubernetes_cloud
agent-version: 3.4.4
agent-
controller-
mongo-version: 4.4.18
ca-fingerprint: 25:67:DD:
ca-cert: |
-----BEGIN CERTIFICATE-----
MIIEEzCCA
CwUAMCExD
MTkxMzU0W
EwdqdWp1L
cCXVo1NT8
fFzgClDrX
KABPDzMgH
A531G8Znx
LOXPMDLb6
w4/
wpPPZXzQn
RuSNby8IV
QjBAMA4GA
Jm3fPHdtO
K/
MvjxoyenX
ZPyzKn0f2
9RLBUt3Y2
I2oGEmiXz
r4nAdfuWB
1rfkwL5vg
e5HDhcTJx
-----END CERTIFICATE-----
models:
controller:
model-uuid: cff75055-
unit-count: 1
current-model: admin/controller
account:
user: admin
access: superuser
description: | updated |
Changed in juju: | |
assignee: | nobody → Ian Booth (wallyworld) |
status: | Triaged → In Progress |
Changed in juju: | |
status: | In Progress → Fix Committed |
Changed in juju: | |
milestone: | 3.4.5 → 3.4.6 |
I tested with the 3.4 snap and confirmed an upgrade from 3.4.3 breaks. It seems the docker client we use to publish the manifests may have changed, since we don't support the fat manifest header and that's what we're now getting.
$ juju status
Model Controller Cloud/Region Version SLA Timestamp
controller test microk8s/localhost 3.4.3 unsupported 10:33:57+10:00
App Version Status Scale Charm Channel Rev Address Exposed Message
controller active 1 juju-controller 3.4/stable 101 no
Unit Workload Agent Address Ports Message
controller/0* active idle 10.1.78.73 37017/TCP
$ juju upgrade-controller 3.4.3: can not get manifests for jujud-operator: 3.4.3: manifest response version "application/ vnd.docker. distribution. manifest. list.v2+ json" not supported (not supported)
ERROR cannot get architecture for jujud-operator: