stuck k8s workload unit following upgrade-charm with new image
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Canonical Juju |
Fix Released
|
High
|
Yang Kelvin Liu |
Bug Description
Following a "juju upgrade-charm --path /local/mycharm --resource mycharm_
[agnew(~)] snap info juju | grep ^installed:
installed: 2.8-rc1 (11992) 134MB classic
[agnew(~)] juju version
2.8-rc1-focal-amd64
[agnew(~)] juju debug-log --no-tail
application-
application-
application-
application-
application-
application-
application-
application-
application-
application-
ERROR failed to remove unit tools dir /var/lib/
: command terminated with exit code 1
[agnew(~)] juju status --format yaml
model:
name: mattermost
type: caas
controller: beta
cloud: k8s
region: localhost
version: 2.8-rc1
model-status:
current: available
since: 13 May 2020 15:17:30+12:00
sla: unsupported
machines: {}
applications:
mattermost:
charm: local:kubernete
series: kubernetes
os: kubernetes
charm-origin: local
charm-name: mattermost
charm-rev: 3
charm-version: 89783cd-dirty
scale: 1
provider-id: 4fb6cece-
address: 10.152.183.25
exposed: false
application
current: active
since: 13 May 2020 15:46:04+12:00
units:
mattermost/0:
current: terminated
message: unit stopped by the cloud
since: 13 May 2020 15:33:04+12:00
current: idle
since: 13 May 2020 15:33:04+12:00
open-ports:
- 8000/TCP
address: 10.1.1.24
mattermost/1:
current: waiting
message: Not a leader
since: 13 May 2020 15:33:02+12:00
current: idle
since: 13 May 2020 15:38:51+12:00
leader: true
open-ports:
- 8000/TCP
address: 10.1.1.25
storage: {}
controller:
timestamp: 15:48:51+12:00
[agnew(~)] kubectl get all -n mattermost
NAME READY STATUS RESTARTS AGE
pod/mattermost-
pod/mattermost-
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mattermost ClusterIP 10.152.183.25 <none> 8000/TCP 30m
service/
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.
NAME DESIRED CURRENT READY AGE
replicaset.
NAME READY AGE
statefulset.
[agnew(~)] _
summary: |
- stuck k8s workload unit following upgrade-charm with new imagea + stuck k8s workload unit following upgrade-charm with new image |
Changed in juju: | |
milestone: | none → 2.8-rc2 |
importance: | Undecided → High |
status: | New → Triaged |
Changed in juju: | |
assignee: | nobody → Yang Kelvin Liu (kelvin.liu) |
status: | Triaged → In Progress |
Changed in juju: | |
status: | In Progress → Fix Committed |
Changed in juju: | |
status: | Fix Committed → Fix Released |
The root cause here is the issue identified in bug 1877935
Juju needs to take account the pod's phase - whether it is still running init containers - before deciding how to interact with it.