failed to retrieve image from rocks, no useful output or indication of retrying

Bug #1912812 reported by Jason Hobbs
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Calico Charm
Triaged
High
Unassigned

Bug Description

during the install hook, the charm failed to retrieve an image from ROCKS and we just get this traceback:

2021-01-22 01:06:49 DEBUG jujuc server.go:211 running hook tool "juju-log" for calico/1-install-948787281929013171
2021-01-22 01:06:49 ERROR juju-log Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-calico-1/charm/reactive/calico.py", line 662, in pull_calico_node_image
    CTL.pull(image)
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/conctl/containerd.py", line 139, in pull
    return self._exec(*args)
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/conctl/containerd.py", line 29, in _exec
    return super()._exec(*['ctr'] + list(args))
  File "/var/lib/juju/agents/unit-calico-1/.venv/lib/python3.8/site-packages/conctl/base.py", line 30, in _exec
    return sub_run(args, stdout=PIPE, stderr=PIPE, check=True)
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('ctr', 'image', 'pull', 'rocks.canonical.com:443/cdk/calico/node:v3.10.1')' returned non-zero exit status 1.

There is no useful output, i.e. an HTTP error code or something like that, and no indication that it retried either.

https://solutions.qa.canonical.com/testruns/testRun/ff25f9f1-c6df-4f46-a685-5ff8eab8f94e

description: updated
Revision history for this message
George Kraft (cynerva) wrote :

Looks like where the charm calls CTL.pull[1], it will need to catch the CalledProcessError, log stdout/stderr from the process, and then enter a Blocked state instead of Error.

[1]: https://github.com/charmed-kubernetes/layer-calico/blob/e8a3b6b81982ee8f5060c4584eb04897ac6bd6ff/reactive/calico.py#L664

Changed in charm-calico:
importance: Undecided → High
status: New → Triaged
Revision history for this message
Aymen Frikha (aym-frikha) wrote :

subscribed ~field-medium

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.