docker login failed, see juju debug-log

[Joseph Borg]

I'm seeing "docker login failed, see juju debug-log" on each docker unit when deploying with a registry. I've tested the registry login with the test shown on https://ubuntu.com/kubernetes/docs/docker-registry, which works fine.

Looking at the logs, there is nothing obvious.

unit-docker-1: 14:51:13 INFO unit.docker/1.juju-log Reactive main running for hook update-status
unit-docker-1: 14:51:13 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/container_runtime_common.py:15:enable_grub_cgroups
unit-docker-1: 14:51:13 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:181:publish_config
unit-docker-1: 14:51:13 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:264:fix_iptables_for_docker_1_13
unit-docker-1: 14:51:13 INFO juju.worker.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)
unit-docker-1: 14:55:59 INFO unit.docker/1.juju-log Reactive main running for hook update-status
unit-docker-1: 14:55:59 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/container_runtime_common.py:15:enable_grub_cgroups
unit-docker-1: 14:55:59 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:181:publish_config
unit-docker-1: 14:55:59 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:264:fix_iptables_for_docker_1_13
unit-docker-1: 14:55:59 INFO juju.worker.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)
unit-docker-1: 15:00:58 INFO unit.docker/1.juju-log Reactive main running for hook update-status
unit-docker-1: 15:00:58 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/container_runtime_common.py:15:enable_grub_cgroups
unit-docker-1: 15:00:58 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:181:publish_config
unit-docker-1: 15:00:58 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:264:fix_iptables_for_docker_1_13
unit-docker-1: 15:00:59 INFO juju.worker.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)
unit-docker-1: 15:05:03 INFO unit.docker/1.juju-log Reactive main running for hook update-status
unit-docker-1: 15:05:03 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/container_runtime_common.py:15:enable_grub_cgroups
unit-docker-1: 15:05:03 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:181:publish_config
unit-docker-1: 15:05:03 INFO unit.docker/1.juju-log Invoking reactive handler: reactive/docker.py:264:fix_iptables_for_docker_1_13
unit-docker-1: 15:05:04 INFO juju.worker.uniter.operation ran "update-status" hook (via explicit, bespoke hook script)

[George Kraft (cynerva)]

I can repro this. Deployed cs:~containers/charmed-kubernetes with overlay:

applications:
  containerd: null
  docker:
    charm: cs:~containers/docker
relations:
- - docker
  - kubernetes-master
- - docker
  - kubernetes-worker

Deployed and related docker-registry:

juju deploy cs:~containers/docker-registry
juju add-relation docker-registry easyrsa:client
juju config docker-registry \
  auth-basic-user='admin' \
  auth-basic-password='password'
juju relate docker-registry docker

Same symptom as original report, docker units in blocked/idle state with "docker login failed, see juju debug-log" and nothing informative in debug-log.

The docker login calling code[1] captures stdout/stderr but does not log it on error.

[1]: https://github.com/charmed-kubernetes/charm-docker/blob/b24b84d0fc2f1cc837a2d97b100d5cf263cecd08/reactive/docker.py#L850-L863