Comment 0 for bug 1867398

[Environment]

Kubernetes 1.16.17
Containerd 1.3.3
Ubuntu Bionic

[Description]

Reported upstream: https://github.com/containerd/containerd/issues/4108

The bump of to version 1.3.3 through [0] https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1854841

Caused a regression.

The following endpoint description works with containerd 1.2.X without defining
a protocol scheme. (/etc/containerd/config.toml).

    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."niedbalski-bastion.cloud.sts:5000"]
      endpoint = ["niedbalski-bastion.cloud.sts:5000"]
This stopped working on 1.3.X , scheduling pods with k8s 1.16-1.17 doesn't
works using the same registry mirror definition.

The pod definition is:

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
    - name: busybox
      image: niedbalski-bastion.cloud.sts:5000/busybox:latest
      command:
        - sleep
        - "3600"
  imagePullSecrets:
    - name: regcred
  restartPolicy: Always
New pods fail with the following error:

" failed to do request: Head niedbalski-bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol scheme "niedbalski-bastion.cloud.sts"

Normal Scheduled default-scheduler Successfully assigned default/busybox to juju-3a79d2-00268738-4
Normal Pulling 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"
Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Failed to pull image "niedbalski-bastion.cloud.sts:5000/busybox:latest": rpc error: code = Unknown desc = failed to pull and unpack image "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to resolve reference "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to do request: Head niedbalski-bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol scheme "niedbalski-bastion.cloud.sts"
Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ErrImagePull
Warning Failed 8m27s (x6 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ImagePullBackOff
Normal BackOff 4m56s (x21 over 10m) kubelet, juju-3a79d2-00268738-4 Back-off pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"

[Steps to reproduce]

Configure a private docker repository repository

Modify the containerd registry mirror config as follows:
** http://paste.ubuntu.com/p/yP63WMkVT6/

Execute the following pod (http://paste.ubuntu.com/p/BVYQFMfCmk/)

Status of the scheduled pod should be ImagePullBackOff
and the before mentioned error should be raised.

[Possible workaround and solution]

As a workaround change the endpoint to support the scheme (https://)
Provide a fallback mechanism for URL parsing validation to fallback to http or https.
I suspect that this change introduced on 1.3.3 through
0b29c9c) may be the offending commit.