upgrade to from 18.06.1 to 18.09.7 failed

[Laurent Sesquès]

This is on a xenial host, with ESM updates (all packages up-to-date).
Upgrade from 18.06.1-0ubuntu1.2~16.04.1 to 18.09.7-0ubuntu1~16.04.9+esm1.
Due to dependency changes, the upgrade also installs containerd 1.2.6-0ubuntu1~16.04.6 and runc 1.0.0~rc7+git20190403.029124da-0ubuntu1~16.04.4+esm2.

Steps:
After upgrading docker.io (see apt-install-logs.txt), the containers stop working.
Docker shows errors in its journal (see journal-logs.txt).
A reboot allows docker to return to normal operations and containers are back.

[Romain Couturat (romaincout)]

Notably after the upgrade an old version docker-containerd-shim process was still running :
 root 13189 1 0 2020 ? 02:10:52 docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc

Docker would not restart properly after killing process 13189 but it worked with a reboot.

As I suspect it could be related to kernel (cgroup abilities for example?)
please note that if the latest installed kernel was linux-image-4.4.0-217-generic (4.4.0-217.250)
the running kernel was 4.4.0-148 :
$ uname -a
Linux hostnameRedacted 4.4.0-148-generic #174-Ubuntu SMP Tue May 7 12:20:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$