Docker snap cannot start containers in Classic (but does in Core)

Bug #1626019 reported by Ara Pulido on 2016-09-21
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
snap-confine
High
Zygmunt Krynicki

Bug Description

The docker snap (available in --devmode in the edge channel) can run containers successfully in all-snaps images.

But in classic, there seem to be a namespace issue, and the docker daemon cannot find the binaries in the container.

Test case to reproduce it.

1) sudo snap install --edge --devmode docker

2) sudo docker run hello-world

Expected results:

The "hello" command for the hello-world container runs successfully

Actual results:

docker: Error response from daemon: Container command '/hello' not found or does not exist..

This works fine in Core images

Zygmunt Krynicki (zyga) wrote :

I reproduced this issue myself but I didn't analyse the root cause. Jdstrand suggested the following:

01:03 < jdstrand> zyga: perhaps related to cgroups: ubuntu-core-launcher[31408]: time="2016-09-21T18:02:44-05:00" level=error msg="containerd: start container" error="oci runtime
                  error: could not synchronise with container process: stat
                  /tmp/snap.rootfs_XA9nN8/sys/fs/cgroup/systemd/docker/3fafa1d878809e4c01791ab3bface93344feca86859c37386052c95468f23303: no such file or directory"
                  id=3fafa1d878809e4c01791ab3bface93344feca86859c37386052c95468f23303

Jamie Strandboge (jdstrand) wrote :

This is an issue with '/tmp/snap.rootfs_...' in the leading path. Ie, if I create a docker snap with a new 'sh' command, I can get things to work with:

$ sudo docker.sh
bash-4.3# docker run ubuntu:trusty uptime
docker: Error response from daemon: Container command 'uptime' not found or does not exist..

bash-4.3# service snap.docker.dockerd stop

bash-4.3# cat /proc/self/mountinfo |grep rootfs | tail -1
1188 1171 0:40 / /tmp/snap.rootfs_Rh2CZH/sys/fs/fuse/connections rw,relatime master:26 - fusectl fusectl rw

bash-4.3# mkdir -p /tmp/snap.rootfs_Rh2CZH

bash-4.3# ln -s /sys /tmp/snap.rootfs_Rh2CZH/sys

bash-4.3# service snap.docker.dockerd start

bash-4.3# docker run ubuntu:trusty uptime
 18:19:29 up 4 days, 22:22, 0 users, load average: 0.56, 0.71, 0.85

Changed in snap-confine:
status: New → Triaged
Jamie Strandboge (jdstrand) wrote :

Here is a workaround in the snappy packaging for this issue:
https://github.com/jdstrand/docker-snap/commit/fec83575437fabd70ddd9c8ed6a78d6bd0e29569

Obviously we'll want to fix this properly in snap-confine, but this would at least make it work on classic without patching docker.

Zygmunt Krynicki (zyga) wrote :

After initially looking at the workaround proposed by jdstrand I was unsure how it can fix the problem as it semantically made no sense. I had a hunch that it might involve parsing /proc/self/mountinfo and it is indeed the case.

I'm not sure if there's anything we should do in snap-confine proper, from the architecture point of view. We can add a quirk that will perform the same task if the snap is called "docker" but the true problem is in the upstream docker code and should be reported and fixed there IMHO.

Zygmunt Krynicki (zyga) on 2016-09-28
Changed in snap-confine:
status: Triaged → Invalid
Loïc Minier (lool) wrote :

Reopening as discussed with Zygmunt as snap-confine should unmount unaccessible sysfs etc. after pivot_root – this seems to confuse docker

Changed in snap-confine:
assignee: nobody → Zygmunt Krynicki (zyga)
status: Invalid → In Progress
Zygmunt Krynicki (zyga) on 2016-11-24
Changed in snap-confine:
milestone: none → 1.0.45
status: In Progress → Fix Committed
importance: Undecided → High
Ara Pulido (ara) wrote :

Zygmunt, it would be useful to link here the pull request that fixes this issue. Thanks!

Loïc Minier (lool) wrote :

This is the one:
https://github.com/snapcore/snap-confine/pull/183

Albeit I'm not sure how to best link it to LP

Loïc Minier (lool) wrote :

Perhaps we should have a git mirror in LP and use LP #nnn in commit log to autolink?

On Thu, Nov 24, 2016 at 10:35 AM, Loïc Minier <email address hidden>
wrote:

> Perhaps we should have a git mirror in LP and use LP #nnn in commit log
> to autolink?
>
>
I think that is a bit overkill. When closing the bug we can add a comment
to the pull request.

Cheers,
Ara.

> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1626019
>
> Title:
> Docker snap cannot start containers in Classic (but does in Core)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/snap-confine/+bug/1626019/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=snap-confine; milestone=1.0.45; status=Fix
> Committed; importance=High; <email address hidden>;
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: apulido jdstrand lool zyga
> Launchpad-Bug-Reporter: Ara Pulido (apulido)
> Launchpad-Bug-Modifier: Loïc Minier (lool)
> Launchpad-Message-Rationale: Subscriber
> Launchpad-Message-For: apulido
>

Zygmunt Krynicki (zyga) on 2019-03-25
Changed in snap-confine:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers