docker.io 20.10.25-0ubuntu1~22.04.1 loses file permissions on docker-build

[Dmitry Rogozhkin]

docker-build loses file permissions configured on the previous layer. Consider this trivial dockerfile example:

$ cat Dockerfile
FROM ubuntu:22.04
RUN groupadd -r user && useradd -lrm -s /bin/bash -g user user && ls -al /home/
RUN ls -al /home/

Mind that ls command on the first layer correctly reports that user home directory is owned by 'user'. Next layer however reports it's owned by 'root':

$ docker --version
Docker version 20.10.25, build 20.10.25-0ubuntu1~22.04.1

$ docker build -f Dockerfile -t bug --no-cache .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu:22.04
---> 5a81c4b8502e
Step 2/3 : RUN groupadd -r user && useradd -lrm -s /bin/bash -g user user && ls -al /home/
---> Running in 25321c5d3936
total 12
drwxr-xr-x 1 root root 4096 Jul 20 21:13 .
drwxr-xr-x 1 root root 4096 Jul 20 21:13 ..
drwxr-x--- 2 user user 4096 Jul 20 21:13 user
Removing intermediate container 25321c5d3936
---> 517d81d51c42
Step 3/3 : RUN ls -al /home/
---> Running in 9612f47d6fe6
total 12
drwxr-xr-x 1 root root 4096 Jul 20 21:13 .
drwxr-xr-x 1 root root 4096 Jul 20 21:13 ..
drwxr-x--- 2 root root 4096 Jul 20 21:13 user
Removing intermediate container 9612f47d6fe6
---> a782363e2ab0
Successfully built a782363e2ab0
Successfully tagged bug:latest

This isues was found on 22.04.2 Ubuntu server with enabled jammy-proposed repository. Issue seems to be a regression on the docker.io proposed update to 20.10.25-0ubuntu1~22.04.1. I did confirm that I do not see this issue on the latest qualified version of docker.io available out-of-the box on 22.04.2, i.e. docker.io=20.10.21-0ubuntu1~22.04.3 does not have this bug.

Some more details on the system I was using:

$ uname -a
Linux docker-bug 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"

$ apt-cache policy docker.io
docker.io:
  Installed: 20.10.25-0ubuntu1~22.04.1
  Candidate: 20.10.25-0ubuntu1~22.04.1
  Version table:
 *** 20.10.25-0ubuntu1~22.04.1 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     20.10.21-0ubuntu1~22.04.3 500
        500 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages
        500 http://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages
     20.10.12-0ubuntu4 500
        500 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages

[Dmitry Rogozhkin (dvrogozh)]

Other observations from my side:
1. If you build docker image with 20.10.21-0ubuntu1~22.04.3 ("good") then you have correct permissions if you later use this image on docker run with 20.10.25-0ubuntu1~22.04.1 ("bad").
2. If you build docker image with 20.10.25-0ubuntu1~22.04.1 ("bad") then you have wrong permissions if you later use this image on docker run with 20.10.21-0ubuntu1~22.04.3 ("good").

Also, so far I did not see any indication of error reported during docker-build or elsewhere on a system. Docker build seems to go just fine though later permissions are wrong. It might be I just don't know where to look for appropriate log... let me know of what needs to be checked.

[Leroy (leroy84)]

same issue:

FROM ubuntu:22.04

RUN mkdir -p /home/leroy/test && \
    chown _apt /home/ -Rv && \
    find /home -ls

USER _apt

RUN find /home -ls

---------------
Sending build context to Docker daemon 19.46kB
Step 1/4 : FROM ubuntu:22.04
 ---> 5a81c4b8502e
Step 2/4 : RUN mkdir -p /home/leroy/test && chown _apt /home/ -Rv && find /home -ls
 ---> Running in 5a521630d15b
changed ownership of '/home/leroy/test' from root to _apt
changed ownership of '/home/leroy' from root to _apt
changed ownership of '/home/' from root to _apt
   258517 4 drwxr-xr-x 1 _apt root 4096 Aug 3 17:53 /home
   262627 4 drwxr-xr-x 3 _apt root 4096 Aug 3 17:53 /home/leroy
   262628 4 drwxr-xr-x 2 _apt root 4096 Aug 3 17:53 /home/leroy/test
Removing intermediate container 5a521630d15b
 ---> 9eec62f1ee21
Step 3/4 : USER _apt
 ---> Running in c9a9b619efea
Removing intermediate container c9a9b619efea
 ---> 4d15b5c19813
Step 4/4 : RUN find /home -ls
 ---> Running in aa0566d9d46e
   262639 4 drwxr-xr-x 1 root root 4096 Aug 3 17:53 /home
   262643 4 drwxr-xr-x 3 root root 4096 Aug 3 17:53 /home/leroy
   262644 4 drwxr-xr-x 2 root root 4096 Aug 3 17:53 /home/leroy/test
Removing intermediate container aa0566d9d46e
 ---> e069be9bfba3
Successfully built e069be9bfba3
root@instance-1:~#

output:

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Leroy (leroy84)]

After switching to user, everything is set back to root, any further actions like e.g. git checkout will fail because of permission denied

[Leroy (leroy84)]

Rolling back to 20.10.25-0ubuntu1~22.04.1 "solves" the issue, that is a confirmation

[Joe Francis (1-joe-s)]

I believe correct version to roll back to is actually docker.io=20.10.21-0ubuntu1~22.04.3

[Leroy (leroy84)]

That's correct, i made a typo

[Chris Wilson (jakdaw)]

I have the same problem - both on Jammy (20.10.25-0ubuntu1~22.04.1) and focal (20.10.25-0ubuntu1~20.04.1) systems. Previous version (20.10.21) works as expected.

Using btrfs Storage Driver, if that makes any difference.

[TREVOR GETTY (trevorgettywandisco)]

Same problem experienced with overlay2 driver although I dont think that is important.

20.10.25-0ubuntu1~22.04.1

Permissions on any specific user file creation ends up being ROOT.
thanks.