Activity log for bug #2056442

Date Who What changed Old value New value Message
2024-03-07 11:33:26 Faustin bug added bug
2024-03-07 12:18:50 John Chittum bug task added libpod (Ubuntu)
2024-03-07 12:19:07 John Chittum bug task added crun (Ubuntu)
2024-03-07 12:25:14 John Chittum description The problem is very well described in https://github.com/actions/runner-images/issues/9425. The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System Ubuntu 22.04.4 LTS Runner Image Image: ubuntu-22.04 Version: 20240225.1.0 Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-07 12:59:19 Julian Andres Klode summary Podman (crun) regression in Ubuntu 22.04 Podman (crun) regression in Ubuntu 22.04: OCI runtime error: chmod `run/shm`: Operation not supported
2024-03-07 13:30:29 Julian Andres Klode description The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System Ubuntu 22.04.4 LTS Runner Image Image: ubuntu-22.04 Version: 20240225.1.0 Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. podman run --rm some image Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-07 13:46:21 Julian Andres Klode crun (Ubuntu): status New Incomplete
2024-03-07 13:54:35 Julian Andres Klode nominated for series Ubuntu Jammy
2024-03-07 13:54:35 Julian Andres Klode bug task added crun (Ubuntu Jammy)
2024-03-07 13:54:35 Julian Andres Klode bug task added libpod (Ubuntu Jammy)
2024-03-07 13:54:42 Julian Andres Klode crun (Ubuntu Jammy): status New Incomplete
2024-03-07 13:54:46 Julian Andres Klode crun (Ubuntu): status Incomplete Fix Released
2024-03-07 14:18:46 Julian Andres Klode bug added subscriber Julian Andres Klode
2024-03-07 14:18:55 Julian Andres Klode crun (Ubuntu Jammy): assignee Julian Andres Klode (juliank)
2024-03-07 18:02:47 Launchpad Janitor libpod (Ubuntu): status New Confirmed
2024-03-07 18:02:47 Launchpad Janitor libpod (Ubuntu Jammy): status New Confirmed
2024-03-08 10:05:37 Julian Andres Klode description [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. podman run --rm some image Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y ENTRYPOINT /usr/bin/systemd 3. Run it `podman run --systemd always systemd` Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-08 10:09:04 Julian Andres Klode description [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y ENTRYPOINT /usr/bin/systemd 3. Run it `podman run --systemd always systemd` Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 3. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-08 10:10:38 Julian Andres Klode crun (Ubuntu Jammy): status Incomplete In Progress
2024-03-08 10:10:47 Julian Andres Klode bug task deleted libpod (Ubuntu)
2024-03-08 10:10:52 Julian Andres Klode bug task deleted libpod (Ubuntu Jammy)
2024-03-08 11:47:15 Markus Falb bug added subscriber Markus Falb
2024-03-08 12:08:50 Markus Falb bug watch added https://github.com/containers/crun/issues/1308
2024-03-21 20:41:51 Andreas Hasenack crun (Ubuntu Jammy): status In Progress Incomplete
2024-03-21 22:17:29 Julian Andres Klode description [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] Needs to be finalized, but roughly: 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 3. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 3. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-21 22:17:36 Julian Andres Klode crun (Ubuntu Jammy): status Incomplete In Progress
2024-03-28 19:10:27 Andreas Hasenack nominated for series Ubuntu Mantic
2024-03-28 19:10:27 Andreas Hasenack bug task added crun (Ubuntu Mantic)
2024-03-28 19:11:05 Andreas Hasenack bug added subscriber Andreas Hasenack
2024-03-28 19:12:38 Andreas Hasenack description [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 3. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. Make sure you have crun installed, and not runc, because podman can also use runc and this bug is about crun. 3. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 4. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-28 19:17:30 Andreas Hasenack description [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. Make sure you have crun installed, and not runc, because podman can also use runc and this bug is about crun. 3. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 4. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/ [Impact] podman containers using the default crun backend do not work anymore with the 6.5 HWE kernel in 22.04 [Test plan] 1. Boot jammy with 6.5 HWE kernel (e.g. install linux-virtual-hwe-22.04 in a VM) 2. Make sure you have crun installed, and not runc, because podman can also use runc and this bug is about crun. 3. `podman build -t systemd .` with the Dockerfile: FROM ubuntu:noble RUN apt install -U systemd -y CMD ["/lib/systemd/systemd"] 4. Run it `podman run --systemd always systemd` you should not get Error: OCI runtime error: chmod `run/shm`: Operation not supported 5. Repeat the above steps on the jammy GA kernel, to make sure we did not regress that use case which is unaffected by this bug. Optimally submitter can do end-to-end-verification on their side. [Where problems could occur] The patch ignores ENOTSUP for fchmodat() in one function, so at most we could silently hide some other issues in fchmodat() inside that function, e.g. AppArmor denials. But generally that is what you would want as a behavior for ENOTSUP... [Original bug report] The problem is very well described in https://github.com/actions/runner-images/issues/9425. ## COPY FROM LINK I think there might be a regression in this release of the ubuntu-22.04 image which breaks podman.[1] The image updated the kernel from 6.2.y to 6.5.y, but podman/crun don't seem to be updated. Our build fails with this error link to run:[2] STEP 1/1: FROM ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac Trying to pull ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac... Getting image source signatures Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying blob sha256:041b542221cfde2f9fa4ac13f8b5804e25b23ab48ba47db2822c382a134256e1 Copying config sha256:1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Writing manifest to image destination Storing signatures COMMIT localhost/builder --> 1eba10d0345 Successfully tagged localhost/builder:latest Successfully tagged ghcr.io/gardenlinux/builder:3ab2bb52bc46bb200c761369c087e9413d1ce0ac 1eba10d0345cc6df78b7c3a6ced45da9db675d05eb20d5d286996e4f7ffb24d5 Error: OCI runtime error: chmod `run/shm`: Operation not supported Error: Process completed with exit code 126. This is with this image version: Current runner version: '2.313.0' Operating System   Ubuntu   22.04.4   LTS Runner Image   Image: ubuntu-22.04   Version: 20240225.1.0   Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240225.1/images/ubuntu/Ubuntu2204-Readme.md   Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 Trying to reproduce, it seems like I am only able to get this image version 20240218.1.0 where the issue does not appear. Tried to reproduce in this repo[3], but I'm not able to get this with image version 20240225.1.0. Is this a known issue and version 20240225.1.0 is not in use anymore? This blog post seems to suggest that the crun version is too old.[4] [1]https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240225.1 [2]https://github.com/gardenlinux/gardenlinux/actions/runs/8061893233/job/22020522535 [3]https://github.com/fwilhe/turbo-lamp/blob/main/.github/workflows/podman.yml [4]https://noobient.com/2023/11/15/fixing-ubuntu-containers-failing-to-start-with-systemd/
2024-03-28 19:20:20 Andreas Hasenack crun (Ubuntu Jammy): status In Progress Fix Committed
2024-03-28 19:20:22 Andreas Hasenack bug added subscriber Ubuntu Stable Release Updates Team
2024-03-28 19:20:25 Andreas Hasenack bug added subscriber SRU Verification
2024-03-28 19:20:28 Andreas Hasenack tags verification-needed verification-needed-jammy
2024-03-29 20:33:50 Markus Falb tags verification-needed verification-needed-jammy verification-done-jammy verification-needed
2024-04-12 17:18:07 Launchpad Janitor crun (Ubuntu Mantic): status New Confirmed
2024-04-12 21:14:14 Jacob Godserv merge proposal linked https://code.launchpad.net/~fun2program8/ubuntu/+source/crun/+git/crun/+merge/464233
2024-04-25 21:16:02 Andreas Hasenack crun (Ubuntu Mantic): status Confirmed Won't Fix
2024-04-25 21:37:13 Launchpad Janitor crun (Ubuntu Jammy): status Fix Committed Fix Released
2024-04-25 21:37:17 Andreas Hasenack removed subscriber Ubuntu Stable Release Updates Team
2024-07-15 20:37:31 John Chittum cloud-images: status New Invalid