lxd

Activity log for bug #2046486

Date Who What changed Old value New value Message
2023-12-14 19:28:10 Nick Rosbrook bug added bug
2023-12-14 19:28:21 Nick Rosbrook systemd (Ubuntu): status New Confirmed
2023-12-14 19:28:25 Nick Rosbrook systemd (Ubuntu): status Confirmed New
2023-12-14 19:28:29 Nick Rosbrook systemd (Ubuntu): importance Undecided High
2023-12-14 19:28:33 Nick Rosbrook systemd (Ubuntu): assignee Nick Rosbrook (enr0n)
2024-05-30 19:23:19 Nick Rosbrook systemd (Ubuntu): status New Triaged
2024-06-18 20:00:40 Nick Rosbrook summary units with SetCredential= fail in LXD containers units with credentials fail in LXD containers
2024-06-18 20:01:17 Nick Rosbrook tags block-proposed
2024-06-18 20:05:07 Nick Rosbrook bug watch added https://github.com/canonical/lxd/issues/13631
2024-06-19 13:49:11 Nick Rosbrook tags block-proposed
2024-06-24 18:57:47 Nick Rosbrook description To demonstrate this, in an unprivileged LXD container, create the following unit (taken from the systemd test suite): $ cat > /etc/systemd/system/exec-set-credential.service << EOF # SPDX-License-Identifier: LGPL-2.1-or-later [Unit] Description=Test for SetCredential= [Service] ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' Type=oneshot SetCredential=test-execute.set-credential:hoge EOF $ systemctl daemon-reload $ systemctl start exec-set-credential.service Job for exec-set-credential.service failed because the control process exited with error code. See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details. With debug logs enabled, we see: $ journalctl -u exec-set-credential.service -b --no-pager Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183 Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=... Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184. Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "") Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")... Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")... Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1. Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting) Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources... Many units shipped by systemd use credentials in some way by default now (in v256). So this issue is now about much more than the original test case failure. For example, root@oracular:~# apt policy systemd systemd: Installed: 256-1ubuntu1 Candidate: 256-1ubuntu1 Version table: *** 256-1ubuntu1 100 100 http://archive.ubuntu.com/ubuntu oracular-proposed/main amd64 Packages 100 /var/lib/dpkg/status 255.4-1ubuntu8 500 500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages root@oracular:~# for service in $(find /usr/lib/systemd/system -maxdepth 1 -name "systemd-*.service"); do grep -q "Credential.*=" "$service" && echo "$service"; done /usr/lib/systemd/system/systemd-sysusers.service /usr/lib/systemd/system/systemd-resolved.service /usr/lib/systemd/system/systemd-firstboot.service /usr/lib/systemd/system/systemd-network-generator.service /usr/lib/systemd/system/systemd-journald.service /usr/lib/systemd/system/systemd-sysctl.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service /usr/lib/systemd/system/systemd-tmpfiles-setup.service /usr/lib/systemd/system/systemd-udev-load-credentials.service /usr/lib/systemd/system/systemd-tmpfiles-clean.service /usr/lib/systemd/system/systemd-networkd.service [Original Description] To demonstrate this, in an unprivileged LXD container, create the following unit (taken from the systemd test suite): $ cat > /etc/systemd/system/exec-set-credential.service << EOF # SPDX-License-Identifier: LGPL-2.1-or-later [Unit] Description=Test for SetCredential= [Service] ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' Type=oneshot SetCredential=test-execute.set-credential:hoge EOF $ systemctl daemon-reload $ systemctl start exec-set-credential.service Job for exec-set-credential.service failed because the control process exited with error code. See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details. With debug logs enabled, we see: $ journalctl -u exec-set-credential.service -b --no-pager Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183 Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=... Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184. Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "") Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")... Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")... Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1. Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting) Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources...
2024-06-24 19:00:20 Nick Rosbrook description Many units shipped by systemd use credentials in some way by default now (in v256). So this issue is now about much more than the original test case failure. For example, root@oracular:~# apt policy systemd systemd: Installed: 256-1ubuntu1 Candidate: 256-1ubuntu1 Version table: *** 256-1ubuntu1 100 100 http://archive.ubuntu.com/ubuntu oracular-proposed/main amd64 Packages 100 /var/lib/dpkg/status 255.4-1ubuntu8 500 500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages root@oracular:~# for service in $(find /usr/lib/systemd/system -maxdepth 1 -name "systemd-*.service"); do grep -q "Credential.*=" "$service" && echo "$service"; done /usr/lib/systemd/system/systemd-sysusers.service /usr/lib/systemd/system/systemd-resolved.service /usr/lib/systemd/system/systemd-firstboot.service /usr/lib/systemd/system/systemd-network-generator.service /usr/lib/systemd/system/systemd-journald.service /usr/lib/systemd/system/systemd-sysctl.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service /usr/lib/systemd/system/systemd-tmpfiles-setup.service /usr/lib/systemd/system/systemd-udev-load-credentials.service /usr/lib/systemd/system/systemd-tmpfiles-clean.service /usr/lib/systemd/system/systemd-networkd.service [Original Description] To demonstrate this, in an unprivileged LXD container, create the following unit (taken from the systemd test suite): $ cat > /etc/systemd/system/exec-set-credential.service << EOF # SPDX-License-Identifier: LGPL-2.1-or-later [Unit] Description=Test for SetCredential= [Service] ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' Type=oneshot SetCredential=test-execute.set-credential:hoge EOF $ systemctl daemon-reload $ systemctl start exec-set-credential.service Job for exec-set-credential.service failed because the control process exited with error code. See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details. With debug logs enabled, we see: $ journalctl -u exec-set-credential.service -b --no-pager Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183 Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=... Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184. Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "") Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")... Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")... Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1. Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting) Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources... Many units shipped by systemd use credentials in some way by default now (in v256). So this issue is now about much more than the original test case failure. For example, root@oracular:~# apt policy systemd systemd:   Installed: 256-1ubuntu1   Candidate: 256-1ubuntu1   Version table:  *** 256-1ubuntu1 100         100 http://archive.ubuntu.com/ubuntu oracular-proposed/main amd64 Packages         100 /var/lib/dpkg/status      255.4-1ubuntu8 500         500 http://archive.ubuntu.com/ubuntu oracular/main amd64 Packages root@oracular:~# for service in $(find /usr/lib/systemd/system -maxdepth 1 -name "systemd-*.service"); do grep -q "Credential.*=" "$service" && echo "$service"; done /usr/lib/systemd/system/systemd-sysusers.service /usr/lib/systemd/system/systemd-resolved.service /usr/lib/systemd/system/systemd-firstboot.service /usr/lib/systemd/system/systemd-network-generator.service /usr/lib/systemd/system/systemd-journald.service /usr/lib/systemd/system/systemd-sysctl.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service /usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service /usr/lib/systemd/system/systemd-tmpfiles-setup.service /usr/lib/systemd/system/systemd-udev-load-credentials.service /usr/lib/systemd/system/systemd-tmpfiles-clean.service /usr/lib/systemd/system/systemd-networkd.service root@oracular:~# systemctl status systemd-sysusers.service systemd-resolved.service systemd-firstboot.service systemd-network-generator.service systemd-journald.service systemd-sysctl.service systemd-tmpfiles-setup-dev-early.service systemd-tmpfiles-setup-dev.service systemd-tmpfiles-setup.service systemd-udev-load-credentials.service systemd-tmpfiles-clean.service systemd-networkd.service ○ systemd-sysusers.service - Create System Users Loaded: loaded (/usr/lib/systemd/system/systemd-sysusers.service; static) Active: inactive (dead) Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 1min 0s ago ├─ ConditionNeedsUpdate=|/etc was not met └─ ConditionCredential=|sysusers.extra was not met Docs: man:sysusers.d(5) man:systemd-sysusers.service(8) × systemd-resolved.service - Network Name Resolution Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago Invocation: b1aaa662750f48868fe3388e4524c462 Docs: man:systemd-resolved.service(8) man:org.freedesktop.resolve1(5) https://systemd.io/WRITING_NETWORK_CONFIGURATION_MANAGERS https://systemd.io/WRITING_RESOLVER_CLIENTS Process: 258 ExecStart=/usr/lib/systemd/systemd-resolved (code=exited, status=243/CREDENTIALS) Main PID: 258 (code=exited, status=243/CREDENTIALS) ○ systemd-firstboot.service - First Boot Wizard Loaded: loaded (/usr/lib/systemd/system/systemd-firstboot.service; static) Active: inactive (dead) Condition: start condition unmet at Mon 2024-06-24 18:58:48 UTC; 59s ago └─ ConditionFirstBoot=yes was not met Docs: man:systemd-firstboot(1) ○ systemd-network-generator.service - Generate network units from Kernel command line Loaded: loaded (/usr/lib/systemd/system/systemd-network-generator.service; disabled; preset: enabled) Active: inactive (dead) Docs: man:systemd-network-generator.service(8) × systemd-journald.service - Journal Service Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static) Drop-In: /usr/lib/systemd/system/systemd-journald.service.d └─nice.conf Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 7caace7a15c749f3a86fb15fcfb94dff TriggeredBy: × systemd-journald-dev-log.socket × systemd-journald.socket ○ systemd-journald-audit.socket Docs: man:systemd-journald.service(8) man:journald.conf(5) Process: 124 ExecStart=/usr/lib/systemd/systemd-journald (code=exited, status=243/CREDENTIALS) Main PID: 124 (code=exited, status=243/CREDENTIALS) FD Store: 0 (limit: 4224) × systemd-sysctl.service - Apply Kernel Variables Loaded: loaded (/usr/lib/systemd/system/systemd-sysctl.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 5e90310a27b043ceae80c96e35c41451 Docs: man:systemd-sysctl.service(8) man:sysctl.d(5) Process: 97 ExecStart=/usr/lib/systemd/systemd-sysctl (code=exited, status=243/CREDENTIALS) Main PID: 97 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev-early.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 78e3c68cfa9a4a7982950b08c0f1385f Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 73 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot --graceful (code=exited, status=243/CREDENTIALS) Main PID: 73 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: 46458c7b6e134ef8be299900db7cc288 Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 98 ExecStart=systemd-tmpfiles --prefix=/dev --create --boot (code=exited, status=243/CREDENTIALS) Main PID: 98 (code=exited, status=243/CREDENTIALS) × systemd-tmpfiles-setup.service - Create Volatile Files and Directories Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup.service; static) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: f4e64afdc8774170a9b29b8cf2919f46 Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) Process: 147 ExecStart=systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev (code=exited, status=243/CREDENTIALS) Main PID: 147 (code=exited, status=243/CREDENTIALS) × systemd-udev-load-credentials.service - Load udev Rules from Credentials Loaded: loaded (/usr/lib/systemd/system/systemd-udev-load-credentials.service; disabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:48 UTC; 1min 0s ago Invocation: cb5a1f43cde248de80fcf701b4b5d381 Docs: man:udevadm(8) man:udev(7) man:systemd.system-credentials(7) Process: 75 ExecStart=udevadm control --load-credentials (code=exited, status=243/CREDENTIALS) Main PID: 75 (code=exited, status=243/CREDENTIALS) ○ systemd-tmpfiles-clean.service - Cleanup of Temporary Directories Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.service; static) Active: inactive (dead) TriggeredBy: ● systemd-tmpfiles-clean.timer Docs: man:tmpfiles.d(5) man:systemd-tmpfiles(8) × systemd-networkd.service - Network Configuration Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; preset: enabled) Active: failed (Result: exit-code) since Mon 2024-06-24 18:58:49 UTC; 59s ago Invocation: 5d960369ea944d5cbac4382e42ded1d0 TriggeredBy: × systemd-networkd.socket Docs: man:systemd-networkd.service(8) man:org.freedesktop.network1(5) Process: 280 ExecStart=/usr/lib/systemd/systemd-networkd (code=exited, status=243/CREDENTIALS) Main PID: 280 (code=exited, status=243/CREDENTIALS) FD Store: 0 (limit: 512) [Original Description] To demonstrate this, in an unprivileged LXD container, create the following unit (taken from the systemd test suite): $ cat > /etc/systemd/system/exec-set-credential.service << EOF # SPDX-License-Identifier: LGPL-2.1-or-later [Unit] Description=Test for SetCredential= [Service] ExecStart=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStartPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStop=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' ExecStopPost=/bin/sh -x -c 'test "$$(cat %d/test-execute.set-credential)" = "hoge"' Type=oneshot SetCredential=test-execute.set-credential:hoge EOF $ systemctl daemon-reload $ systemctl start exec-set-credential.service Job for exec-set-credential.service failed because the control process exited with error code. See "systemctl status exec-set-credential.service" and "journalctl -xeu exec-set-credential.service" for details. With debug logs enabled, we see: $ journalctl -u exec-set-credential.service -b --no-pager Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Trying to enqueue job exec-set-credential.service/start/replace Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Installed new job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Enqueued job exec-set-credential.service/start as 2740 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_start): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to set 'trusted.invocation_id' xattr on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.delegate' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed to remove 'trusted.survive_final_kill_signal' xattr flag on control group /system.slice/exec-set-credential.service, ignoring: Operation not permitted Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Passing 0 fds to service Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2183 Dec 14 19:24:24 noble (sh)[2183]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2183]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed dead -> start Dec 14 19:24:24 noble systemd[1]: Starting exec-set-credential.service - Test for SetCredential=... Dec 14 19:24:24 noble (sh)[2183]: Successfully forked off '(sd-mkdcreds)' as PID 2184. Dec 14 19:24:24 noble (sd-[2184]: Changing mount propagation /dev (MS_REC|MS_SLAVE "") Dec 14 19:24:24 noble (sd-[2184]: Mounting ramfs (ramfs) on /dev/shm (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOSYMFOLLOW "mode=0700")... Dec 14 19:24:24 noble (sd-[2184]: Changing mount flags /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND "")... Dec 14 19:24:24 noble (sd-[2184]: Failed to mount n/a (type n/a) on /dev/shm (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_REMOUNT|MS_NOSYMFOLLOW|MS_BIND ""): Permission denied Dec 14 19:24:24 noble (sh)[2183]: (sd-mkdcreds) failed with exit status 1. Dec 14 19:24:24 noble (sh)[2183]: exec-set-credential.service: Failed to set up credentials: Protocol error Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2183 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Main process exited, code=exited, status=243/CREDENTIALS Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Will spawn child (service_enter_stop_post): /bin/sh Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: About to execute: /bin/sh -x -c "test \"1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential)\" = \"hoge\"" Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Forked /bin/sh as 2186 Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed start -> stop-post Dec 14 19:24:24 noble (sh)[2186]: PR_SET_MM_ARG_START failed: Operation not permitted Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble (sh)[2186]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy Dec 14 19:24:24 noble sh[2186]: + test 1031(cat /run/credentials/exec-set-credential.service/test-execute.set-credential) = hoge Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Child 2186 belongs to exec-set-credential.service. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Control process exited, code=exited, status=1/FAILURE Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Got final SIGCHLD for state stop-post. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Failed with result 'exit-code'. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Service will not restart (restart setting) Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Changed stop-post -> failed Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Job 2740 exec-set-credential.service/start finished, result=failed Dec 14 19:24:24 noble systemd[1]: Failed to start exec-set-credential.service - Test for SetCredential=. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Unit entered failed state. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Consumed 23ms CPU time. Dec 14 19:24:24 noble systemd[1]: exec-set-credential.service: Releasing resources...
2024-06-24 19:02:13 Nick Rosbrook bug task added systemd
2024-06-24 19:02:22 Nick Rosbrook bug task deleted systemd
2024-06-24 19:03:15 Nick Rosbrook bug task added lxd (Ubuntu)
2024-06-24 19:03:34 Nick Rosbrook bug task added lxd
2024-06-25 01:15:31 Bug Watch Updater lxd: status Unknown New
2024-07-09 18:42:59 Nick Rosbrook bug task added samba (Ubuntu)
2024-07-09 18:43:11 Nick Rosbrook tags update-excuse
2024-07-09 18:56:25 Launchpad Janitor merge proposal linked https://code.launchpad.net/~enr0n/ubuntu/+source/samba/+git/samba/+merge/469048
2024-07-10 11:55:10 Launchpad Janitor samba (Ubuntu): status New Fix Released
2024-07-12 06:35:20 Launchpad Janitor lxd (Ubuntu): status New Confirmed
2024-07-15 14:52:22 Rik Mills bug added subscriber Rik Mills
2024-07-17 11:23:08 Alessandro Astone bug added subscriber Alessandro Astone
2024-07-22 14:36:17 Nick Rosbrook tags update-excuse