systemd-localed requires read-only-etc hacks for /etc/vconsole.conf and /etc/locale.conf

Bug #2091657 reported by Nick Rosbrook
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Invalid
Undecided
Unassigned
Noble
Fix Released
Medium
Nick Rosbrook

Bug Description

[Impact]

Ubuntu Core has a read-only /etc, but a writable partition in /etc/writable. In some versions of Ubuntu, the systemd package ships a patch to follow symlinks from /etc/ -> /etc/writable, e.g. for files like /etc/locale.conf.

In Core24, this patch needs to be expanded for /etc/locale.conf and /etc/vconsole.conf in systemd-localed.

[Test Plan]

1. On a pristine Ubuntu Core system, run "localectl status". Verify that the system locale is set to the default (seems to be "C.UTF-8" based on the current core24 snap).

2. Modify the system locale by running "localectl set-locale LANG=en_AU.UTF-8". The command should complete without error.

3. Check that the locale has changed by running "localectl status".

4. Check that the locale has been saved to /etc/locale.conf

5. Reboot the system, and run "localectl status" again to check that the change has persisted.

[Where problems could occur]

This hack is designed to be a no-op in non-Ubuntu Core cases. I.e., when /etc/writable does not exist or does not have a link for the given path in etc, the original path is used.

As is evident by the ever-expanding nature of this patch, this is not a complete solution. It's possible that more cases will be found where this hack needs to be applied.

[Other information]

This patch is not carried in supported releases newer than Noble. While this patch is far from ideal, expanding this hack has become the go-to for existing stable releases.

Nick Rosbrook (enr0n)
Changed in systemd (Ubuntu):
status: New → Invalid
Changed in systemd (Ubuntu Noble):
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Nick Rosbrook (enr0n)
Revision history for this message
James Henstridge (jamesh) wrote :

As far as a test plan goes, I think this can mostly be done using the localectl tool that is already shipped in the base snap. This exercises the same D-Bus API that higher level tools like ubuntu-desktop-init use. As a command line tool, it could likely be used by a spread test.

To test system locale changes, it might look something like this:

1. On a pristine Ubuntu Core system, run "localectl status". Verify that the system locale is set to the default (seems to be "C.UTF-8" based on the current core24 snap).

2. Modify the system locale by running "localectl set-locale LANG=en_AU.UTF-8". The command should complete without error.

3. Check that the locale has changed by running "localectl status".

4. Check that the locale has been saved to /etc/locale.conf

5. Reboot the system, and run "localectl status" again to check that the change has persisted.

Testing the key map changes is a little more difficult, since the relevant commands have been patched out of our localectl:

    $ localectl set-keymap foo
    Setting X11 and console keymaps is not supported in Debian.

It should still be possible to call the D-Bus API directly via "busctl call", but it will be a bit more verbose. Longer term, it might be worth looking at whether we can drop the patch that disables set-keymap.

Revision history for this message
Philip Meulengracht (the-meulengracht) wrote (last edit ):

Hey!

So tested with a new UC 24 system that includes the systemd from enr0n ppa and has the following setup

$ systemctl --version
systemd 255 (255.4-1ubuntu8.6~ppa3)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

$ ls -l /etc/vconsole.conf
lrwxrwxrwx 1 root root 22 Jan 30 09:51 /etc/vconsole.conf -> writable/vconsole.conf

$ ls -l /etc/default/keyboard
lrwxrwxrwx 1 root root 25 Jan 30 09:51 /etc/default/keyboard -> ../writable/vconsole.conf

$ ls -l /etc/locale*
lrwxrwxrwx 1 root root 20 Jan 29 14:48 /etc/locale.conf -> writable/locale.conf

$ ls -l /etc/default/locale
lrwxrwxrwx 1 root root 23 Jan 30 08:28 /etc/default/locale -> ../writable/locale.conf

$ cat /etc/locale.conf
LANG=C.UTF-8

$ localectl status
System Locale: LANG=C.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

$ sudo localectl set-locale LANG=en_AU.UTF-8

$ localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

$ cat /etc/locale.conf
LANG=en_AU.UTF-8

$ sudo reboot

$ localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

So it seems to work now!

Revision history for this message
James Henstridge (jamesh) wrote :

I think the root cause here is this bind mount:

https://github.com/canonical/core-base/blob/82c8fadb4cf77df5f771220d27c40dc011675f23/static/etc/system-image/writable-paths#L32

When systemd-localed tries to update the configuration file, it writes the new content to a temporary file and then renames it over the top of the destination. Trying to rename over the bind mount fails.

In core24-desktop, we'd removed the bind mount and instead set it up as a symlink to /etc/writable/locale.conf:

https://github.com/canonical/core-base-desktop/blob/2149717d911493e26b3519a51798ae6a287189a7/hooks/009-locale-archive.chroot#L28-L30

We did similar for the vconsole.conf file used for keymap settings.

Revision history for this message
Philip Meulengracht (the-meulengracht) wrote :

As James pointed out that was the root of my issue, after removing the bind-mount everything worked

Nick Rosbrook (enr0n)
description: updated
Revision history for this message
Nick Rosbrook (enr0n) wrote :

A patch for this is now awaiting SRU review in the noble unapproved queue.

Changed in systemd (Ubuntu Noble):
status: Triaged → In Progress
Revision history for this message
Philip Meulengracht (the-meulengracht) wrote :

Thank you Nick!

The core side of things are in this PR: https://github.com/canonical/core-base/pull/313

Revision history for this message
Andreas Hasenack (ahasenack) wrote : Please test proposed package

Hello Nick, or anyone else affected,

Accepted systemd into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/255.4-1ubuntu8.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Noble):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-noble
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/255.4-1ubuntu8.6)

All autopkgtests for the newly accepted systemd (255.4-1ubuntu8.6) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

ayatana-indicator-session/24.2.0-1build2 (arm64, ppc64el, s390x)
ceph/19.2.0-0ubuntu0.24.04.2 (ppc64el)
certspotter/unknown (armhf)
clamav/unknown (armhf)
clevis/unknown (armhf)
collectd/5.12.0-17.1build2 (s390x)
cron/3.0pl1-184ubuntu2 (ppc64el)
csync2/unknown (armhf)
cups/unknown (armhf)
dbus/unknown (armhf)
dbus-broker/35-2 (s390x)
debci/unknown (armhf)
dovecot/1:2.3.21+dfsg1-2ubuntu6 (s390x)
dpdk/23.11.2-0ubuntu0.24.04.1 (amd64, ppc64el)
exim4/4.97-4ubuntu4.2 (ppc64el)
freedom-maker/unknown (armhf)
gpsd/unknown (armhf)
haproxy/unknown (armhf)
hddemux/unknown (armhf)
hwloc/unknown (armhf)
kodi/2:20.5+dfsg-1ubuntu1 (ppc64el)
liblinux-systemd-perl/unknown (armhf)
libqb/unknown (armhf)
libvirt/unknown (armhf)
libvirt-dbus/unknown (armhf)
lighttpd/unknown (armhf)
linux-azure-6.11/6.11.0-1008.8~24.04.1 (amd64, arm64)
linux-gcp-6.11/6.11.0-1006.6~24.04.2 (arm64)
linux-gke/6.8.0-1019.23 (amd64)
linux-hwe-6.11/6.11.0-21.21~24.04.1 (amd64, arm64)
linux-hwe-6.11/unknown (armhf)
linux-lowlatency/6.8.0-56.58.1 (amd64)
linux-lowlatency-hwe-6.11/6.11.0-1010.11~24.04.1 (arm64)
linux-nvidia/6.8.0-1024.27 (arm64)
linux-nvidia-lowlatency/6.8.0-1024.27.1 (arm64)
linux-oem-6.8/6.8.0-1022.22 (amd64)
logiops/unknown (armhf)
logrotate/unknown (armhf)
mandos/unknown (armhf)
mariadb/1:10.11.8-0ubuntu0.24.04.1 (armhf)
mediawiki/1:1.39.7-1 (ppc64el)
mosquitto/2.0.18-1build3 (amd64, arm64, armhf, ppc64el, s390x)
mpd/unknown (armhf)
munin/unknown (armhf)
mutter/unknown (armhf)
netplan.io/1.1.1-1~ubuntu24.04.1 (arm64)
open-build-service/unknown (armhf)
openbsd-inetd/unknown (armhf)
openssh/1:9.6p1-3ubuntu13.8 (ppc64el)
pipewire/unknown (armhf)
policykit-1/unknown (armhf)
polkit-qt-1/unknown (armhf)
postgresql-16/unknown (armhf)
procps/unknown (armhf)
prometheus-homeplug-exporter/unknown (armhf)
prometheus-ipmi-exporter/1.8.0-1ubuntu0.24.04.2 (ppc64el)
prometheus-postfix-exporter/unknown (armhf)
prometheus-postgres-exporter/unknown (armhf)
prometheus-squid-exporter/unknown (armhf)
pulseaudio/unknown (armhf)
pystemd/unknown (armhf)
rsyslog/8.2312.0-3ubuntu9 (s390x)
rust-whoami/unknown (armhf)
rust-zram-generator/unknown (armhf)
sbws/unknown (armhf)
seatd/unknown (armhf)
shibboleth-sp/unknown (armhf)
swupdate/unknown (armhf)
systemd-cron/unknown (armhf)
systemd-hwe/unknown (armhf)
tinyssh/unknown (armhf)
tmux/unknown (armhf)
usbauth/unknown (armhf)
yder/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Philip Meulengracht (the-meulengracht) wrote :

Tested on a pristine UC24 built with a custom `core24` base that uses noble-proposed instead of noble-updates, and ran the same sequence of commands (updating locale, rebooting, ensuring it was the same)

$ systemctl --version
systemd 255 (255.4-1ubuntu8.6)
+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified

$ localectl status
System Locale: LANG=C.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

$ sudo localectl set-locale LANG=en_AU.UTF-8

$ localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

$ cat /etc/locale.conf
LANG=en_AU.UTF-8

$ sudo reboot

$ localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

Things are looking good!

tags: added: verification-done verification-done-noble
removed: verification-needed verification-needed-noble
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 255.4-1ubuntu8.6

---------------
systemd (255.4-1ubuntu8.6) noble; urgency=medium

  * stub: add magic cmdline option to force EFI handover
    (LP: #2088069)
  * localed: use ubuntu core hack for locale.conf and vconsole.conf
    (LP: #2091657)

 -- Nick Rosbrook <email address hidden> Fri, 21 Feb 2025 16:18:31 -0500

Changed in systemd (Ubuntu Noble):
status: Fix Committed → Fix Released
Revision history for this message
Timo Aaltonen (tjaalton) wrote : Update Released

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.