systemd translations are not synced with upstream

Bug #1707898 reported by AsciiWolf
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Translations
Fix Released
Medium
Gunnar Hjalmarsson
systemd
Fix Released
Unknown
systemd (Ubuntu)
Fix Released
Medium
Gunnar Hjalmarsson

Bug Description

Translations for systemd are outdated because they do not seem to be synced with the upstream ones.

For example the Czech one:
Launchpad: https://translations.launchpad.net/ubuntu/artful/+source/systemd/+pots/systemd/cs/+translate - 0% translated
Upstream: https://github.com/systemd/systemd/blob/master/po/cs.po - 100% translated

Revision history for this message
AsciiWolf (asciiwolf) wrote :

Some languages are even unavailable via Launchpad even though they are present in upstream.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

I will require assistance to solve this.

Changed in systemd (Ubuntu):
assignee: nobody → Ubuntu Desktop (ubuntu-desktop)
Revision history for this message
AsciiWolf (asciiwolf) wrote :

Sorry, I don't know how to help you with this issue. Anyway, any news?

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@seb128 @laney any idea or help on what i need to do to have the translations right? should systemd be rebuild without stripping translations into langpacks? or how do I import translations into language packs? note that systemd uses journal translated entries, which I do not believe are currently handled by language packs correctly. Should I ship all of those in systemd-i18n package, which langpack building process cherrypicks the right translated messages into individual langpacks?

tags: added: rls-aa-incoming rls-bb-incoming
Changed in systemd (Ubuntu):
importance: Undecided → Critical
AsciiWolf (asciiwolf)
tags: added: bionic
Revision history for this message
AsciiWolf (asciiwolf) wrote :

https://help.launchpad.net/Translations/YourProject/ImportingTranslations
Here's something about Launchpad translation updates.

Revision history for this message
AsciiWolf (asciiwolf) wrote :

Any news?

Changed in ubuntu-translations:
assignee: nobody → Canonical Desktop Team (canonical-desktop-team)
Jeremy Bícha (jbicha)
Changed in systemd (Ubuntu):
assignee: Ubuntu Desktop (ubuntu-desktop) → Canonical Desktop Team (canonical-desktop-team)
Revision history for this message
AsciiWolf (asciiwolf) wrote :

Still not fixed. :-(

Revision history for this message
AsciiWolf (asciiwolf) wrote :

I have updated the Czech translation by hand. Unfortunately, the translation template itself seems to be outdated so I was not able to add some translated strings.

Steve Langasek (vorlon)
tags: removed: rls-bb-incoming
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

I checked out the latest tarball with systemd translations:

https://launchpad.net/ubuntu/bionic/+upload/17531208/+files/systemd_237-1ubuntu2_amd64_translations.tar.gz

Two observations:

1. The template is named incorrectly: "untitled.pot" instead of
   "systemd.pot"

2. The template only includes strings for C files; the strings from
   the policy XML files are missing.

The proposed patch fixes those issues, even if it's in a clumsy manner. I couldn't figure out how to make intltool-update() extract strings from the XML files, so I made use of xgettext(). Anyway, I think this would update LP with the latest translatable strings.

Changed in systemd (Ubuntu):
assignee: Canonical Desktop Team (canonical-desktop-team) → Gunnar Hjalmarsson (gunnarhj)
importance: Critical → Medium
status: New → In Progress
Changed in ubuntu-translations:
assignee: Canonical Desktop Team (canonical-desktop-team) → nobody
tags: added: patch
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@AsciiWolf: I have some doubts that the proposed patch is sufficient so really fix the systemd translations. Can you please provide a couple of examples of actions when the strings in the .policy files show up?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Translations are added at build time (meson?) to the .policy files like this:

<description xml:lang="cs">Nastavit systémovou konfiguraci klávesnice</description>

In the light of that, is there a reason at all to include translatable strings of the .policy files in LP and the language packs? I.e. will they ever be queried?

It would be another thing, I suppose, if we changed it so the .policy files were changed to include 'gettext-domain="systemd"' for 'message' and 'description'. But would such a difference to upstream be motivated?

Revision history for this message
AsciiWolf (asciiwolf) wrote :

gunnarhj: They show up when some action from systemctl/loginctl/hostnamectl/localectl/timedatectl/machinectl requires additional privileges. For example systemctl start/restart.

Revision history for this message
AsciiWolf (asciiwolf) wrote :

(Both the GUI and CLI polkit dialogs.)

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Once above is done, we will need to either send translations upstream and/or re-include them as part of the systemd build, then also teach pkgbinarymangler to extract the translated catalog files back into language packs, no?

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Do strings that are in the catalog files end up in the .pot template?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks AsciiWolf. I realized that I had to test from a standard user to see them.

I attached a second patch which also 'gettextizes' the .policy files, so the language pack translations are actually used. This makes the setup more consistent.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@Dimitri: Sorry, I saw your comments only after having added the second patch.

No need to send translations upstream. Translators know that they at first hand should do the translation work upstream.

No need either to re-include possible additional LP translations in the build. Just keep the upstream .po files there.

pkgbinarymangler knows what to do already AFAIU.

The .pot template is created by extracting translatable strings from the source files listed in po/POTFILES.in. It should reasonably consist of the very same strings as in the upstream .po files.

Hopefully that answers your questions. Please let me know if I misunderstood anything.

I have asked Sebastien to look at this, and I would suggest that you await his review before uploading.

Revision history for this message
Martin Pitt (pitti) wrote :

I committed the first hunk to Debian, this makes sense: https://salsa.debian.org/systemd-team/systemd/commit/18d8c2df133b8af

The second is too hackish for a permanent downstream delta, IMHO: This should rather be fixed upstream, as upstream polkit (as well as Debian's and Ubuntu's older versions) have proper runtime gettext support.

Changed in systemd:
status: Unknown → New
Revision history for this message
Martin Pitt (pitti) wrote :

@Gunnar: This patch does not actually work:

❱❱❱ xgettext -f "po/POTFILES.in" -o "build-deb/po/systemd.pot" --join-existing
xgettext: warning: file 'src/core/org.freedesktop.systemd1.policy.in.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/hostname/org.freedesktop.hostname1.policy.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/import/org.freedesktop.import1.policy.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/locale/org.freedesktop.locale1.policy.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/login/org.freedesktop.login1.policy.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/machine/org.freedesktop.machine1.policy.in' extension 'policy' is unknown; will try C
xgettext: warning: file 'src/timedate/org.freedesktop.timedate1.policy.in' extension 'policy' is unknown; will try C

And systemd.pot is unchanged.

I now committed https://salsa.debian.org/systemd-team/systemd/commit/09c6423728319 to simplify the .pot generation, but it has the exact same issue.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@Martin: That's disappointing. At least it was imported, as a result of the name change. :\

I just built 237-2ubuntu1 locally. No such warnings, and:

$ ls -l build-deb/po/systemd.pot
-rw-r--r-- 1 gunnar gunnar 16268 feb 14 22:07 build-deb/po/systemd.pot

i.e. systemd.pot includes the messages from the .policy files (the size of the one which LP imported is 1700 bytes).

Why does it work for me? Can it be that I have something installed which ought to be a builddep? The attached file shows some of my installed gettext related packages. Does it possibly ring a bell?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

The good news is that is doesn't matter much as long as upstream hasn't dropped the translations from the .policy files and added gettext-domain="systemd" ...

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

I may have solved it. :)

Added policykit-1 to Build-Depends and built systemd 237-2 in a PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/systemd

No warnings in the buildlog due the explicit xgettext() command. This might be the explanation:

$ dpkg -L policykit-1 | grep its/
/usr/share/gettext/its/polkit.its
/usr/share/gettext/its/polkit.loc

Revision history for this message
Martin Pitt (pitti) wrote :

Thanks Gunnar for tracking this down! Adding a policykit-1 build dependency requires some thought, as that also build-depends on systemd [1], thus this is circular. Also, there was a lot of effort with making systemd bootstrappable without excessive dependencies. But I think it's fine to add this as a [!stage1] b-dep.

[1] https://anonscm.debian.org/cgit/pkg-utopia/policykit.git/tree/debian/control

Revision history for this message
Martin Pitt (pitti) wrote :

I confirmed that the current "ninja -C build-deb/ systemd-pot" command also builds a complete .pot file with policykit-1 installed (unsurprisingly, as this also just calls gettext). So that part is fine.

What is really bad however, is to build-depend against policykit-1:

The following NEW packages will be installed:
  cgmanager dbus libcgmanager0 libnih-dbus1 libnih1 libpam-systemd libpolkit-backend-1-0 libprocps6 policykit-1 procps systemd
  systemd-shim

This is an awful lot to pull into a buildd schroot, in particular it makes systemd build-depend on itself. I'd really like to avoid that.

Are the files /usr/share/gettext/its/polkit.{its,loc} only being used at build time, or does polkit need these at runtime for dynamic gettext translations? My gut feeling is that it's the former, and then it would make sense to move these two into libpolkit-gobject-1-dev instead. systemd already build-depends on that, thus it will automatically pick these up.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

This is my test on artful:

I made this modification:

$ cat /usr/share/polkit-1/actions/org.freedesktop.locale1.policy | grep '"systemd"' -A 1
                <message gettext-domain="systemd">Authentication is required to set the system locale.</message>
                <defaults>

i.e. I dropped the inline translations for that message.

Then I renamed /usr/share/gettext/its/polkit.{its,loc}:

$ ls /usr/share/gettext/its/polkit*
/usr/share/gettext/its/polkit.its.bak /usr/share/gettext/its/polkit.loc.bak

I switched to a non-admin user and run:

localectl set-locale LANG=en_US.UTF-8

and the prompt was translated dynamically. So yes, they are probably only used at build time in some sense.

But people use xgettext() and similar tools manually too, and if we move those files to libpolkit-gobject-1-dev, you'd need to install that package in order to extract messages from polkit files.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Maybe put polkit.{its,loc} in a separate package (e.g. policykit-1-gettext) which policykit-1 depends on, and which systemd could add to Build-Depends.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

I notice that gschema.{its,loc} is included in libglib2.0-dev, so simply moving polkit.{its,loc} to libpolkit-gobject-1-dev wouldn't be the only example of xml format descriptions provided by a -dev package.

I attached a policykit patch (Debian) with this change.

Changed in policykit-1 (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Jeremy Bícha (jbicha) wrote :

Martin, a small number of packages Build-Depend on policykit-1 only for the .its file so moving that file will mean we'll need to update those packages. One example is gnome-multi-writer.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Since moving polkit.{its,loc} would affect other packages, I made another effort to fix it within systemd, and I think I made it this time. Please see the attached patch (for Debian).

It's new variant of xgettext() call, but this time it makes use of polkit.its from the systemd source. It's a bit ugly; the "|| true" is there to compensate for a segmentation fault, but that happens only after xgettext() has done its job, so systemd.pot actually gets updated.

It's worth mentioning that the upstream source currently includes po/its/polkit.{its,loc}, but OTOH - as a result of the upstream issue I submitted - they are about to drop those files. So that's the reason why the patch adds polkit.its.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

All this has now been fixed upstream. You can disregard the previous patches I've polluted this bug report with. The attached patch is a concatenation of the applicable upstream commits. It includes:

* Translations are no longer merged into the .policy files
* 'gettext-domain="systemd"' added to the .policy files
* The ninja() call in debian/rules (so far only in Debian git) now
  extracts strings also from the .policy files.

I suppose that this patch should better be committed to Debian.

(And no need any longer from a systemd POV to consider a change of the policykit-1 packaging.)

no longer affects: policykit-1 (Ubuntu)
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

And this is the debdiff with which I tested the patch successfully on top of systemd 237-2ubuntu2.

https://launchpad.net/~gunnarhj/+archive/ubuntu/systemd

Please note the need to refresh
debian/patches/debian/fsckd-daemon-for-inter-fsckd-communication.patch.

Revision history for this message
Martin Pitt (pitti) wrote :

Thanks Gunnar, nice work! I cherry-picked the patches in https://salsa.debian.org/systemd-team/systemd/commit/87f54958bc24 . The debian/ changes were already in Debian master.

Changed in systemd (Ubuntu):
status: In Progress → Fix Committed
Changed in systemd:
status: New → Fix Released
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks!

I changed the source tree path of the template in Rosetta from build-deb/po/systemd.pot to po/systemd.pot.

Let's follow up it when systemd 237-4 has been built on Ubuntu.

Changed in ubuntu-translations:
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

systemd 237-3ubuntu1 has been uploaded including these changes, and a complete systemd.pot was imported to LP. Case closed. :)

Changed in ubuntu-translations:
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 237-3ubuntu3

---------------
systemd (237-3ubuntu3) bionic; urgency=medium

  * tests/control: drop qemu-system-ppc.
    Whilst some tests pass, many regress / fail to boot. This is not a regression,
    as qemu-based tests were not run previously.

 -- Dimitri John Ledkov <email address hidden> Tue, 20 Feb 2018 17:40:02 +0000

Changed in systemd (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.