KDE4 in Intrepid lacks translations for messages with context or plural forms

[KAMiKAZOW]

KDE4 has changed to using native gettext support for plural forms and context messages. However, we've copied over KDE3 templates which carried over source_file_format of KDEPO (=4 in DB). We still correctly imported them, but exported them in legacy KDE3 format.

This means that none of the context-containing or plural-using messages in Kubuntu are translated.

We have to fix this by finding all the affected templates (made harder by some packages remaining in legacy KDEPO format), and setting their source_file_format to PO (=1). This should not be too hard and I already started compiling a list of templates, but I want to clean up the entire KDE4 set-up in Launchpad along the way. We need to be careful to also update the timestamp on templates when doing this, or they might not be exported in "delta" language packs.

After this is done, we'll have to discuss preparing a new language pack for Intrepid as well.

[Harald Sitter (apachelogger)]

You should have seen it 5 days before release... We are trying our best, but with the Launchpad translation system our best is not even close to the best of KDE.

[Jannick Kuhr (jakuhr-deactivatedaccount)]

Yes, it's really a shame what happened. Especially from the point of view of an upstream translator this is so disappointing. Is there a bug report tracking this issue?

[Harald Sitter (apachelogger)]

Yes, this one.

I raised the topic of continuing issues with Rosetta various times at Kubuntu meetings we never came up with anything other than complaining. Though to be honest, I am tired of complaining and I would actually understand if upstream translators denied supporting Kubuntu in any way.

[Sel Goona (alieneye)]

Are there any problems to be expected if we use another Language-Pack? I have installed the "kde-l10n-de_4.1.2-1_all.deb" from Debian experimental from here:

http://packages.debian.org/de/experimental/all/kde-l10n-de/download

After that my Intrepid is, until a few exceptions nearly full translated into German. Do I have to fear that I get a problem later, if regular Intrepid-Updates are coming?

[Harald Sitter (apachelogger)]

Yes, once we start deploying 4.1.3 kde-l10n-de will be replaced with our crippled version. However you should be able to prevent this by fiddeling with apt's priority settings (google will know more).

[jpfle (jpfle)]

I use few KDE applications in Gnome (Ubuntu 8.10) and I have language problems with some.

Here's the applications that don't have language problem:

- Amarok 1.4.10
- K3b 1.0.5

Here's a list of the applications that have language problems:

- Akregator 1.3.1
- Dolphin 1.1
- KAlarm 2.0.4
- Kate 3.1.2
- Konqueror 4.1.2
- Konsole 2.1
- KRuler 4.1.2
- KSnapshot 0.8.1
- Le centre d'aide de KDE 4.1.2
- System Settings

I observed that the applications that have language problems often not translate "Get Help Online" and "Translate This Application" in the Help main menu entry, and the little "Help" link to the bottom of the pop up windows.

Some screen shots would be useful or not?

[KAMiKAZOW (kamikazow)]

Amarok and K3b are both KDE3 apps. While both had problems in 8.04 <http://www.flickr.com/photos/19616885@N00/2535182834/in/set-72157608562200171/>, their shape was never as bad as the one of KDE4 apps.

[jpfle (jpfle)]

I attach few screen shots to show some examples of my language problems with the French translations.

In the screen shots, I mark in blue the words not translated in French.

[Данило Шеган (danilo)]

Launchpad Translations had KDE templates marked as legacy KDE format ones, so it exported context and plural form messages incorrectly.

I've been looking at this, and I want to make sure we clearly mark all relevant KDE4 templates. Digging which are which is not the easiest thing around, so it may take a few days.

Once that's done, we'll still have to prepare a new language pack for Ubuntu, but we'll have to coordinate that with Ubuntu guys.

[Данило Шеган (danilo)]

I would also like to thank you guys for posting screenshots: that helped pin-point the problem.

However, I can't find the strings at all from initial Kontact screenshot, and I looked upstream on l10n.kde.org, in OpenSuse kde4-l10n-de packages and our tarballs (eg. a string like "Check Mail"). This makes it hard for me to see what's going on, so if anybody can dig out where the string comes from, I might be able to see if this is the same problem, or maybe something else.

For reference (I am going to change the initial description to better reflect what appears to be a major problem), initial screenshots are at (you can also use "view original description" link):
 http://www.flickr.com/photos/19616885@N00/2991042741/
 http://www.flickr.com/photos/19616885@N00/2991047111/

[Harald Sitter (apachelogger)]

svn co svn://anonsvn.kde.org/home/kde/branches/stable/l10n-kde4/templates
grep -r -i 'check &mail' messages/

Mail is actually &Mail due to the keyboard shortcut.

[Harald Sitter (apachelogger)]

That should of course be:
svn co svn://anonsvn.kde.org/home/kde/branches/stable/l10n-kde4/templates/messages

[Niels Slot (nielsslot)]

I'd like to mention that this bug also effects KTurtle. In Intrepid's KTurtle it's not possible to executes scripts written in any other language then English.

KTurtle uses contexts for translation (the ki18nc method is used). A workaround is installing the Debian kde-l10n-* packages.

[Mauricio Piacentini (piacentini)]

We are starting to get scattered reports of problems with translated scripts in KTurtle. In our case a broken translation means a broken application, not only a broken UI, as scripts are translated on-the-fly. I am not familiar with how *ubuntu updates are processed, what is the expected (or average) turnaround time for a fix to this to be available in the form of an update to users, now that the source of the problem is identified?

[Harald Sitter (apachelogger)]

Danilo needs to fix the issue in Rosetta (few days), then we might be lucky and the strings don't need to get reimported into Rosetta (otherwise that would be another 2 days or so), then the updated packages have to be exported and uploaded (1 day) and then they usually stay in a special repository for about a week until it's proven that they actually fix the issue (and don't cause regressions) following which they will be moved to the actual updates repository.
So I guess we are facing anything between a couple of days and 2 weeks.

Is a bug report about the KTurtle issues around? I'd like to find out if the bug triage squad may have messed up, because a broken KTurtle would certainly have forced us to choose a more reliable translation approach.

[Mauricio Piacentini (piacentini)]

I do not think we had a KTurtle bug in this system: users found out the problem after 8.10 was released. I only registered here yesterday to track the issue, so we can assist them better. Niels Slot (also recently registered) is another KTurtle developer.
Grigoris Markakis for example reported it in the Kturtle discussion group on this thread:

http://groups.google.com/group/kdeedu-kturtle/browse_thread/thread/d0806c28a887285a

See the message from Oct 31, concerning Ubuntu 8.10 and missing functionality. Basically if there is no context then automatic scripting translation does not work, and the application only works with English scripts. There is a menu item named Script Language that controls this. I do not have a Kubuntu 8.10 system to test, however, but others have confirmed it.
I know it is a moot point, but if you need one more voice against reprocessing upstream translations, please add mine. The pt-BR team is also facing lots of issues with this, why not simply use the upstream translations, which are the ones we do QA against?
Thanks for your efforts in solving this, Harald!

[Harald Sitter (apachelogger)]

I find it concerning that official KDE applications don't get enough testing from the Kubuntu community to find such issues before a release, especially considering that KTurtle probably was broken all along, since the partial translation packages only got a created a few days before release of 8.10 (i.e. 6 months of brokenness without anyone noticing). We definitely need to do something about that.

About Rosetta:
There is no reason for _re_processing. Still it is happening and the Kubuntu team members who will be attending the Ubuntu Developer Summit in December are going to discuss this in deep. It's not just the possibility of reprocessing already translated strings but other thins that contribute to the awkward status of localization in Kubuntu. The current plan is to aggregate a list of shortcomings and issues in Rosetta, which should be taken care of in order to make everyone happy with the way we handle translations. Also it is possible that we are going to assign someone to take care of the Rosetta <=> Kubuntu interaction to minimize problems, speed up communication and to have a contact person for both teams.
In short: we are going to work on a sensible solution for all parties.

[Данило Шеган (danilo)]

Launchpad issues have been fixed yesterday (and they affected "only" 113 templates, which is not too bad considering the total number of KDE4 templates). This is partly the problem of Launchpad translations, which I filed as bug 294198.

We also found an issue with language pack building in Ubuntu (eg. kturtle was not affected by Launchpad issues at all, it's just that PO files were processed on the build machine which had a really old gettext which ignored all msgctxt entries).

So, the next step is for us to roll out a new language pack. We'll be doing that tomorrow after we also copy new translations done only in Hardy to Intrepid (see https://lists.ubuntu.com/archives/ubuntu-translators/2008-November/001911.html for details). Once we roll the language pack out (basic reason is to fix KDE4 issues), please help us test it.

As far as your other concerns go, part of the problem is that Ubuntu lacks a dedicated translations coordinator (or a team). This is being actively worked on (Canonical is hiring a new person to take that job full time), and I am sure this will make huge impact on how Ubuntu translations work and appear.

[Данило Шеган (danilo)]

We've fixed all the issues we are aware of in Launchpad, and a new language pack tarball (sources for language packs) are ready. Once actual language pack packages are ready, please test them and report any problems you find (that's the only way we can actually resolve any problems that come up): preferably in new bug reports unless it's as widespread as this issue.

Thanks!

[Chusslove Illich (caslav-ilic)]

KDE language packages also miss POs from qt/ module, e.g. for German KDE 4.1.2:

  http://websvn.kde.org/tags/KDE/4.1.2/kde-l10n/de/messages/qt/

Possibly an import problem, because it's the only module in upstream packages which does not start with kde*?

[Данило Шеган (danilo)]

This is a bug in Ubuntu's qt4-x11 package which is not generating any POT files. I've manually imported POTs (kdepo.pot, libphonon.pot, phonon_gstreamer.pot) for KDE 4.1.2 on request of Jonathan Riddell (Kubuntu guy), which is hardly a solution (changes from upstream Qt won't propagate to Launchpad). Everything else (translations for these potemplates) should be picked up automatically by Launchpad from now on, but one should probably report a bug against qt in Ubuntu to make sure it's fixed permanently (Riddell promised to fix it in Jaunty)

We might need to regenerate full language packs for Intrepid again (after all translations are imported), though.

[Chusslove Illich (caslav-ilic)]

I'm not sure that packaging of kdeqt.po and others should be linked with Qt. These templates have nothing to do with Qt proper; rather, it is KDE project which is extracting them, with custom (and not quite straightforward) mapping of Qt message elements to PO format fields, and kdecore.so which is sourcing translated catalogs. E.g. a pure Qt, non-KDE application wouldn't be able to make use of these POs (unless, of course, it replicates internals of KDE libraries).

[Данило Шеган (danilo)]

Launchpad Translations has a well defined interface towards Ubuntu packages (basically, they have to produce a POT file during *build*, and that file is not included in the binary package). It's impossible for us in Launchpad to track what Ubuntu packages are doing differently. Thus, it's up to Ubuntu packagers to worry about this (they are also free to ask us for different interfaces, but that's up to them as well).

[Jonathan Riddell (jr)]

This script is used to generate kdeqt.pot http://websvn.kde.org/trunk/l10n-kde4/scripts/MessagesQt.sh, I can add that to our Qt package.

Phonon just needs QTONLY=yes set before running extract-messages.

I don't have time to fix this before I go away on holiday this evening but I'll make sure to do it as soon as possible thereafter.

[Arne Goetje (arnegoetje)]

We currently have new language packs in intrepid-proposed. Please test if they fix this issue or not and give feedback. Thanks.

[jpfle (jpfle)]

From intrepid-proposed, I've updated these packages:

language-pack-kde-en (1:8.10+20081025) to 1:8.10+20081107
language-pack-kde-en-base (1:8.10+20081025) to 1:8.10+20081107
language-pack-kde-fr (1:8.10+20081025) to 1:8.10+20081107
language-pack-kde-fr-base (1:8.10+20081025) to 1:8.10+20081107

The update resolved a lot of language bugs, but there's still some problems (see next messages).

[jpfle (jpfle)]

First, these sentences aren't still translated: "Get Help Online" and "Translate This Application" in the Help main menu entry of programs.

[jpfle (jpfle)]

In Akregator, the welcome message in the main window is now translated, but it has now a style bug (see the blue arrows in the attached screen shot).

[jpfle (jpfle)]

In Dolphin, there's still not translated strings in the Places panel (see the blue dots in the attached screen shot).

[jpfle (jpfle)]

In Kate: in the Tools menu, there's a string not translated: "External Tools".

In KRuler, there's still a string not translated in the "Choose Color..." menu (see the blue dot in the attached screen shot).

[jpfle (jpfle)]

In the khelpcenter, there's no help translated. I put here an example with the Konqueror handbook.

[karaluh (karaluh)]

In polish "cancel" is translated as "delete", that is after uprading language packs from proposed.

[nh2 (nh2)]

Question: Is this related to https://bugs.launchpad.net/ubuntu/+source/kdebase-workspace/+bug/298592 in any kind?

[karaluh (karaluh)]

Yakuake isn't translated at all.

[Harald Sitter (apachelogger)]

nh2: not impossible, needs furhter investigation though
karaluh: that is only partly related, it appears the yakuake strings didn't get imported at all because it is using bogus packaging. please report a bug against yakuake.

[karaluh (karaluh)]

> please report a bug against yakuake.
https://bugs.launchpad.net/ubuntu/+source/yakuake/+bug/305414

The following toolbar items in kmail aren't translated:
New
Check Mail
Previous
Next

menu items:
File->New->Message from template
File->Check mail in favourite folders
View->Size column
Folder->Rebuild index
Message->Forward->With custom template
Message->Mark message->Mark message as action item
Message->Mark message->All three message tags
Message->Create To-do/Reminder
Help->Get help online
Help->Translate this application

[jpfle (jpfle)]

I have also a translation problem with Lokalize (version 0.2). See the blue marks on the attached screen shot.

[KAMiKAZOW (kamikazow)]

I don't understand why Canonical claims that a fix has been released. I just compared KDE 4.2 on openSUSE to KDE 4.2 on Kubuntu:
http://www.flickr.com/photos/19616885@N00/3277153714/
http://www.flickr.com/photos/19616885@N00/3276333503/

Stragely this time other strings are untranslated (eg. Dolphin's status bar).

[Jonathan Thomas (echidnaman)]

"I don't understand why Canonical claims that a fix has been released."
The problem stated in this bug that brought about a massive lack of translation was a bug in Launchpad that caused strings with context information or plural forms to not be imported. The software *has* been fixed to this point and translations with context info or plural forms *are* being imported by Launchpad. You can see for yourself: https://translations.launchpad.net/ubuntu/jaunty/+source/kdebase/+pots/dolphin/es/+translate

The first string there "Empty trash" has context information right below it. It's existence on that page shows that this bug is indeed fixed and that the upstream translations that include context and plural forms are, in fact, being imported.

Which brings us to your comparison. Regardless of whether you are using the KDE 4.2 packages in Kubuntu 8.10 or Kubuntu 9.04, both are using the language packs from Kubuntu 8.10, which are in turn based on the KDE translations for KDE 4.1. As strings have changed since KDE 4.1, the translations included in the KDE 4.1 langpacks become outdated and useless in KDE 4.2 where new, different strings are in place.

As translation focus has only very recently shifted to Kubuntu 9.04, no lang packs have been generated for it yet. Once they are, most of the untranslated strings you are seeing due to the old langpacks should be gone.

[KAMiKAZOW (kamikazow)]

I'm not sure what you guys did, but you actually managed to make the translations even worse in 9.04.
Compare 9.04 to my older 8.10+KDE4.2 screenshot:
http://www.flickr.com/photos/19616885@N00/3276333503/in/set-72157608562200171/
http://www.flickr.com/photos/19616885@N00/3433123984/in/set-72157608562200171/

Almost exactly 2 months ago Jonathan Thomas wrote "translation focus has only very recently shifted to Kubuntu 9.04". Isn't the final release of Kubuntu 9.04 not supposed to ship in 11 days from now?? Is there any chance you actually fix something in the four days until you hit LanguagePackTranslationDeadline??

Oh, and PS: When I opened this bug, it wasn't about plural forms, etc. -- just "ship upstream translations of KDE" which you still haven't done. Hence I treat your "Fix Released" claim as nonsense.

[Jonathan Thomas (echidnaman)]

That's bug 355814. It's being worked on, so things should be much better next time the langpacks are generated. Up until the point that the aformentioned bug hit, the translations were nearly perfect.

[Jonathan Thomas (echidnaman)]

Anyway, current translations should have messages with context and plural forms.