Translations for all languages in Launchpad partly lost

[Ferdinand]

on September 8th I updated almost all "addons" German translation
today many terms are marked as untranslated again.

IMHO someone must have loaded old data.

opening the untranslated terms in
https://translations.launchpad.net/openobject-addons/trunk/+lang/de
launchpad suggests all my former translations

IMHO it's necessary to make sure that the translation process does not loose data.

[Jan Verlaan (jan-verlaan)]

Same issue encountered for the Dutch language, and that is a moderated team with permissions!
Resulting that I have the feeling that something is changed by someone with more rights as this is noticed with more then one language.

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello,

We are working on synchronization of launchpad translation with addons branch automatically, And we have specified export branch with trunk addons branch.

Thanks

[Ferdinand (office-chricar)]

please specify what translators should do .... translate again ? .. wait ? for what ?

[Jan Verlaan (jan-verlaan)]

Good to see that a automatic sync is made from trunk to stable version if I understand correctly.
That means we still do our translations in trunk and will be after sync in stable too.

nevertheless I have worked 8 hours last period on translations and they are gone now (well, it is switched from translated to suggestion now)

What is your advice for now. Will our translation be back after a while or have we to start over again, and most important, can we start now or should we wait till your sync work is done?
It would be good to inform the translation teams about this change and mention the risk of losing translations during a period! Thus prevent them for useless work.
Thanks

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello jan,

Automatic translation will be apply (or synchronize automatically) from launchpad translation to bzr branch (lp:openobject-addons) not from trunk to stable.
So it means if the launchpad user changes some translation files for any module launchpad will commit that changes on addons branch next day..

So whatever changes made by someone from launchpad traslation will be commited(exported) to bzr branch daily bases (launchpad team will commit translation on addons branch daily) if they found changes.

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

mra, please explain more precisely where should we make translation not to lose it.
Should we translate in Trunk series?
(what is recomended in Launchpad on
https://translations.launchpad.net/openobject-addons on Translation tab)
Can you give url for example language
Should it be:
https://translations.launchpad.net/openobject-addons/trunk/+lang/pl
for my language?

Are they any rules that translation cannot be translated and verified by the same person? My translation was also dismissed to suggestions.

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello,
I have asked the problem to launchpad team today, So hope they can help us to solve this problem.Please wait for 1-2 days..
We will also check this problem here.

[Jeroen T. Vermeulen (jtv)]

This may be because of an issue we discussed a number of days ago, but I think also longer ago: the language code for German is de, the one for Dutch nl, and so on. But some translation files in the bzr branch use codes like de_DE ("German as spoken in Germany only") and nl_NL ("Dutch as spoken in the Netherlands only").

These translations are being imported as de and nl, respectively; and the bzr export procedure exports them with names based on these normalized language codes. I think in 2008 I contacted the project about this, since there were wildly incorrect language codes such as ar_AR ("Arabic as spoken in Argentina"), cs_CS ("Czech as spoken in the Republic of Serbia and Montenegro," a country that doesn't exist any more). I still see those mis-named PO files in the branch. So sometimes there are three competing PO files: cs.po (correct), cs_CS.po (incorrect), and cs_CZ.po (the redundant spelling of cs.po).

So now you're importing both de_DE.po and de.po from your branch, both for the same German translation. That introduces the risk of overwriting translations, especially since the de_DE etc. have nonstandard language codes and so may not be auto-approved. In your branch now I see 112 cases of de.po and also 112 of de_DE.po; same for nl.po and nl_NL.po.

There's a way of checking whether this is what's going on: see whether the translations that are affected are all ones that used the problematic language codes. Looking at the "wiki" template as an example, the only completely correct language codes I see are: el, es_AR, gl, ko, pt_BR, sr, tlh, zh_CN, and zh_TW. If none of these is affected but many of the others are, then this is probably the source of the problem.

If this is the problem, how to fix it? The best long-term solution of course is to normalize the language codes: de_AT for Austrian German but de for German in general, nl_BE for Flemish but nl for Dutch in general, and so on. If that is not an option, you could consider replacing de_DE.po, nl_NL.po etc. with softlinks to de.po and nl.po etc.

To restore the deactivated strings once the naming is sorted out, I'd produce single up-to-date unified PO files by merging de_DE.po and de.po (etc.) using the msgmerge tool. You may have to figure out which of the two sets of PO files, if any, is the most up to date. Above all, try out the exports in a separate branch to see that everything is completely as expected before sending them back into your development/import branch.

[Raphaël Valyi - http://www.akretion.com (rvalyi)]

BTW, I may request here that you update the pt-BR translations from Rosetta to the .po OpenERP files at the same time because the pt-BR (Brazil) localization is making good progress. Thanks.

[Jan Verlaan (jan-verlaan)]

I can see our translations are restored :-)
Thank you very much. (happy)

But can someone tell me the procedure we have to follow from now on? As far I understand now.
We do preferred our translations in trunk. These will be synced with addons (and vise versa)
But should we do something manually to get them in stable? I can export and then import, but I hope this will bedone with every new version automaticly. (but it isn't done by anyone from April onward yet)

Perhaps someone can write a instruction for the language teams so they can adapt the procedures into their instructions?

[Ferdinand (office-chricar)]

I can not confirm Jan's statement - I think German translation is not restored completely yet

[Pieter J. Kersten (EduSense BV) (pieterj)]

A quick survey of nl.po and nl_NL.po files suggests that the latter contains invalid - read: very old - translation terms. That raises some questions:

1. Where does the nl.po and nl_NL.po come from? The contents suggests nl.po gets updated, but nl_NL.po does not.

2. How can we differentiate between the two in launchpad? (AFAIKT, I can't)

3. As the default res_lang table in OpenERP only contains xx_YY and not xx for ANY language, how does the loading process get the right - updated - translations? Does it even recognise/touch the xx.po file? As a user, I can't select Dutch (nl) as a language, only Dutch_Netherlands (nl_NL)

I suspect a discrepancy between OpenERP's language scheme and launchpad's language scheme, resulting in loading the wrong files.

Can someone clarify / correct / second this?

If this appears to be true, then I suggest one of the following routes to get out of this mess:

1. Clean up OpenERP's language scheme, meaning extending it with two-letter language tags and dropping unneeded localized language tags. Verify that the loader loads both and create fallback language schemes in the software. Needs adaptation of the translation software in OpenERP itself (when updating terms from within a client, which language tag should be used?) (Best route, takes a while)

2. For every xx.po, map the xx.po file to a default xx_YY.po file, remove the lingering xx.po file and hope for the best. (Shortest route, quick fix, may ignite a war).

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

Polish also is not restored. Story is that 17 sep all my new translation (appr 60% of whole translation) was marked as "packaged" (green) and next day 18 sep was dismissed to "needed review". I have worked only on Launchpad. I have never uploaded any files. In Polish shouldn't be code language problem we always have pl_PL with no variations. Maybe it could be a problem that one uses pl and other pl_PL but I even don't know haw to manage with it. I don't see any language code in PO file. pl or pl_PL is not on the list in Jeroen T. Vermeulen answer so it something wrong with it.
Regarding unifying code language I think we should use just pl. In downloaded files I see the name like this:
addons_account_i18n_account-pl.po
So I think anybody working on translation uploads file with the same name. Who should manage the country codes and inform everyone what is the current standard?

[Jeroen T. Vermeulen (jtv)]

Don't remove the xx.po files; they'll be re-written by the export process!

Also, addons_account_i18n_account-pl.po is not a good filename for a Polish translation. The filename should be just the language code followed by ".po": pl.po. The extra text only makes it impossible for Launchpad to guess that the -pl stands for a language code and not for, say, Perl. You indicate the template by putting the PO file in the same directory as the template.

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

Thats the name ("addons_account_i18n_account-pl.po") we got when downloading file. According to your indication I understand when we download such file, edit in POEdit and upload back the name before uploading should be changed to "pl.po". Am I right?

If so someone should change the Launchpad tip in uploading option which now is:
"Updated translation
You have exported a PO file from Launchpad, worked on it off-line, and now you want to contribute your updates back."

But problem of disappearing translation still exist. I wrote I have never uploaded files. I translated everything using web interface. I stopped to work because I don't want to lose translation again. What is the suggestion for now. Should I review translation and mark it as reviewed again? Eventually I can do it it. Its fresh translation. But I would like someone to revert translation I have already made. It was 95% correct. Otherwise I would think Tiny or Launchpad doesn't like contributors.

[Jan Verlaan (jan-verlaan)]

Please can someone explain how we (the translationteams and/or individual translators) must proceed now in launchpad translations? The Dutch team did use only launchpad for translations so how could we corrupt the translations?
I guess I'm lost now! and can not proceed with translations work.

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

Again polish translation is lost. This time all my translation from module base is dismissed to suggestions. This module was left (not dismissed) on September. I see that quite a lot of countries is developing their translation before 5.2. Again everything will be lost?

[Jan Verlaan (jan-verlaan)]

Dammed, same for Dutch!!
All overwitten bij commit of Fabien 18 hours ago!
Please restore the situation. All our work done in the last month is gone. Not happy at all.

[Ferdinand (office-chricar)]

for German I have a complete tar file dated Nov 5th with my latest contributions (learned from the last desaster :-| )

[Pieter J. Kersten (EduSense BV) (pieterj)]

I too learned from the past - Jan, I have a - sort of - backup. Not all
is lost.

Op dinsdag 10-11-2009 om 12:31 uur [tijdzone +0000], schreef Ferdinand @
ChriCar:

> for German I have a complete tar file dated Nov 5th with my latest
> contributions (learned from the last desaster :-| )
>

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

In new version of base module for translation in web interface I see terms like below. Is it really for translation? And now I have doubt because base module terminology probably will be changed in launchpad again droping my work again.

It is term 237 in base:

Model %s Does not Exist !" % vals['relation']))

if self.pool.get(vals['model']):
self.pool.get(vals['model']).__init__(self.pool, cr)
self.pool.get(vals['model'])._auto_init(cr, {})

return res
ir_model_fields()

class ir_model_access(osv.osv):
_name = 'ir.model.access'
_columns = {
'name': fields.char('Name', size=64, required=True),
'model_id': fields.many2one('ir.model', 'Object', required=True),
'group_id': fields.many2one('res.groups', 'Group

[Grzegorz Grzelak (OpenGLOBE.pl) (grzegorz-og.pl)]

I see that automatic translation update has run. Among them this for base module. But still text for translation is destroyed (as see above). I checked that it is for other languages too. Fe for french. I checked PO and POT files in trunk. It looks that POT file for base module is wrong. I cannot read it in poedit. I can read POT files for other modules.

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello grzegorz,

yes you are right, before two days we have made setting on LP for automatic synchronization with bzr branch of addons, so that we can see updates :)

And for the base module problem please check this bug:
https://bugs.launchpad.net/openobject-addons/+bug/484298

We will work on this bug soon.

thanks..

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello,

I have tested synchronization between LP and bzr branch of trunk addons, It works fine for me.
I tested like this:

I have modified de.po file available on branch and commit that changes on that branch, after that i have opened LP translations and i got that changes there .
And in reverse i have modified LP transaltion and i have get updated that changes next day on trunk addons branch... For this i have used base_module_quality module for test..

So at our side its works fine.

thanks,

[Jan Verlaan (jan-verlaan)]

Thanks mga, that would mean we would have a stable situation right now and could proceed with translations.
But what about the already translated, corrected and accepted translations done before the 9th of November? The so called "lost translations" as discriped in this bug tracker. These are set back in status "Needs review". For e.g. Dutch in addons these count 3693 translations now and it was arround 60 before this date.

Can we expect those will be restored soon or do we have a situation where we have to accept this and start over again?

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello jan,

Yes we have stable situation, I just telling you whatever you modify on LP translation will be modify on bzr branch addons with file name like fr.po not with fr_FR.po, So fr_FR.po will not getting updated but fr.po will modify by LP translation auto synchronization...

i am not sure about your problem about restoring lossed transalted terms. But i think it will not restore again but we can wait for 4-5 days ? if its not going to restore we have to do manually again..

And yes we will update all trunk addons module's transalation files with correct iso code...i mean => fr_FR.po will be rename by fr.po soon, i will inform you people soon about this changes..So please wait for it...

thanks

[Raphaël Valyi - http://www.akretion.com (rvalyi)]

Hello guys,

I'm wondering if I'm missing something, because on the 5.0 series of addons, I still see there is NO automatic launchpad synchronisation:
https://translations.launchpad.net/openobject-addons/5.0/+translations
"This project is currently not using any synchronization with bazaar branches."

So @mra, my question is: why isn't the 5.0 synch activated yet? Do you plan to activate it? Do you need more trunk testing, are you asking to wait 4-5 days before activating it?

Thanks for any explanation (I'm taking care of the Brazilian localization and lot's of translators are waiting to see their work included).

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello Raphael,

yes we are working right now on trunk addons synchronization and stable 5.0 series is currently not activated , we will activate synchronization of stable 5.0 series after completely apply on trunk addons series soon. So it may take some days to implement on stable 5.0 series..

thanks

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello guys,

We have updated trunk server and trunk addons, so kindly update your same branches to get launchpad translation work.

i mean now when you load translation wizard from client that time fr.po file will consider now not fr_FR.po to load french language..

Note: If you have fr_BE.po (french language speak in belgium) that time it will consider that file only not fr.po so no problem for that.

We are still working on launchpad translation with server,client and try to make more clear it..

Thank,

[Ferdinand (office-chricar)]

German - modules base and dm still contain strings from other modules.

[Ferdinand (office-chricar)]

@mra
pls can you set up a help page for translators how to translate strings in correct context
frequently a translation does not fit into context, but this is only "visible" opening the application (form or report)
while it is more or less easy to fix translations in forms even reports using the in form translation or the translation menu in openerp it do not see yet an easy/comfortable way to bring the translation back to bzr or launchpad.
thank you

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello Ferdinand,

I have two question in context of your last post:

1. Where do you want to set up help of translations?
2. You mean to say that when Administration/Translations/Applications Terms/All terms , when user update Transalation value then it should be modify on bzr or launchpad ?

can u please clear this points?

And for the German - modules base and dm still contain strings from other modules. => We will try to separate modules (base, dm, etc..) from server and trunk-extra-addons to its tranalastion branch...

thanks..

[Ferdinand (office-chricar)]

@ Jay

ad 1). Where do you want to set up help of translations?
may be here
http://doc.openerp.com/developer/Appendices/appendices_b.html#translations

ad 2.)You mean to say that when Administration/Translations/Applications Terms/All terms , when user update Transalation value then it should be modify on bzr or launchpad ?
-> well there are
a) English words which have multiple meaning which can only be translated correctly seeing the context (form/report)
b) German translations (especially many compound words) tend to be much longer than English sources messing up the layout - especially of reports.
Sometimes it is only possible to find out which term is used where by trial and error or inspecting the code which is very time consuming.
IMHO it would be a great help for translators to be able to push "online" translations to bzr/launchpad using a button "publish translation"
the "export translation" does not "work" for this, because it's not easily possible to store the exported file on the server side bzr dictionary.
And translation errors of the kind mentioned above are typically to be fixed in a production system on the fly. To expensive to set up a test system just for this.

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello,

Just to inform you people that we have synchronized trunk addons, trunk client, trunk server branches translations with LP translations...

https://translations.launchpad.net/openobject-client
https://translations.launchpad.net/openobject-server
https://translations.launchpad.net/openobject-addons

thanks,

[Ferdinand (office-chricar)]

pls see base
https://translations.launchpad.net/openobject-addons/trunk/+pots/base/de/+translate?show=untranslated
first item
Located in code:addons/addons/account/wizard/wizard_open_closed_fiscalyear.py:0

same for dm

[Christophe CHAUVET (christophe-chauvet)]

Hi

Why in base module, we have term from product, mrp, account,....

I think, there is a problem when terms are extracted

Regards,

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello,

We have removed "base" module translations from openobject-addons series and move this to openobject-server translation. So you can now modify base module translation's from openobject-server branch only..

We are working on trunk-extra-addons branch translation, And yes we will also remove modules (base_module_doc_rst,dm.....etc) from addons branch also and move them to trunk-extra-addons branch.

And we will check also why base module translation files constains terms from product,account..

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

hello,

trunk-extra-addons translations available on https://translations.launchpad.net/openobject-addons/extra-trunk

Thanks

[Borja López Soilán (NeoPolus) (borjals)]

There are still problems with this "automatic translation updates", at least with Spanish and Catalan translations.

We just recently uploaded a module (account_balance_reporting) to the extra-addons with Spanish and Catalan translation files: es_ES.po ("Spanish as spoken in Spain") and ca_ES.po ("Catalan as spoken in Spain")

We added the module in the revno 4198, and then we saw this on the revno 4200:

revno: 4200
committer: Launchpad Code Hosting <codehost@crowberry>
timestamp: Sat 2010-01-23 04:49:00 +0000
message:
  Launchpad automatic translations update.
added:
  account_balance_reporting/i18n/ca.po
  account_balance_reporting/i18n/es.po
  account_renumber/i18n/es.po

As you can see, Launchpad imported our es_ES.po and ca_ES.po translations, and afterwards it exported them to the es.po and ca.po files. So now we have two files for the Spanish language (and another two for Catalan): "es_ES.po" is not being updated by Launchpad while "es.po" is.

Now comes the critical question:
- Which one is OpenERP loading? Is OpenERP loading the 'updated' "es.po" file or is using the 'old' "es_ES.po"? :)

[Mustufa Rangwala (Open ERP) (mra-tinyerp)]

Hello Borja,

es.po is correct name for Launchpad Transaltions and es_ES.po is correct name for Openerp Transaltions,

 If you have checked trunk addons/server/client we have updated/renamed all po files as they can automatic synchronize with LP Transaltions (for that we have also changed the server code for translation so that they can load new po files (es.po)) .. for e.g: we have changed the name of fr_FR.po => fr.po so now when LP commit on our branch it will directly commit with right file..
But in case of stable we did not renamed files so it creates problem..

 Which one is OpenERP loading? Is OpenERP loading the 'updated' "es.po" file or is using the 'old' "es_ES.po"? => It is loading es_ES.po file ...

So when LP User will change the translation from LP it will commit/update in es.po on extra-addons branch ..

so i request you to check one thing at your side... update es_ES.po file on branch commit it and see the changes come on LP translation (may be it wl tak 1 day to synchronize) or not (https://translations.launchpad.net/openobject-addons/extra-5.0/+lang/es) ... If the changes are comming in LP then we can stop the exporting from LP so that it will not create wrong files on branch .. And then we will check this issue..

Thanks for repoting...
Mustufa