& strings not stayed "as-is" on Rosetta

[Jordi Mallach]

---------- Forwarded Message ----------

Subject: & strings not stayed "as-is" on Rosetta
Date: Sunday 23 April 2006 15:11
From: atie <email address hidden>
To: <email address hidden>
Cc: <email address hidden>

I'm translating the packaging guide from Rosetta, having a problem which is:
&lt; and &gt; are not stayed "as-is" somehow. To begin with, I never
translate &lt;, &gt; or other & special characters to <, > or something. But,
problem was caught by Matthew at Doc Team yesterday, I corrected them, the
packaging guide successfully imported to the ubuntu-doc package.

However, before writing this mail, I opened the translations from Rosetta
again and could see the message strings which I corrected yesterday were
changed back with < and >. I have corrected few of them again, but I am
feeling it's probably wasting of time.

I am wondering someone else having same problem. And, hope Rosetta admins (or
developers) take a look at this problem. I am translating the guide to Korean
from Rosetta with Firefox. The problematic strings have at least two
translation suggestions - one with &lt; and &gt;, another with < and > - but,
unfortunately latter taken in place somehow.

--
rgds,
Atie

[Matthew Paul Thomas (mpt)]

This may be a duplicate of bug 6446.

[Björn Tillenius (bjornt)]

On Wed, Apr 26, 2006 at 12:25:29PM -0000, Matthew Paul Thomas wrote:
> This may be a duplicate of bug 6446.

I don't think this is a duplicate of 6446. I think this bug will be
fixed by
sftp://chinstrap/home/warthogs/archives/carlos/launchpad/bug-39879,
which is up for review. Carlos, can you confirm?

[Carlos Perelló Marín (carlos)]

Looks like that, yes.

The only thing that makes me doubt is that I understand this bug report as they were writing '&lt;' and '&gt;' and Rosetta was giving them back '<' and '>' and this sounds like a problem with the submit, my fix is for the rendering that is related but not the only problem here.

I can only debug this problem if I get a URL for the concrete message that is giving problems.

atie, would you point me to the concrete message?

Thank you.

[Björn Tillenius (bjornt)]

On Wed, Apr 26, 2006 at 02:19:42PM -0000, Carlos Perelló Marín wrote:
> Looks like that, yes.
>
> The only thing that makes me doubt is that I understand this bug
> report as they were writing '&lt;' and '&gt;' and Rosetta was giving
> them back '<' and '>' and this sounds like a problem with the submit,
> my fix is for the rendering that is related but not the only problem
> here.

That's what happen when you use structure in the textarea. They write
'&lt;', and Rosetta correctly stores it as '&lt;'. But then when it's
displayed, it's not quoted, since 'structure' is used, so the browser
displays it as '<'.

Since the reporter said that the translation was successfully imported
into the ubuntu-doc package, I'd assume that this was a display problem
only, and that the translation was stored correctly in the database.

[atie (atie-at-matrix)]

Carlos, here are some instances in distros/ubuntu/dapper/+source/ubuntu-docs/+pots/packagingguide/ko

- C/ubuntu.xml:281(screen)
- C/ubuntu.xml:346(screen)
- C/getting_started.xml:192(screen) <-- it's just copy&paste, but <,> are there at the end (means when I pasted it, it was with &lt; and &gt;.
- C/basic.xml:109(para) <-- I was told by mdke, this case causes the xml doc gen failure. So I changed it, but you may see <,> are back.

[atie (atie-at-matrix)]

No, this is not only displaying problem, time to time I am translating the packaging guide from Rosetta, somehow the displayed <,> are stored to the translations. Which means the successful xml doc was after I corrected them by hand, for next gen, I got to correct them again unless this problem fixed.

[Carlos Perelló Marín (carlos)]

If it's not only a displaying problem there is something else to take care of, but I need a concrete message to debug the problem.

[Carlos Perelló Marín (carlos)]

Oh, you gave me the location already... sorry, I didn't see it.

I'm going to debug the problem now.

[Carlos Perelló Marín (carlos)]

Yes, bug #39879 is the cause, when you submit it twice it gets converted to '<' and '>'

The fix is now in our code repository, it should land into production next week.

[atie (atie-at-matrix)]

Good, I am just asking whether the fix contains auto-replacement of the <,> string with the &lt;,&gt; string or not - because one message string already has two translations, the translated one with &lt;,&gt is valid for my case.

[Carlos Perelló Marín (carlos)]

No, unfortunatelly we don't have an easy way to know if someone decided to put '<' or '&lt;'.

I will try to guess it, get a list of things that looks like something we should fix and ask the translation teams to do it.

[Christian Reis (kiko)]

Actually, I don't think you're right, Carlos. If the end-user typed "<" into the box, we stored "<" into the database. If he typed "&lt;" into the box, we stored "&lt;" in the database. In no cases did we misquote -- we just did no quoting at all!

The only reason this causes database corruption is related to the inclusion or omission of html elements in the translated content. In no case did we incorrectly escape.

[Carlos Perelló Marín (carlos)]

I checked this with unfixed code, and:
- I submt '&lt;', first time, it was stored as '&lt;'.
- After reloading it, the form show me '<' and the html raw source has '&lt';
- Submit that form without any change.
- After reloading the form again, it shows me '<' and the html raw source has '<';

It was not a guess, it was a real test so I'm pretty sure about what's going on. I'm not saying that we misquote it on submit, but the problem is that we were not quoting it on render so after the second submit, we store the wrong value.

[atie (atie-at-matrix)]

I believe what Carlos said above causes this problem since my translation pattern for the packaging guide :

1. Translated easy messages.
2. Went to the first page, translated/skipped some others.
3. Again went to the first page, tranlsated skipped messages.

I didn't sumbit second translated message(the mis-displayed one when I translated other messages on same web page), but Rosetta recognized second one was new by the reason Carlos mentioned above and stored the second one when I submitted the others on same page, that's my theory.

So, I hope the fix should prevent this mis-behavior with somehow smarter checking of & strings :), in addition to the fix, what I was asking is that whether replacement by fix code is feasible because, for my case, there are two translated messages - one by myself and another by the mis-behavior - available, only difference should be with & string or not.

I am OK if the fix doesn't contain my replacement request, but as you can imagine it will be a big relief at least for me.