Untranslated parts of web staff client

[Eva Cerninakova]

In Evergreen web staff client (2.10.7)

1) Some language strings remain untranslated even if there shoud not be reason for it i. e.
- the translation has been included to /openils/var/data/locale/staff/cs-Cz.po (in the case of the Czech localization)
and
- the string in TT2 file seems to be "translatable".

E.g.:
- The "Cataloging" string is included in /openils/var/data/locale/staff/cs-Cz.po (line 786):
#: ../../Open-ILS/src/templates/staff/circ/share/circ_strings.tt2:30 ../../Open-ILS/src/templates/staff/navbar.tt2:178
msgid "Cataloging"
msgstr "Katalogizace"
- The string in the the /openils/var/templates/staff/navbar.tt2 (line 178) looks like this:
        <a href="javascript:;" class="dropdown-toggle"
          data-toggle="dropdown">[% l('Cataloging') %]
- But the "Cataloging" in the nav. bar is displayed untranslated.

But e.g.:
- The "Search" string is included in /openils/var/data/locale/staff/cs-Cz.po (line 3629):
#: ../../Open-ILS/src/templates/staff/navbar.tt2:185 ../../Open-ILS/src/templates/staff/navbar.tt2:59
msgid "Search the Catalog"
msgstr "Hledat v katalogu"
- The string in the the /openils/var/templates/staff/navbar.tt2 (line 33)looks like this:
    <a href="javascript:;" class="dropdown-toggle"
          data-toggle="dropdown">[% l('Search') %]
- in this case the string is TRANSLATED.

(seed attached image web_staff_translations_1.png)

2) Some portions of translations for the web staff client are untranslated probably because the translation strings are not includedin the cs-CZ.po localization file for the staff client. It concerns e.g., the labels of the form for patron edit or some particular strings, see attached image web_staff_translations2.jpg.

[Linda Jansova (skolkova-s)]

I have extracted a couple of strings from web_staff_translation2.jpg and have checked the translation files at Launchpad to see where these are included.

It seems that those strings that are included in the "webstaff" translation file are correctly translated in the interface. Example: "Telefonní číslo přes den" ("Day Phone" in English).

However, those which are not included in the "webstaff" translation file are not translated in the web client interface. Examples: "Internet Access Level" (contained in the "fm-idl.dtd" translation file) or "Privilege Expiration Date" (also contained in the "fm-idl.dtd" translation file).

Could that be the reason why some strings fail to show up as translated? And if so, how could it be fixed?

Thanks for sharing any ideas!

Linda

[Bill Erickson (berick)]

My guess is the locale is not making it all the way to the web IDL generator.

If you load the IDL with a manually applied locale, do you see the correctly translated strings in the output? (You may have to control-f to search for strings, since the output is large and dense).

For example: https://SERVER/IDL2js?locale=cs-CZ

If you see translated strings here, but not in the UI, then the locale is getting lost along the way to the IDL generator.

==

Some notes for reference:

The web IDL determines what locale to use based on either a "locale" URL parameter (as above) or from the browser's Accept-Language header. The browser client is not (currently) passing a locale URL parameter, so it would be relying solely on the header. Reviewing the code, I see the Accept-Language code is making no attempt to massage the value into something Evergreen can actually understand.

Here's a patch that attempts to address this part of the problem.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1629078-idl2js-locale-parse

We also need to support changing the locale from the browser client without changing the locale of the actual browser. For that, we probably have to pass a locale URL parameter when loading the IDL, based on the locale selected in the browser client. Before we head down that path, though, I'd like to confirm the above questions.

[Eva Cerninakova (ece)]

I tried to search the strings in IDL generated file and generally I would say that the locale is geeing lost along the way to the IDL generator. I came to following results (but can´t se any pattern in it):
- When a string is untranslated in UI, I was not able to find in in IDL generated file.
- However, it is not always true, that if strings that are included in the "webstaff" translation file, they are correctly translated in the interface. I found some strings included in "webstaff" translation file but untranslated in UI and missing in IDL generated file.
- Moreover, I found strings translated correctly in UI, but I was not able to find them in IDL generated file

Some examples:

In Patron search (https://eg-test.osvobozena-knihovna.cz/eg/staff/circ/patron/search)
- strings translated in UI: e.g., "Last name" (patron search form), present in "webstaff" template (# 632), generated by IDL
- strings u untranslated in UI: e.g..,
      "First name" (column label) - present in "webstaff" template (# 488), not generated by IDL)
      "Patron search results" - present in "webstaff template" (# 860), not generated by IDL),

In staff client homepage (https://eg-test.osvobozena-knihovna.cz/eg/staff/?set_eg_locale=cs_cz)
 - strings translated in UI: e.g.., "Search the catalog" - present in webstaff "template (# 1088), not found in IDL generated file
- string untranslated in UI: e.g.., "Cataloging" - present in "webstaff" template (# 207), not found in IDL file

[Linda Jansova (skolkova-s)]

Searching for something else, I have stumbled upon template files for OPAC and for web client, more specifically upon /templates/opac/parts/locale_picker.tt2 (for OPAC) and /templates/staff/navbar.tt2 (for web client).

The template file for the web client appears to be significantly shorter (uses just one simple IF clause, FOR instead of FOREACH, probably does not make sure locale is in the correct format etc.) than the one for the OPAC.

Could it be that this is the reason why not all translations make it to the web client interface?

[Eva Cerninakova (ece)]

I wonder if the problem can not also be related to following bugs:

https://bugs.launchpad.net/evergreen/+bug/1661327
https://bugs.launchpad.net/evergreen/+bug/1075615
https://bugs.launchpad.net/evergreen/+bug/1183386

[Eva Cerninakova (ece)]

The state haven´t change in EG 2.12 (version 0master.53150bb)

[Eva Cerninakova (ece)]

I am almost sure the problem of untranslated strings in Web Staff Client has something to do with OILSWebLocale setting in Appache.

I had noticed that the few strings displaaed translated in web staf client had been the strings included both in the "tpac" and in the "webstaff" templates. I tried to change the PerlAddVar OILSWebLocale path to PerlAddVar OILSWebLocale "/openils/var/data/locale/staff/cs-CZ.po" (instead of PerlAddVar OILSWebLocale "/openils/var/data/locale/opac/cs-CZ.po"). After restarting the Apaache all menu texts in the web staff login page have been displayed translated to Czech (however, the page has been broken and impossible to handle), see the attached picture.

[Václav Jansa (vaclav-jansa)]

We found a source of error. Data in curly brackets cannot be translated.

After correcting it in .po file the page seems to be ok

/openils/var/data/locale/staff/cs-CZ.po

#: ../../Open-ILS/src/templates/staff/navbar.tt2:447
msgid "{{username}} @ {{workstation}}"
msgstr "{{username}} @ {{workstation}}"

I have repaired it on launchpad too.

Thank you, Firefox Developer mode ;-)

[Ben Shum (bshum)]

Interesting find! For note, that translation is coming from Evergreen/Open-ILS/src/templates/staff/navbar.tt2

 [% l('{{username}} @ {{workstation}}') %]

Given that these are variables and I don't think translated workstation names are supported at the moment, we can probably remove the localization template variable here and drop the entry from the PO template and Launchpad. The same issue exists for Spanish where they translated the username / workstation too.

[Linda Jansova (skolkova-s)]

Dropping it from the PO would certainly help as it would prevent translators from making the same mistake all over again :-)!

[Eva Cerninakova (ece)]

I have removed the translation support for the username + workstation and set up the cs-cz locale for the Web Staff Client.

After some testing of the Czech locale in the Web Clinet I came to following results:

1)Every translation string currently included to "webstaff" translation template has been correctly displayed as translated to Czech. But if the translation string is included to another template (like db.seed, lang.dtd ...) the string has remain untranslated.

2) If strings from another templates are added to cs-CZ.po file (e.g., from db.seed, lang.dtd ...) in some cases it results in correctly translated expressions in web staff client interface (e.g., Locale Admin, see local-admin-transl.jpg).
However, in some cases the Czech does not apply even if the string has been added to cs-CZ.po file (e.g., column headers in Patron Search Screen or field labels in Patron Edit screen - see patron-account-transl.jpg).

3)If I'm not mistaken, there are some translation string totally missing in the templates, e.g., column header labels in the Patron Search Screen (like DoB, Mailing:Street 1 etc.)

4) In some interfaces of the web staff clients translations don´t apply at all. This is e.g., the case of the Acquisition, Administration, Authority Management...
Those interfaces have never been shown translated even in XUL Staff Client (see e.g., the bug 1075615)

[Eva Cerninakova (ece)]

This bug has relation to the bug 1709239

[Linda Jansova (skolkova-s)]

Using a sandbox localized to Czech (a big thank you for setting it up goes to Ben Shum!), I have compiled a list of correct and troublesome top menu item labels. It appears that Search (Hledat in Czech) keeps being in Czech even if other labels incorrectly switch to English.

[Linda Jansova (skolkova-s)]

Also, when looking more in depth into how strings are localized, I have discovered that there are cases with "l" missing, e.g.:

<eg-grid-field label="[% ('First Name') %]" path='first_given_name' visible sortable multisortable></eg-grid-field>

(taken from http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/templates/staff/circ/patron/t_search_results.tt2 - it is then used to generate /eg/staff/circ/patron/search)

Besides the "l" thing - is the path='first_given_name' surely correct? I mean when looking at a variety of other templates it appears that double quotes are a bit more common than single quotes used in this example and a handful of others.

On the contrary, it seems that localized strings should be enclosed in single quotes and there are cases when double quotes are used instead (http://git.evergreen-ils.org/?p=Evergreen.git&a=search&h=HEAD&st=grep&s=%5B%25+l%28%22).

[Linda Jansova (skolkova-s)]

Is it also enough to have the string which should appear translated without a path?

Example:
 <td><label for="existing_pl">[% l('Add to Selection List: ') %]</label></td>

(taken from http://git.evergreen-ils.org/?p=working/Evergreen.git;a=blob;f=Open-ILS/src/templates/acq/common/li_table.tt2 and used in /eg/staff/acq/legacy/picklist/brief_record)

This one remains untranslated in the web client, although it is most likely translated as can be seen at https://translations.launchpad.net/evergreen/master/+pots/acq/cs/+translate?batch=10&show=all&search=Add+to+selection+list. BTW it does not come from webstaff translation file but from acq.

[Linda Jansova (skolkova-s)]

Getting back to a list of top level menu items in Czech/English - using the same sandbox server, now the language of these items seems to be switched again (pages with labels previously in Czech are now in English and vice versa). So maybe some kind of a caching issue?

[Ben Shum (bshum)]

Linda, yes, any missing l() for localization for that string would likely keep it from getting caught for the template file and then subsequently being added to the PO for translation. I would suggest noting that one you found and any others you find so that we can create some patches for the web client to make those strings available for translation.

As for the menus, etc. it does appear to be an issue where Apache is only loading one of the translation POs, either the opac or staff one. And depending on which one gets loaded, we only see strings from that in either place. So this can sometimes lead to a broken OPAC or a broken web client, depending on which PO we see loaded first. Checked with berick and he isolated the problem to this line in EGWeb.pm's load_locale_handlers() function:

next if grep { $_ eq $tag } keys %registered_locales;

We found that if we comment out this line in the file, and restart apache, that the locale POs seem to load properly. Needs further testing to determine what else needs to be done here.

(note: alternatively, I also found that making a combined single PO out of opac + staff using msgcat to merge the two PO files, and defining it just once for all of /eg/ instead of separate for /eg/staff/ also worked)

Not sure why acq strings are not loading yet though. I think there's a problem with all the past dojo pages that got embedded not respecting the locale choices (or language set in browser). In XUL this was handled with some locale checks for the accept-language header? but here I'm not certain it's working. Same with Reports for me, those show in English, not Czech. Further study needed...

[Linda Jansova (skolkova-s)]

Thank you, Ben and Bill! Looks like we are on the right track :-)!

Using GIT grep I have compiled a list of strings without "l" - please see the attached file.

[Linda Jansova (skolkova-s)]

The same file in ODT - maybe it would be a better choice (both files contain the same table).

[Linda Jansova (skolkova-s)]

/eg/staff/cat/catalog/batchEdit in our sandbox also contains untranslated strings such as:

Advanced Matching Restriction (Optional)

Using GIT I have found out that this string is contained only in

[Evergreen.git] / Open-ILS / src / perlmods / lib / OpenILS / WWW / TemplateBatchBibUpdate.pm

(http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/perlmods/lib/OpenILS/WWW/TemplateBatchBibUpdate.pm)

In this module this string and the other ones from the same web page are used just like usual HTML text.

Example:

<tr>
<th colspan='3' style='padding-top: 20px; text-align: center;'>Advanced Matching Restriction (Optional)</th>
</tr>

Maybe something like

[% l('Advanced Matching Restriction (Optional)') %]

should be used instead?

[Linda Jansova (skolkova-s)]

/eg/staff/acq/legacy/picklist/user_request contains - among others - an untranslated string Article Pages which - using GIT - can be found only in PO files (and one POT file) and in Open-ILS/examples/fm_IDL.xml on line 7984:

 <field reporter:label="Article Pages" name="article_pages" reporter:datatype="text" />

Should this be sufficient for the string to make it through all the way to the web client?

[Linda Jansova (skolkova-s)]

Using the sandbox, Eva has added a bib record with holdings information and confirms that all tasks involved (such as importing a bib record using Z39.50, editing a bib record, adding a call number and creating holdings information) are properly localized, the only exception being a list of fields in Z39.50 client (which have been in English in all current and past versions of XUL client) and Cancel button which appears when editing the imported record.

[Linda Jansova (skolkova-s)]

One more thing:

One of the untranslated HTML tables in web client is the one at eg/staff/cat/catalog/verifyURLs.

Looking at http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/templates/url_verify/sessions.tt2 one can find:

<th field="id" fpath="id" formatter="module.format_id"></th>
<th field="name" fpath="name" width="auto"></th>
<th field="attempts" fpath="attempts.id" formatter="module.format_attempts" width="auto">[% l("Verification Attempts") %]</th>
<th field="creator" fpath="creator.usrname">[% l("Creator ") %]</th>
<th field="create_time" fpath="create_time" width="auto"></th>
<th field="search" fpath="search" width="auto"></th>
<th field="selectors" fpath="selectors.xpath" width="auto">[% l("URL Selectors") %]

Only Verification Attempts, Creator and URL Selectors (three of the column headings) are explicitly included as localized. Still, neither of these are translated in the interface.

Just a guess but maybe it would help if single quotes were used in these three cases and the other strings included?

[Linda Jansova (skolkova-s)]

As Eva has indicated that the Z39.50 import interface has field labels in English (and always has), I have found out that these labels probably come from Z3950.pm (http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/perlmods/lib/OpenILS/Application/Search/Z3950.pm).

Example:
title => {code => 'title', label => 'Title'},

There seems to be no indication that the string should be translated.

[Ben Shum (bshum)]

I took all the strings noticed by Linda and put them into this working branch for consideration:

user/bshum/webclient-missing-i18n-strings

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/bshum/webclient-missing-i18n-strings

We'll want to merge those changes, then run PO template sync to make them available for translation in LP for the webstaff.pot template file.

Thanks Linda!

[Linda Jansova (skolkova-s)]

I thank you, Ben!

[Eva Cerninakova (ece)]

I can see now translated those parts of Evergreen, that have never been translated yet (e.g. purchase order interface). I know it's still probably going to be a long run, but I am excited at the moment. ;-)
Thank you both.

[Galen Charlton (gmc)]

I've cherry-picked the tip of user/bshum/webclient-missing-i18n-strings into master in preparation for 3.0-beta2. Thanks, Ben!

[Galen Charlton (gmc)]

Since this bug has turned into a grab-bag of i18n issues, I have intentionally not marked it fix-committed pending a discussion about how best to wrangle this.

[Linda Jansova (skolkova-s)]

Thank you, Galen! It's exciting to see another step of making the web client properly speak the desired language :-)!

[Eva Cerninakova (ece)]

We are continually testing the issues described above, currently in Evergreen version 3.1.1. Because some issues have already been resolved (many thanks for the progress that has been made since when I opened this bug) and some have not, I have divided the unresolved ones into separated bugs:

bug 1771002
bug 1770983
bug 1770981
bug 1770979
bug 1770978
bug 1770973
bug 1770959
(other bugs are likely to follow)

[Eva Cerninakova (ece)]

Another separated bugs:
bug 1772614 (Untranslatable strings in the print template t_bills_current.tt2)
bug 1772631 (Untranslatable strings in the print template t_bills_historical.tt2)
bug 1772741 (Untranslatable strings in the print template t_patron_data.tt2)
bug 1773191 (Untranslatable Last Billing Type values)
bug 1773257 (Web staff client: Some column headers are untranslated)
bug 1773450 (Reporter legacy interface is untranslated)
bug 1773453 (Core source and source path in report template editor are untranslated)

[Eva Cerninakova (ece)]

I recommend closing this bug, because it includes various problems that are not always related. The individual problems have already been split to individual bugs -some of them fixed already ;-)

[Terran McCanna (tmccanna)]

Thanks Eva, I will mark this one Invalid since it's been split apart.