improve sentence structure

[matthiaskrgr]

Moving the cursor over a tavern, a label "skipped Produce ration because not: economy need Ration" popped up.
Is it possible to improve the sentence structure somehow?
A screenshot will be attached.

rev. 4987

Thank you for your fine work!
Matthias Krüger

[matthiaskrgr (matthiaskrgr)]

The file sentence.jpg was added: None

[Sigra (sigra)]

Not that I know.

[Nasenbaer (nasenbaer)]

The sentences should definitely be improved! The way it is now, makes it hard and even nearly impossible to write a understandable translation for some languages (e.g. German)
Better write these sentences with "... .. %s ... .. ", ware

[LennStar (lennstar)]

That german really is awful.

not producing because: economy does not need X

That should be (at least understandable) translateable in any language.
"because not" is always the wrong way

[SirVer (sirver)]

The problem is that the sentence is currently constructed from some boolean construction. It is easy to make it understandable, but it is hard to chain ands and ors correctly so that it can be translated in any language. Someone should come up with a clever idea here.

[Tino (tino79)]

Perhaps the following solution would fit:

1.) Production stopped
2.a) Production of XXX skipped. Reason: No need for XXX.
2.b) Production of XXX skipped. Reason: No YYY available.
3.) Producing XXX.

Did i miss one case?

Tino

2010/5/3 SirVer <email address hidden>:
> The problem is that the sentence is currently constructed from some
> boolean construction. It is easy to make it understandable, but it is
> hard to chain ands and ors correctly so that it can be translated in any
> language. Someone should come up with a clever idea here.
>
> --
> improve sentence structure
> https://bugs.launchpad.net/bugs/536360
> You received this bug notification because you are subscribed to
> widelands.
>

[Kiscsirke (csirkeee)]

Yes:

Production of XXX skipped. Reason: Economy doesn't need XXX, but needs YYY.

András

On 4 May 2010 07:03, Tino <email address hidden> wrote:
> Perhaps the following solution would fit:
>
> 1.) Production stopped
> 2.a) Production of XXX skipped. Reason: No need for XXX.
> 2.b) Production of XXX skipped. Reason: No YYY available.
> 3.) Producing XXX.
>
> Did i miss one case?
>
> Tino
>
> 2010/5/3 SirVer <email address hidden>:
>> The problem is that the sentence is currently constructed from some
>> boolean construction. It is easy to make it understandable, but it is
>> hard to chain ands and ors correctly so that it can be translated in any
>> language. Someone should come up with a clever idea here.
>>
>> --
>> improve sentence structure
>> https://bugs.launchpad.net/bugs/536360
>> You received this bug notification because you are subscribed to
>> widelands.
>>
>
> --
> improve sentence structure
> https://bugs.launchpad.net/bugs/536360
> You received this bug notification because you are subscribed to
> widelands.
>
> Status in Widelands: Confirmed
>
> Bug description:
> Moving the cursor over a tavern, a label "skipped Produce ration because not: economy need Ration" popped up.
> Is it possible to improve the sentence structure somehow?
> A screenshot will be attached.
>
> rev. 4987
>
> Thank you for your fine work!
> Matthias Krüger
>
>
>

[Nicolai Hähnle (nha)]

On Tuesday 04 May 2010 08:26:10 Kiscsirke wrote:
> Yes:
>
> Production of XXX skipped. Reason: Economy doesn't need XXX, but needs
> YYY.

There's also the case where workers needing experience are involved
(Microbrewery / Brewery)

[Astuur (wolfsteinmetz)]

Guess it would be fine to have some translators here to check whether such a construction is translatable in their respective language.

Production of XXX skipped because the Economy doesn't need XXX but YYY instead.
Die Produktion von XXX wurde überspungen, denn die Wirtschaft braucht kein(e?) XXX sondern YYY.
Grammatical gender may make problems.

[SirVer (sirver)]

I took the freedom to vote this down to medium. While I can see that it affects many people, it is not a game changing issue as the sentence is understandable at least.

[Joachim Breitner (nomeata)]

The gramps project had similar issues when automatically calculating relationship descriptions (great-grand-grandson etc.). Their lession was that you cannot handle everything with gettext and created one class per supported language. This gave them the freedom to really create good-sounding sentences.

The current state is IMHO a ugly spot in the game, which otherwise really benefits from its nice and good-looking UI. If the WiHack would just be starting, I could imagine working on this. Maybe at the next one :-)

[Slondruwir (slondruwir)]

I´m from Germany.

The translation is:

1.) Production stopped
2.a) Production of XXX skipped. Reason: No need for XXX.
2.b) Production of XXX skipped. Reason: No YYY available.
3.) Producing XXX.
4. ) Production of XXX skipped. Reason: Economy doesn't need XXX, but needs YYY.

1. ) Produktion gestoppt.
2.a) Produktion von XXX übersprungen. Grund: XXX wird nicht gebraucht.
2.b) Produktion von XXX übersprungen. Grund: YYY ist nicht verfügbar.
3. ) XXX wird produziert.
4. ) Produktion von XXX übersprungen. Grund: Wirtschaft braucht XXX nicht, sondern YYY.

That would sound quite well. ;-)

[Foppe Benedictus (foppe-benedictus)]

I think it would be logical to separate effect and cause (as I state in the duplicate Bug #680960 ). In that case you get still 5 translatable strings (+ those for the inexperienced workers), but at least you know for sure that "Production of XXX skipped." is translated the same way.

[Leif Sandstede (lcsand)]

Where would one find the code that generates these messages?

[Nicolai Hähnle (nha)]

Follow the act() code of the ProductionSite class (i.e. productionsite.cc and friends). This is where production sites check whether preconditions for production are satisfied, and react accordingly.

[Leif Sandstede (lcsand)]

I looked at what would be needed to fix this.
The code generating this message is in "src/logic/production_program.cc" in function
"void ProductionProgram::ActReturn::execute(Game & game, ProductionSite & ps) const".
Problem is that the fuction uses premade textblocks generated by other funtions wich are tied to actual game logic.
This message could be fixed by analyzing what message it creates right now and then giving out a proper sentence from that, but that would be a pretty inelegant solution. And you would have to know all the possible cases it could generate. Also it would not do anything to improve any other messages.
So to fix this properly larger parts of production_program.cc and productionsite.cc would have to be rewriten, maybe even more.
I propably wont fix this, but here is some more info if anyone else wants to try:
Textblocks:
ps.top_state().program->descname(); //Produce ration
m_conditions //economy need Ration

And whatever text gets written into ps.m_result_buffer is the hover text.

[Jason Stumpf (jason-stumpf)]

Since there are already images of each of the wares, why not use them instead of text that needs translating. Something like:

not producing (picture of thing)
reason:
economy needs: (pictures of things)
economy doesn't need: (pictures of things)
economy doesn't have: (pictures of things)

[Hans Joachim Desserud (hjd)]

Hi Jason and welcome to Launchpad!

While I initially like your suggestion, I'm unsure how well it would work. How easy will it be to distinguish the different wares in the text, especially at higher resolutions. Then again, I haven't checked so I'm not sure if this is an issue or not. Could someone make a mock-up of this?

I also wonder if a combination of text and picture, like "failed to produce warename (pic)" might be an idea. I don't know if it's just a myth, but I seem to remember combination of image+text in UIs is easier to understand than either one alone.

[Gabriel Margiani (gamag)]

I did some painting ...

[SirVer (sirver)]

I like the One wirh the images. Good suggestions

[SirVer (sirver)]

This is a bigger change - won't be for b17.

[Joachim Breitner (nomeata)]

Looking some more into this issue...

I would find it very slick if widelands would produce correct and smooth sentences. The proper solution would be to represent the sentence semantically, e.g. as a tree of with elements such as verbs, subjects, subordinate clauses. This tree would still be language independent, and then code for each language would, following the particular rules of the language, construct the proper sentence. For this to work the names of programs, for example, should not just be a hard-coded string "Produce ration", but rather the information "verb:produce" and "object:ration", and either all needed grammatical forms of these words need to be specified in the conf file, or the code that generates the sentences need a database of words and how they behave gramatially (e.g. that the plural of sheep is sheep).

Of course that is a lot of language specific code that does not really belong into widelands. The problem at hand is called “Natural Language Generation” (http://en.wikipedia.org/wiki/Natural_language_generation). See http://www.fb10.uni-bremen.de/anglistik/langpro/kpml/pix/gratuitous-ml2-big.jpg for an example.There is a list of such systems onhttp://www.nlg-wiki.org/systems/Table_of_NLG_systems, but I doubt that we find one that is Free Software, has C++ bindings and supports enough languages.

Another approach would be the following. Instead of generating a tree that supports arbitrary sentences, maybe we can get the result with a simpler tree tailord precisely for our purpose, here described as a grammar:

sentence := result "because" list_of_reasons "."
result := program_failed | program_completed | program_skipped
list_of_reasons := reason | more_reasons "and" reason | more_reasons "or " reason
more_reasons := reason ", " reason
reason := "the economy needs " something | "the economy does not need " something | ...

For the translation, strings with placeholders are used, e.g. not just _"because", but _"%s because %s." to give translators more flexibility. And also to increase the chance that they can write whatever their language requires, the various variatns of result are not constructed in the code (as that would require taking a sentence apart etc.), but rather the program description in the conf files contains all variants, e.g. _"Production of rations failed", _"Sucessfully produced rations", _"Skipping production of rations" (note that this allows for variations).

Maybe even the _"the economy needs %s" should be part of the description of a ware, to allow for variation (e.g. with countable versus non-countable wares: "the economy does not need any tongs" vs. "the economy does not need water"). And just because it might be possible to build a correct sentence with _"the economy needs %s" in English and German does not mean that this is sufficient for all other languges .

The tree does not necessarily have to be represented as such in the code; the mixing of logic and sentence building could be left like it is now.

This does put a bit more load on the translators and increases the number of strings in the configuration. But it is very flexible without requiring actual natural languge generating logi...

[Raul Ferriz (raul.ferriz)]

El 27/10/12 13:07, Joachim Breitner escribió:
> Looking some more into this issue...
>
> I would find it very slick if widelands would produce correct and smooth
> sentences. The proper solution would be to represent the sentence
> semantically, e.g. as a tree of with elements such as verbs, subjects,
> subordinate clauses. This tree would still be language independent, and
> then code for each language would, following the particular rules of the
> language, construct the proper sentence. For this to work the names of
> programs, for example, should not just be a hard-coded string "Produce
> ration", but rather the information "verb:produce" and "object:ration",
> and either all needed grammatical forms of these words need to be
> specified in the conf file, or the code that generates the sentences
> need a database of words and how they behave gramatially (e.g. that the
> plural of sheep is sheep).
>
> Of course that is a lot of language specific code that does not really
> belong into widelands. The problem at hand is called “Natural Language
> Generation” (http://en.wikipedia.org/wiki/Natural_language_generation).
> See http://www.fb10.uni-bremen.de/anglistik/langpro/kpml/pix/gratuitous-
> ml2-big.jpg for an example.There is a list of such systems onhttp://www
> .nlg-wiki.org/systems/Table_of_NLG_systems, but I doubt that we find one
> that is Free Software, has C++ bindings and supports enough languages.
>
>
> Another approach would be the following. Instead of generating a tree that supports arbitrary sentences, maybe we can get the result with a simpler tree tailord precisely for our purpose, here described as a grammar:
>
> sentence := result "because" list_of_reasons "."
> result := program_failed | program_completed | program_skipped
> list_of_reasons := reason | more_reasons "and" reason | more_reasons "or " reason
> more_reasons := reason ", " reason
> reason := "the economy needs " something | "the economy does not need " something | ...
>
> For the translation, strings with placeholders are used, e.g. not just
> _"because", but _"%s because %s." to give translators more flexibility.
> And also to increase the chance that they can write whatever their
> language requires, the various variatns of result are not constructed in
> the code (as that would require taking a sentence apart etc.), but
> rather the program description in the conf files contains all variants,
> e.g. _"Production of rations failed", _"Sucessfully produced rations",
> _"Skipping production of rations" (note that this allows for
> variations).
>
> Maybe even the _"the economy needs %s" should be part of the description
> of a ware, to allow for variation (e.g. with countable versus non-
> countable wares: "the economy does not need any tongs" vs. "the economy
> does not need water"). And just because it might be possible to build a
> correct sentence with _"the economy needs %s" in English and German does
> not mean that this is sufficient for all other languges .
>
> The tree does not necessarily have to be represented as such in the
> code; the mixing of logic and sentence building could be left like it is
> now.
>
> This does put a bit more load on the tr...

[Joachim Breitner (nomeata)]

Hi,

Am Samstag, den 27.10.2012, 11:28 +0000 schrieb Raul Ferriz:
> As summary, keep full sentences together instead of splitting them into
> something like:
> cYouNeed = "You need";
> cUnitsOf = "units of";
> cToBuildA = "to build a";
>
> Currently I did not use C++, so I didn't remember exactly what the
> syntax was to build that parametric string, but I think that you get the
> idea. It's better to have full sentence so translator have a chance to
> get better translation.

That is correct, but it seems that for this use case, the form of the
sentences varies too much (e.g. the number of failing conditions is not
known). Hence my suggestion to have the translator give complete of the
_subsentences_, and a generic way of combining them.

Greetings,
Joachim
--
Joachim Breitner
  e-Mail: <email address hidden>
  Homepage: http://www.joachim-breitner.de
  Jabber-ID: <email address hidden>

[Joachim Breitner (nomeata)]

I started to work on this, see the linked branch at https://code.launchpad.net/~nomeata/widelands/sentence-structure. Review welcome.

I started with a commit that does not require changes to the tribe configuration, but also still gives not gramatically correct sentences, e.g. „Skipped Produce felling axe because Felling Axe is not needed.“

It would be great if there is a way to test the code without trying to find a game situation where it is used in its varying results. Can this be done in a separate binary, similar to the testsuite?

[Joachim Breitner (nomeata)]

And for the translators reading there, here are the newly introduced text components that you can try to translate into your language to still make sense:
"the economy needs %s"
"%s is not needed" (Yay, more varyation in the text :-))
"site has "
"site does not have " (Could be re-done with %s)
 "workers need experience"
"workers do not need experience"
"%s and %s" (the first %s contains a comma-separated list of explanations like „%s is not needed“)
"Failed %s because %s."
"Completed %s because %s."
"Skipped %s because %s." (The second %s is an explanataion or a list of explanations with "and", the first the name of the program)

If the names of the programs are only used by this code, they could be made to fit the structure, e.g. "Production of felling axe“ instead of „Produce felling axe“; that would go quite well with „%s failed because %s“, „%s skipped because %s“, ... and other language are likely to find similar working combinations, z.B. „Die Produktion von Äxten“ mit „%s wurde eingestellt, da %s.“, „%s wurde übersprungen, weil %s.“ und „%s war erfolgreich, da %s“.

[SirVer (sirver)]

Great that youre working on this joachim!

A few comments:

1) I once thought about moving productionsite programs to Lua. I still feel this is a good idea and this would handle this case quite elegantly: each house could have their own error messages defined in their Lua file. Common sentences could be factored into tribe lua files which we already have. I see that this is a major undertaking and a 80% solution would be nice to have earlier (or ever :) )
2) Use boost::Format which allows numbered placeholders [1]. This makes translation possible. We use it around the code base a lot and it is really easy to use and performs okay. A plus could be that we only need to translate each sentence once in the codebase - the many calls to gettext are quite expensive currently.

Code review:
L 150: What is a positive negation?
L 174: Having all of this in one string would make it easier for translations.

[1] http://www.boost.org/doc/libs/1_51_0/libs/format/doc/format.html

[Joachim Breitner (nomeata)]

Hi Holger,

Am Mittwoch, den 31.10.2012, 20:09 +0000 schrieb SirVer:
> 2) Use boost::Format which allows numbered placeholders [1]. This
> makes translation possible. We use it around the code base a lot and
> it is really easy to use and performs okay.

True, but it technically it is not required to have them in the code: If
the translator needs to switch around the positions, he can just use
them in his translated format string. But it is of course easier and
more obvious to him if the numbers are already in the original string,
so I’m changing it (rev 6436).

> A plus could be that we only need to translate each sentence once in
> the codebase - the many calls to gettext are quite expensive
> currently.

I don’t follow here: What sentence needs to be translated several times?
And are you talking about the runtime cost or developer time cost?

> Code review:
> L 150: What is a positive negation?

I improved the documentation of ::description in revision 6437:

/// A description of this condition (if the second
/// argument is true), or a description of the negation
/// of the condition (if the second argument is false).
virtual std::string description(Tribe_Descr const &, bool) const

The idea is that there is no general way of (linguistically) negating a
condition; only the condition itself will know how its negation should
be expressed. Hence the parameter tells it what description ought to be
used. This is also used to get rid of the "because not:" construct.

I could have introduced two separate methods as well...

> L 174: Having all of this in one string would make it easier for translations.

I see _("the economy needs %s") and _("%s is not needed") there – how
should that be one string – by the previous argument the translator
needs to be able to separate the condition and its negation separately?

Or do you mean the string should be something like _("the economy needs
%s|%s is not needed") so that the two strings stay together in the .po
file, and the code should take them apart?

Greetings,
Joachim

--
Joachim Breitner
  e-Mail: <email address hidden>
  Homepage: http://www.joachim-breitner.de
  Jabber-ID: <email address hidden>

[Joachim Breitner (nomeata)]

> If the names of the programs are only used by this code, they could be made to fit the structure [...]

These are all locations where the names of programs are used:
src/logic/trainingsite.cc:171
src/logic/production_program.cc:822
src/logic/production_program.cc:456

The latter two are combined with something like "skipped", so for them it would make sense to change the descriptoin to “Production of rations”. The former code looks like this:

std::string TrainingSite::get_statistics_string()
{
 if (State * const state = get_state())
  return state->program->descname();
 else if (m_result == Completed)
  return _("Resting");
 else
  return _("Not Working");
}

so here the grammatical form should be something like “Producing rations”.

Currently it is something like “Upgrade soldier attack 0” which neither goes well with “skipped” appended or prepended, nor is it compatible with the grammatical form “Resting” and “Not working”.

I think a viable solution is to change all program description to the form “Producing ...” or “Upgrading soldier attack 0” and then combine it with "%s in progress" for TrainingSite::get_statistics_string and with "%s skipped/stopped/completed because %s" in production_program. I hope that all languages will find a grammatical form for the description that can be extended to all four use cases.

Actually, for english, it would be sufficient to use "%." or "%s..." in TrainingSite::get_statistics_string, but it is likely that other languge will have to put something else there. But they can, so all is well.

[Joachim Breitner (nomeata)]

On a second thought, this approach may not be flexible enough; already in German it fails. If the program description becomes „Die Produktion von Nahrung“, which goes well with "%s wurde übersprungen, da %s", "%s wurde eingestellt, da %s" and "%s war erfolgreich, da %s", I don’t see a good way to make a statement that fits the use in TrainingSite::get_statistics_string. Maybe "%s läuft.", which is still an improvement over the status quo.

The question is: Is this better than requring two separate descriptions in the tribe’s conf files (or even a separate text for all four variants)? That would be most flexible and allow for nice variations, but puts more burdeon on the tribe settings creators and the translators.

[SirVer (sirver)]

I'd go with as many descriptions as necessary to make it sound good. It is a burden for translators, but they will eventually catch up.

[Joachim Breitner (nomeata)]

Going for the maximal solution, that would mean four strings /per program/: One for the indicative („Producing rations“), and the three for stopped, skipped and succeed... It would be nice and allow for good-sounding, non-repetitive descriptions, but is a lot of text to be written and translated – just checking if this is really what you want :-)

[Joachim Breitner (nomeata)]

After learning the code some better I noticed that not all combinations of program and result can occurr. In fact, it seems that it would be most simple (and similar to what the lua solution would entail for the translators) to just include the whole sentence in the "return" command of the programs:
$ grep -r return= tribes/|wc -l
175
Not a small number, but also doable, and this would free us completely from any grammatical guessing in the code.

[SirVer (sirver)]

Lets do that then. We will never get it right programmatically.

[Joachim Breitner (nomeata)]

Ok, for the "skipped", "failed" etc. commands, this is implemented now (and a lot of code could be removed). The message is given in the program before the return by
return_message=_Could not jump the hoop because gravity was missing.
and hence benefits from the usual translation infrastructure.

Of course someone still needs to write these 175 descriptions.

The ProductionProgram::ActConsume::execute command still constructs a sentence, but here it is more likely to get a good result, because the requirement is always just a list of wares.

[Joachim Breitner (nomeata)]

Hmm, it seems that generally the logic is not very well suited for complex situations, such as metalworks. It will tell me that production is skipped because there are not felling axes needed, although the real reason is that the economy wants shovels, no trunks are available, and the consumes command fails.

[Joachim Breitner (nomeata)]

Sigh. Looking closer at ProductionProgram::ActConsume::execute:

There are many consume= commands (200). Add that to the 175 descriptions for the returns= commands, do we really expect the conf editors and translators to write the corresponding failing messages, which would be quite repetetive, out?

On the other hand, it seems that a logic that works for ActConsume::execute would also be sufficient for the ReturnCommand’s message.

I wonder what the translators think – maybe someone could notify them about this discussion on the forum, or wherever they hang out.

[Joachim Breitner (nomeata)]

I’m still not sure what the best way is, but either way is definitely an improvement over the current situation, so I consider the branch ready for merging. The latest code in my branch now supports both: for "return" commands that have a "return_message" set that one is used, for the others, the message is constructed and closer to good english that what we have right now.

I’m in favor of changing the program descriptions from "Produce foo" to "Producing foo", so that we can write "Producing foo failed", "Producing foo skipped" etc. But before changing the description of 298 programs, this needs consensus.

[SirVer (sirver)]

+1 from my site :-). I'll wait for more opinions before merging.

[Flávio J. Saraiva (flaviojs2005)]

Hi there, I started translating to Portuguese recently. I haven't translated for long, but I can already see the big amount of works it takes.

However, it takes much more work to work around hard-coded constructs. I would rather have a lot more lines to translate than that. =~~

Sentences that have multiple arguments can become awkward when translated if we have to maintain the argument order. I'd like to be able to control where they are placed, so using boost::format or similar in those sentences would be very welcome. =)

[SirVer (sirver)]

flavio, I think your bug report has nothing to do with this particular bug report (not sure though). However, Gun is our translation hero.

[GunChleoc (gunchleoc)]

I have already done simething with these in the i18n branch, but I can have look at what everybody wrote here to make sure I haven't missed anything.

[GunChleoc (gunchleoc)]

I can't find the commit now, but this has been fixed.

[GunChleoc (gunchleoc)]

Fixed in build19-rc1.