Russian 'R' is shown incorrectly in-game

[Lex]

I noticed that the Russian capital R (which looks like latin P) is not shown correctly in-game. Instead, it's depicted as an E with grave. The word in the screenshot should say 'Рыба'. I can't really narrow it down any more, as some of the words with that letter are shown correctly.

Here's the unicode letter that I'm taking about (just in case)
UTF-8 Code point | Letter | Hex | descr
U+0420 | Р | d0 a0 | CYRILLIC CAPITAL LETTER ER

Happens running Windows 7 (64bit)

[GunChleoc (gunchleoc)]

Are you using Windows by any chance?

Do you have the same problem in message box titles?

[Lex (teh-f4ll3n)]

I am using Windows. However the building names like 'Рудник' are just fine. I'm only seeing this in help texts and scenario stories. Weird.

[GunChleoc (gunchleoc)]

this is the same bug as this then:

https://wl.widelands.org/forum/topic/1290/

https://bugs.launchpad.net/widelands/+bug/950767

[Hans Joachim Desserud (hjd)]

Reopening based on comment #14 in bug 950767.

[Hans Joachim Desserud (hjd)]

(Reporter is running Windows 7, not XP.)

[Hans Joachim Desserud (hjd)]

Fwiw, the first image posted seems to be the part of the help for Training Camp. It looks like the attached screenshot in Widelands r6884 on Ubuntu 13.10, which from what I gather seems corret. Looks like it might be a Windows-only issue.

[Launchpad Janitor (janitor)]

[Expired for widelands because there has been no activity for 60 days.]

[SirVer (sirver)]

Setting to incomplete for bug sweeping.

[GunChleoc (gunchleoc)]

We should test this again once we have a Windows build with SDL2. Maybe it went away on its own.

[SirVer (sirver)]

The latest nightlies are build with SDL2. Lex, could you retry if this is still a problem?

[GunChleoc (gunchleoc)]

The problem is still there for Scottish Gaelic "à", but the wrong character is a different one. Might this be some UTF-8/ANSI mixup with conf file gettext in Lua?

Old screenshot:

https://wl.widelands.org/forum/topic/1290/?page=2#post-9746

New screenshot attached.

[SirVer (sirver)]

possible, but unlikely. Lua does not know about encodings at all, it is just passing on the byte data. In our case, all files are encoded using utf-8 (at least that is what I'd expect and strongly hope is true) and Lua just passes it on unchanged.

A bunch of experiments that might be useful here: test strange characters in map decscriptions (in conf files) and in Lua files and see which ones are rendered correctly in which case. Maybe that gets us closer to tracking this down.

[GunChleoc (gunchleoc)]

So far, it is only certain letters in the help files. I checked for other accented vowels in the same context, and lowercase à is the only problematic letter for my language. FS_Menu, lists, tables, window titles are fine.

Messages text is broken, so is the Authors text and the story windows.

So, a richtext renderer problem?

[SirVer (sirver)]

likely. But I have no explanation where that should come from. Afaik the rendering code for all text is similar (though, of course, not everything is through the same code path), but that needs checking.

Basically, if buttons render okay, but help does not there could be bugs in the richtext renderer (not used for buttons right now) or in the way text is loaded from lua files and passed to the renderer. Both is possible, but this needs to be tracked down on windows since it is the only system that shows the problem. I would like to add verbose logging to the code paths to figure out what is wrong, but I do not know where to put it.

[GunChleoc (gunchleoc)]

Let's wait until we have the fh1 branch merged and check for this bug again then. In the long run, I'm thinking of adding a dependency to the ICU package. This would allow us to have real Unicode support and to get rid of custom made time and date functions, e.g. non-Gregorian calendars.

[GunChleoc (gunchleoc)]

The bug is still here, I have seen it both for Scorttish Gaelic and Russian.

[GunChleoc (gunchleoc)]

I have tested with the fh1 branch, the bug is not present with the new font renderer. I suspect the problem lies with how multibyte Unicode strings are segmented, since the characters can have varyig lengths. So, this can be fixed by fixing the segmentation, or by switching to the new font renderer.

[GunChleoc (gunchleoc)]

Fixed in build20-rc1