Unicode in translated dates being lost in app previews

[Richard Somlói]

How to reproduce:
 * set the system language to hungarian
 * go to the installed apps scope
 * search an app with update from this month (for example Music)

what happens?
 * It shows "Utolsó Frissítés (Last Update): 2015 jl. 29"

what was expected?
 * Utolsó Frissítés: 2015 júl. 29

So it doesn't show the "speical" characters like: á, é, ó ... etc.
The following month strings are affected as well: ápr (april), már (march), máj (may) jún (june).

I attached a screenshot too.

[dobey (dobey)]

I don't see Hungarian language on the phone. I've recreated this issue with Greek though.

[dobey (dobey)]

OK. I'm moving this to unity8, as it seems we are sending the correct UTF-8 string data in the JSON, so it's getting through as far as the scope can push it. It seems to be an issue with the table preview widget in the dash itself.

[Lukáš Tinkl (lukas-kde)]

Are you able to reproduce this with other such month names?

[Richard Somlói (ricsipontaz)]

It is weird. It is showing the right string now with the Music app. Did something change? (You can find the Hungarian language with the name "magyar“.

[Albert Astals Cid (aacid)]

Rodney, the JSON data we get varies from

{"title":"Frissítések","values":[["Verziószám","2.2.892"],["Utolsó frissítés","2015 jl. 29"],["Első kiadás","2013 okt. 15"],["Méret","322,6 KiB"]]}

to

{"title":"Frissítések","values":[["Verziószám","2.2.892"],["Utolsó frissítés","2015 júl. 29"],["Első kiadás","2013 okt. 15"],["Méret","322,6 KiB"]]}

randomly, so sometimes it shows up correctly and sometimes not.

Can you tell me how to add debug in the lower layers to verify it's QML breaking it?

[dobey (dobey)]

@Albert, weird. Maybe it could be an intermittent issue in boost::locale or icu then? Or maybe in scopes-api/scopes-shell when handling the JSON?

I recreated this issue with Greek on my N4 yesterday, but I just checked again (with no changes at all on the phone), and it seems to be showing the correct values now.

I started trying to debug this in the click scope yesterday by writing some tests to verify that the values were getting shoved into the attributes correctly for the table, in the scope, and printing the JSON in that test. It was working consistently with that, so I concluded that the bug must be higher up the stack, and moved it over to unity8, as it seemed to only be an issue in the second column of the table.

I'm not quite sure how to debug this further to isolate the issue.

[Albert Astals Cid (aacid)]

I spent some time on this on yesterday and i can confirm that by having this code in various places of unity-scope-click

click::Date d1;
d1.timestamp = 1437758185; // Yes i made the timestamp member public for that to work
qWarning() << "CCC" << d1.formatted().c_str();

And using hungarian as a language (i.e. Magyar) sometimes the output of the warning is "júl" (with utf8 encoding) and sometimes is just "jl" so it seems like for some reason

    std::stringstream s;
    s << boost::locale::as::date << timestamp;

is formatting things different from one time to another.

From my testing it seems that just after boot it would be "júl" and then after uninstalling something it would be "jl", but need someone with more knowledge of boost, click scope and scopes in general to try to figure out why this formatting change is happening.