network disk cache does not take into account the HTTP Vary header

[Albert Astals Cid]

Switching from English to Polish works as expected
But then going back to English leaves me with store in Polish

This survives reboots.

Steps to reproduce:
1. Go to Ubuntu Store scope (with your current language).
2. Go to phone settings, change the language, accept & reboot the phone.
3. Go to Ubuntu Store scope. Note how scope title and some strings such as "installed" changed for the new language (since these are provided locally by the scope for current LANG), but strings such as 'Highlight of the week' or departments names (hit the search lens icon to reveal departments) which come from the store still use the old language.

See upstream bug at https://bugreports.qt.io/browse/QTBUG-14750

[Paweł Stołowski (stolowski)]

It seems this is related to network caching. Removing all cache files from /home/phablet/.cache/unity-scope-click/network (wait till scope is restarted) fixes it (correct language is used).

Server side of the store was just recently updated with more relaxed expiration times, so now cache is actually used in more cases than before. The "Vary" header sent by the server seems to be correct (contains Accept-Language, among others), so I'm wondering if Qt handles Vary header correctly.

[Paweł Stołowski (stolowski)]

Ok, I think this might be problem with Qt, found an old bug and commented on it: https://bugreports.qt.io/browse/QTBUG-14750

[James Henstridge (jamesh)]

Looking inside the files under ~/.cache/unity-scope-click/network, the data files seem to contain the URL, HTTP response headers and body. I don't see anything resembling HTTP request headers, so it isn't clear how the Qt code could even implement Vary.

And in fact, this seems to be an open unresolved bug, filed in 2010:

https://bugreports.qt.io/browse/QTBUG-14750

[Paweł Stołowski (stolowski)]

Yeah, I checked Qt code as per my comment to https://bugreports.qt.io/browse/QTBUG-14750

I think the only reasonable way we can work around this until Qt is fixed is to save current language in a text file and forcefully clear the cache on scope start if we see language has changed.

[Albert Astals Cid (aacid)]

Since you've identified the problem in Qt, would fixing the problem there be much harder than workarounding it on our side?

[Paweł Stołowski (stolowski)]

Yes. According to James' finding, headers of HTTP requests are not currently stored in the cache. Fixing that may involve additions or changes to QAbstractNetworkCache interface. The workaround in scope will be a few lines of code. That being said, the bug needs to be fixed in Qt (I've emailed lpotter today) as it may manifest itself in other areas (Ubuntu serves use Vary header a lot with multiple values, not just language).

[Paweł Stołowski (stolowski)]

Just to clairfy, I think we should implement that quick workaround in scope now and at the same time look for a proper fix in Qt.

[Lorn Potter (lorn-potter)]

I briefly looked into fixing QNetworkRequest/QNetworkAccessManager (I presume that is what you are using), it looks fairly easy three lines at most (famous last words) to fix.

[Lorn Potter (lorn-potter)]

What are the steps to reproduce this?
I tried several times switching system language English->Spanish (reboot) and then back to English (reboot) and store scope did the correct thing.

[Paweł Stołowski (stolowski)]

Hi Lorn, added to the bug description.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-scope-click - 0.1.1+16.10.20160525-0ubuntu1

---------------
unity-scope-click (0.1.1+16.10.20160525-0ubuntu1) yakkety; urgency=medium

  * Add conditional debug for network timings. Force cached data if
    network is 'disconnected' (as per connectivity status reported by
    scopes API).
  * Build-depend on ubuntu-sdk-libs to avoid dependency problem on
    arm64.
  * Clear network cache if language was changed (LP: #1575744)
  * Use text widget to display error strings. (LP: #1542854)

 -- Pawel Stolowski <email address hidden> Wed, 25 May 2016 16:20:15 +0000

[Pat McGowan (pat-mcgowan)]

mark fixed with workaround but leave the real qt fix task open