Queries to remote scopes time out

[Michi Henning]

I'm seeing this on Mako, image #63, devel-proposed.

Periodically, the music scope fails to return any results, leaving a blank screen.

Looking through the log files, I'm seeing tons of messages such as this:

[2015-01-13 01:33:27.056262] INFO: SSRegistry: SmartScopesClient.get_remote_scopes(): GET https://dash.ubuntu.com/smartscopes/v2/remote-scopes?locale=en_US
[2015-01-13 01:33:27.057239] ERROR: SSRegistry: SmartScopesClient.get_remote_scopes(): failed to read /custom/partner-id: unity::FileException: cannot open "/custom/partner-id": No such file or directory (errno = 2)
[2015-01-13 01:33:46.996259] ERROR: SSRegistry: SmartScopesClient.get_remote_scopes(): Failed to retrieve remote scopes from uri: https://dash.ubuntu.com/smartscopes/v2/remote-scopes: unity::ResourceException: Request timed out: https://dash.ubuntu.com/smartscopes/v2/remote-scopes?locale=en_US

The request times out even though the remote end works fine. (Easy to confirm by pasting the same URL into the browser.)

As an aside, the error about /custom/partner-id isn't an error: if the file doesn't exist (errno = 2), we should say nothing. That'll get rid of a lot of noise.

In addition, when I run a query, I see all queries to remote scopes fail (see attached trace).

I'm not familiar with the code. Looking through, I noticed this:

HttpResponseHandle::SPtr response = http_client_->get(remote_scopes_uri.str(), [&response_str](std::string const& replyLine) {
                response_str += replyLine; // accumulate all reply lines
        }, headers);
        response->get();

Following this through, it appears that response_str is being appended to from a different thread without any lock. That looks like it's wrong to me. But that doesn't explain the timeout exception that is thrown from response->get().

I suspect a race condition somewhere because, occasionally, I get a query that works.

[Michi Henning (michihenning)]

I just got the bug with image 161 from ubuntu-touch/ubuntu-rtm/14.09-proposed, so it's definitely not limited to devel only.

[Michi Henning (michihenning)]

RTM image smartscopesproxy log attached.

[Michi Henning (michihenning)]

I'm not sure whether this is related. But, looking at this:

void HttpClientQtThread::run()
{
    QNetworkAccessManager* manager = new QNetworkAccessManager();

    QNetworkRequest request(url_);
    for (auto const& hdr: headers_)
    {
        request.setRawHeader(QString::fromStdString(hdr.first).toUtf8(), QString::fromStdString(hdr.second).toUtf8());
    }

    QNetworkReply* reply = manager->get(request);
    reply->setReadBufferSize(0); // unlimited buffer

The reply is being used without checking that it was created succesfully. Shouldn't there be a call to error here? Something like

if (reply->error() != QNetworkReply::NoError)

[Michi Henning (michihenning)]

Looks like this is a problem QtNetworkAccessManager. Depending on timing during boot, it permanently gets stuck.

Marcus has a branch that gets rid of Qt completely and uses net-cpp instead:

https://code.launchpad.net/~marcustomlinson/unity-scopes-api/switch-to-net-cpp/+merge/246538

It looks like this will put the issue to rest.

[Pat McGowan (pat-mcgowan)]

approving while we look also for a fix to QNetworkAccessManager and the backend to network manager

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-scopes-api - 0.6.11+15.04.20150121.2-0ubuntu1

---------------
unity-scopes-api (0.6.11+15.04.20150121.2-0ubuntu1) vivid; urgency=low

  [ Ubuntu daily release ]
  * debian/libunity-scopes3.symbols: auto-update to released version

  [ thomas-voss ]
  * Switch from QNetwork to net-cpp (LP: #1326816, #1409995)

  [ Michał Sawicz ]
  * Depend on :native version of g++ to allow cross-compiling to work.
    Also run `wrap-and-sort -at` (LP: #1353855)

  [ Marcus Tomlinson ]
  * Switch from QNetwork to net-cpp (LP: #1326816, #1409995)

  [ Michi Henning ]
  * Added utility subdir to header tests.
  * Bunch of miscellaneous fixes:

  [ James Henstridge ]
  * Expose the deserialize() static method on FilterBase and FilterState
    for use by the Go bindings.
 -- Ubuntu daily release <email address hidden> Wed, 21 Jan 2015 12:46:41 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package net-cpp - 1.1.0+15.04.20150123.1~rtm-0ubuntu1

---------------
net-cpp (1.1.0+15.04.20150123.1~rtm-0ubuntu1) 14.09; urgency=low

  [ Marcus Tomlinson ]
  * Explicitly cast milliseconds::count() to long (LP: #1409995)
 -- Ubuntu daily release <email address hidden> Fri, 23 Jan 2015 14:58:31 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-scopes-api - 0.6.9+15.04.20150123.1~rtm-0ubuntu1

---------------
unity-scopes-api (0.6.9+15.04.20150123.1~rtm-0ubuntu1) 14.09; urgency=low

  [ Marcus Tomlinson ]
  * Fixed smartscopesproxy crash due to json-cpp update (LP: #1410125)
  * Switch from QNetwork to net-cpp (LP: #1409995)
 -- Ubuntu daily release <email address hidden> Fri, 23 Jan 2015 15:19:37 +0000