[FFE] Add support for U1 in the music lens

[David Barth]

This bug documents a Feature Freeze Exception request for Unity, and its music lens in particular, to support searching the U1 music store catalogs.

The feature has been added as an extra "scope", which extends the search scope of the lens. The component is running outside of the main unity / compiz process, which limits its impact on the stability of the rest of the system. Further, it's running as an independent process, separate from the normal music lens itself, further limiting impact on the overall Unity infrastructure, in case of timeouts, network or server errors.

The new scope is available in https://launchpad.net/~ubuntu-desktop/+archive/ppa

There are some infrastructure changes. The banshee scope and music store scope share a common base class simple-scope, but this has been tested on the banshee scope in the desktop team ppa for about 2 weeks.

[Scott Kitterman (kitterman)]

Why is this only coming up now? If it wasn't important enough to plan for,
why is it important enough to break feature freeze for?

[Martin Pitt (pitti)]

Didier says that the ubuntu-desktop version is not the current one, he's waiting on a new release which "now does not need an environment variable".

Did this get any kind of large-scale testing yet? In terms of UI bugs, server load, results coming in with high latency, etc?

[David Barth (dbarth)]

@Scott: the feature was planned earlier, but put on hold before the feature freeze because of server availabiliy issues. By that I mean: the HW was not ready.

The feature has been re-prioritized as the server issues have been resolved and because of the business aspects of course. We reprioritized the end of the development because of the revenue implications associated with it.

From a technical point of view, I think we have made our job in finalizing the development, testing it with the staging server, ensuring error cases are handled gracefully, taking latency problems into account (async calls). That helped us solve a rather nasty bug in the process: (bug #76761)

John will comment on the test results for the server side.

[David Barth (dbarth)]

The new release is being rolled and uploaded in the test ppa right as we speak.

Tests were performed initially on the previous ppa version and then by ~5 different developers who have either been involved in the development of the extra scope or in debugging related problems (see previous comment).

[John Lenton (chipaca)]

We've deployed the production server, first with a solandra ring, and when that fell down under testing we threw it away and deployed a haproxy-fronted trio of solrs, and so far testing has indicated this will work. There still is room for improvement, and we will be doing some things to make it better, but as it stands it can handle over 3k concurrent requests without breaking a sweat, delivering 99% of results within the second. When under too much load it starts returning empty results rather than falling over completely.

We expect to improve the throughput by an order of magnitude in time for release.

[Didier Roche-Tolomelli (didrocks)]

@Martin: just got a release now.
FYI, it's in the ubuntu-desktop ppa. You just have to upgrade to it (it will normally install the new unity-scope-musicstores as a recommends of unity-lens-music), and ensure to kill the running unity-music-daemon then (or logout/login). reopening the dash should restarts it in addition to unity-musicstore-daemon.

Then:
1. go to the music lens (the result isn't shown in dash home by design)
2. make a search for a match you can find in the music store (like "crazy")
3. click on the item, it should:
3.1 open banshee (if needed)
3.2 go tho the music store view (if needed)
3.3 show the right album

[Didier Roche-Tolomelli (didrocks)]

Please note, we introduce one new string, I'll get the diff properly and ask the dx team to mail the required team and process if the FFe is approved.

[Scott Kitterman (kitterman)]

It seems quite late to be doing this to me.

[David Barth (dbarth)]

It *is* late I won't disagree with you.

I would argue that the the scope of changes is limited to a component that essentially contributes further results to a search query, so it is not an invasive change. In its absence the system will work. In its presence there can be a few more results.

Technically, it is also rather contained, in a separate process, can be uninstalled easily. It can even be removed from the release if needed by de-listing sub-package.

There is the business factor which should be taken into account. Its positive impact would seem to outweigh the risks associated with the limited scope of change in my opinion. But that is up to the release team to judge. I'm mostly the messenger ;)

And again, I think the contingency plan is rather clear and simple: deinstall if that breaks, don't ship if pre-release reports are negative.

[Martin Pitt (pitti)]

I tried this with the current PPA packages. Search results were indeed retrieved rather quickly. However, when clicking on a result, banshee opens, hangs for some 15 seconds with a blank window, and then puts you into the "music collection" folder, not into the web shop. When I then click on "ubuntu one store", I get a window "create an ubuntu one account" for about one second, then it goes away again (crash?), and nothing further happens.

This was tested in a guest session, i. e. with a clean user account.

[Martin Pitt (pitti)]

The "ubuntu one account" window crashing apparently was transient, when I tried this a second time it did not reproduce.

[Martin Pitt (pitti)]

So the remaining bug for the DX side is that Banshee doesn't switch to the store for results when you are not logged in yet. This does work if I'm already logged in, it just takes some ten seconds (without any throbber or progress bar).

After logging into U1 I get a 404 page (bug 861185).

On my own account previews didn't play, and banshee crashed after doing three or four clicks in the store. This was fixed with "rm -r .config/banshee-1/ .cache/banshee-1/", so something doesn't quite handle old data well here. But that's not a blocker for this bug or is that FFE to blame for this.

In summary, the user experience with this isn't great yet, but I guess saying "no" to this isn't an option. The search itself seems to work well enough, and if it makes the servers break we can turn it off again server-side. So please let's get this in rather sooner than later, so that it gets at least some field testing.

What does the current client code do if the server does not respond?

[Didier Roche-Tolomelli (didrocks)]

FYI, I can't confirm the doesn't switch to the store if I'm not logging in. This works there.

However, the 404 pages after logging into should be looked by the U1 team.

[Didier Roche-Tolomelli (didrocks)]

@Martin: if there is no server on the other side, there is no result and the search time-out (what the previous version was doing, didn't check with the latest code). There is nothing user visible and the local scope still feeds results.

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity-lens-music - 0.2.4-0ubuntu1

---------------
unity-lens-music (0.2.4-0ubuntu1) oneiric; urgency=low

  * New upstream release.
    - Clicking song/album in music lens loads in Banshee but does not
      focus on play queue (LP: #834933)
    - [FFE] Add support for U1 in the music lens (LP: #851691)
  * debian/control:
    - makes u-l-m now recommends u-s-m as we want it by default now.
 -- Didier Roche <email address hidden> Tue, 27 Sep 2011 18:30:39 +0200