Support download of albums as zip files from 7Digital

[Pat McGowan]

The site offers users the option to download an entire purchased album as a single zip file. Currently there is no download target for this so the operation fails.

Need to investigate if there is any mime type to key off when the zip is offered, or some other file naming convention.

[Pat McGowan (pat-mcgowan)]

Ken maybe you can test this to see if there is something we can key off of for the mapping

[Victor Thompson (vthompson)]

Might it be best if ContentHub added an "extract" method to ContentItem [1]? The rest of the functionality to import with the Music app should be rather straight forward.

[1] http://developer.ubuntu.com/api/qml/sdk-14.10/Ubuntu.Content.ContentItem/

[Victor Tuson Palau (vtuson)]

this should be critical

[Pat McGowan (pat-mcgowan)]

This is what the header in the download looks like. We could try to use the custom header fields to id this:

HTTP/1.1 200 OK
Server: nginx/1.6.0
Date: Mon, 15 Sep 2014 18:17:24 GMT
Content-Type: application/zip
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: private
Content-Disposition: attachment; filename="The Script - No Sound Without Silence.zip"
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
x-7dig: PROD-DOWNLOAD02
x-7d-sys-hostname: PROD-DOWNLOAD02

[Ken VanDine (ken-vandine)]

We could add a special case for 7Digital in webbrowser-app, and use the custom header to map to ContentType.Music. This would hand off the zip file to music-app to extract, but in pure QML there isn't a way to extract the files. If we added an extract API in content-hub, I'd be a little worried about possible malicious content being extracted by a trusted helper. Any site could add the custom header exposing us to extracting content in an unconfined process.

I'd like to hear what someone from security thinks.

[Jamie Strandboge (jdstrand)]

I do think the proper way is to add an extract API and I also agree that we don't want to expose untrusted content extraction to unconfined processes. We are doing a pretty good job with media-hub, mediascanner, gallery, webapps, thumbnailing, etc with processes and untrusted input, so we shouldn't be lax here.

That said, this is not particularly difficult to secure-- add an extract API, have the download manager call out to a helper program that is confined. This helper program should only write out to specific folder such as /run/user/*/content-extractor so we could have rules like:
  /run/user/*/content-extractor/ rw,
  /run/user/*/content-extractor/** rw,

When the extractor is done extracting, the content-hub can move the extracted files into place. Note: '/run/user/*/content-extractor/' is just an example, point is, we don't want the extractor extracting to all over the filesystem-- have the extractor extract to somewhere safe and let content-hub do the moving around.

[Olli Ries (ories)]

raising importance to critical as per status meeting

[Sergio Schvezov (sergiusens)]

Assigning bug back to mandel, he says it's easy enough for him to fix

[Manuel de la Peña (mandel)]

The download manager by default is setting the Accept-endoding header to identity to ensure that it can check the hash of a downloaded file without the need of dealing with gzip, zip etc.. the correct solution (pointed out in the following bug https://bugreports.qt-project.org/browse/QTBUG-18239) is not to set the accepted-encoding in the request.

I'm going to solve the bug doing the following:

1. Allow the browser or download manager client to set a flag in the metadata stating that we DO want to deflate the file.
2. Do not set the accepted-encoding to identity so that QNetworkAccessManager does its work.

The download manager will be using the file name provided by the content-disposition header withou the extension when the deflate flag is passed to avoif confusions.

[Manuel de la Peña (mandel)]

Sorry for the typos, is quite late in my time zone.

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:music-app at revision None, scheduled for release in music-app, milestone 1.0

[Ken VanDine (ken-vandine)]

Based on the solution we've come up with, there shouldn't be any work to do in content-hub

[Michael Sheldon (michael-sheldon)]

There might be some slight changes needed in content-hub, as currently the download manager finished() signal (which we use to add content items) just provides a single file path. So we may have to either listen to an additional signal or receive a directory via finished() and walk through it adding the items. So for now I'll re-open the content-hub item and assign it to myself to implement when the download manager work is finished.

[Launchpad Janitor (janitor)]

This bug was fixed in the package content-hub - 0.0+14.10.20141014-0ubuntu1

---------------
content-hub (0.0+14.10.20141014-0ubuntu1) 14.09; urgency=low

  [ CI bot ]
  * Resync trunk

  [ Ubuntu daily release ]
  * New rebuild forced

  [ Michael Sheldon ]
  * Add support for receiving directories of unzipped files from
    download manager. (LP: #1365993)
 -- Ubuntu daily release <email address hidden> Tue, 14 Oct 2014 16:11:13 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package webbrowser-app - 0.23+14.10.20141015.1-0ubuntu1

---------------
webbrowser-app (0.23+14.10.20141015.1-0ubuntu1) 14.09; urgency=low

  [ Michael Sheldon ]
  * Detect 7 digital album downloads and request that they get unzipped
    by download manager. (LP: #1365993)

  [ Ubuntu daily release ]
  * New rebuild forced
 -- Ubuntu daily release <email address hidden> Wed, 15 Oct 2014 11:56:39 +0000