No easy way for users to supply large amounts of arbitrary data for apps

[Neil McPhail]

As of the current image (OTA8) there is no easy way for a user to supply a large quantity of arbitrary data (e.g. game assets or bytecode) to be consumed by an installed click package.

Example:
I have ported GemRB, which is an open source reimplementation of the Infinity Engine (used in many classic RPG games) to Ubuntu as a click package. The user has to supply the actual games assets and data to run on the engine. These assets could total 1--3GB for a typical game.

Where can the user install these files? -
1) Under $XDG_DATA_HOME/apppkgname.devname/
As $XDG_DATA_HOME currently points to ~/.local/share/, this directory is not easy to find or use with any gui tool. To access it with File Manager, you must (i) open the app, (ii) click on the burger menu, (iii) select "Unlock full access", (iv) type PIN, (v) click on display menu, (vi) tick "Show Hidden Files" (vii) navigate to the directory.

This is utterly farcical, and is blatantly user-hostile.

Furthermore, there is no way to use to GUI to get the files into this directory directly. Connecting the device to a desktop brings up nautilus in MTP mode, where this folder tree cannot be accessed and arbitrary files cannot be expected to be sent. The files would have to be installed somewhere else and cut/pasted across.

Asking a user to activate SSH is _not_ an acceptable solution to this. Neither is rsync, mv or cp from terminal. See the note above about user-hostility.

2) Under ~/Documents
This directory is visible via MTP, but still suffers from the same hindrances with regards to MTP as mentioned above. Also, the click package could not access data under ~/Documents without having a red-flagged apparmor profile, or implementing some bizarre arrangement with content-hub which would not be suitable for an action game.

(Dare I mention that content-hub remains woefully documented at this point in time, and implementing such an interface would be challenging?).

3) On the SD card
The card can be removed from the device, inserted in a card reader and used as a mass storage device. Thus, the user can drag and drop arbitrary files directly onto the card. For devices supporting SD-cards, this seems the most logical way to approach the problem.

However, it is not possible to a click app to read the SD card without asking for red-flagged read paths. It seems at _least_ "/proc/mounts" (to find the actual SD card path) and "/media/*/*/whatevernamespaceisdecided" is required. An app asking for these paths with fail click validation and will not be allowed onto the store.

---------------------------

Possible solutions:
1) Abandon MTP and enable USB mass storage. MTP is broken in so many ways it is painful
2) Provide adequate documentation for content-hub, so a numpty developer like me can work out how to use it
3) Stop using a "dot" directory for $XDG_DATA_HOME. Click apps can't access the usual user-visible folders, so please make the accessible folders less invisible.
4) Make File Manager a system app with a profile which does not require the user to jump through hoops to see files or directories
5) Give click apps access to the SD-card, and a way to discover the path, without requiring red-flagged read/write paths

--------------------------

6) As an addendum, it should be noted there is no point having a manual review process for red-flagged app store submissions if no-one is actually going to manually review them. If there was a fast, robust and elegant process where red-flagged paths could be assessed, several of the issues mentioned above would be less annoying. As things stand, it appears that no such process exists and that no manual review actually happens (for anyone outside the coredevs team).

[Jamie Strandboge (jdstrand)]

As for '3':

"Stop using a "dot" directory for $XDG_DATA_HOME. Click apps can't access the usual user-visible folders, so please make the accessible folders less invisible"

Ubuntu Core is using ~/apps/apppkgname.devname/... and Ubuntu Personal will some day too. Perhaps it makes sense to change XDG_DATA_HOME as is suggested? I imagine to ease upgrades we would need to adjust the click desktop hook to create a symlink from ~/apps/apppkgname.devname to ~/.local/share/apppkgname.devname. This would not require a change to the security policy since symlinks are resolved and wouldn't require moving user data. Perhaps MTP could be adjusted to export ~/apps as well?

Perhaps the future storage framework will handle this in another way, but the above could be helpful (perhaps even in this new storage framework world).

[Ted Gould (ted)]

Shared assets between Clicks is an interesting problem, but probably not one we're going to solve in the Click timeframe. For Snappy they are developing a content sharing interface that will allow for exactly this type of use case. In that way a game developer could get files from a game engine and allow that to update independent of their game. Because we plan to move to Snaps as a format for applications, we're not going to develop this feature for Clicks. I realize that doesn't solve your problem in the short term, but please know that we do plan on getting it fixed.

[Pat McGowan (pat-mcgowan)]

Keep open to track the requirement and keep updated as storage framework progresses in the meantime

[Michael Sheldon (michael-sheldon)]

Just a quick comment to note that we're hitting this issue with the ScummVM engine as well; as a result I'm only able to distribute this through the Open Store: https://open.uappexplorer.com/app/scummvm.michael-sheldon so that it can read from locations that users can copy files to over MTP.

Unfortunately the snappy solution proposed above won't help for ScummVM or a number of similar game engines, as whilst the engine itself is freely redistributable, the game content isn't. The user has to purchase a copy of the game from the original developer and then copy across the data files themselves.

Exposing app data directories over MTP as Jamie suggests seems like the nicest solution to me.