App data and config files are not removed when an app is uninstalled

[Michał Karnicki]

If an app uses ~/.config/foo (say, uses Qt.labs.settings), and the user uninstalls the app, ~/.config/foo is not removed. It should be deleted when the app is uninstalled.

Using latest utopic on the phone.

---------------------
Desired resolution:

- When a user deletes an app, 100% of files associated with the app should be deleted.

- A warning notification should be displayed when a user goes to delete an app informing them that this action will also delete any data they have stored inside the app. This notification should give the options of cancelling or proceeding with the action.

- Users should not be allowed to delete the key apps that ship with the phone. These key apps are defined as: Dialler, Address book, Messaging, Browser, Camera, Music player

The above will give the user a simple easy to understand mental model while simultaneously protecting the user from deleting important content within the default apps.

[Colin Watson (cjwatson)]

Don't expect this to be addressed before RTM, since it involves click gaining a lot of new knowledge about how app data is laid out that is currently entirely out of its scope.

[Victor Tuson Palau (vtuson)]

Hi Colin,

To me this seems like a simple check on .config folder to see if there is any folder named with the package name and delete it. I am not sure it needs to be much more clever than that.

I also think that this is a bit of an issue, if the user expects their data to have been deleted from the phone but that is not the case..

[Colin Watson (cjwatson)]

We would also need to consider other directories that might be involved, such as things under ~/.cache/; and we might want to think about whether the user ought to be offered a choice in some of those cases.

[Michael Vogt (mvo)]

I agree with Colin here, we need to get find all directories and consider if removing the data is something we should do or offer a choice to the user. If its important to remove the config, the attached (untested) patch should work , I will try to get it into better shape soon (proper test, cachedir).

[Jamie Strandboge (jdstrand)]

The data directories can be found here: http://developer.ubuntu.com/publish/apps/security-policy-for-click-packages/ (see 'Runtime Environment' at the bottom of the page). That page does not cover scopes though-- probably best to review the apparmor policy in /usr/share/apparmor/easyprof/{templates,policy_groups}/*/* for writable areas.

Note, I acknowledge the bug but I think the counter argument is that users would be surprised to see their data deleted which is why we have the current functionality. Consider any application which contains 'precious' data that cannot be easily restored, such as pictures, recorded audio, recorded video, documents, etc, etc. If we were to do this properly, I think it should be configurable. One idea is to have a check box to delete associated user data that the app created.

There are other questions too: what happens with trust-store permissions, with online account information, and online account cached accesses? There is probably more to consider.

Finally, deleting user data is fraught with danger. The attached patch does drop privileges, but I think we need to be very careful that the directory to be deleted is actually the one that the package created/is using (ie, we need to guarantee that Environment.get_user_config_dir() is the same as what UAL/aa-exec-click is giving to the app at runtime). We need to be sure not to follow symlinks on the toplevel directory being deleted and for files inside the to be deleted directory, not to follow symlinks outside of the app's writable areas. We have application isolation in place to protect the user and the system from malicious applications, but the proposed click removal process is unconfined-- it would be a shame if a malicious app author decided to delete the user's files as part of its removal process.

In addition to defensive coding, we could in theory have a click user data remover helper which click would run under the apparmor profile of the app. This gets fairly complicated with click packages that ship multiple applications and hooks.

[Seth Arnold (seth-arnold)]

I strongly dislike the idea of click deleting user data when applications are uninstalled. The number of ways that this can go wrong is staggering. Different users on a device may wish to make different choices when an application is uninstalled.

We do need some way to clean up applications but it should be handled through some other user-driven mechanism; I suggest an interface in the system configuration dialog that allows users to see how much {cache, configuration, private data, trust-store entries} each installed application is using and allow them to delete the data as they wish.

Thanks

[Michael Vogt (mvo)]

I attached a branch *if* we want to to this. However I share the concerns of Jamie and Seth that this needs some more spec work. Especially from the design team if its desirable to remove the data on uninstall. I like the idea of having a UI for this. But we could remove the cache data.

[Victor Tuson Palau (vtuson)]

This is clearly a UX decision , so we have added Ubuntu-UX and I will ping John Lea on this.

[Michał Karnicki (karni)]

Hi all. Thank you for your constructive feedback on the bug. Regarding what mvo said
("But we could remove the cache data.") -- that is closer to the nature of this bug. We recognize there may be user data involved (that is, XDG_DATA_HOME). However, we believe .cache and .config directories could be safely removed. We understand that the decision of whether to keep user data (again, XDG_DATA_HOME) would certainly require design input and update to the application removal interface. Nevertheless, if you consider the user is *removing* the application and not *logging out* (where the latter would make sense to keep all 3 directories [data, config, cache]), removing .cache and .config would be the natural expectation of the user. And I'm saying that from purely user perspective, not a developer one.

To summarize, we think that removing .cache and .config would be a natural thing to do and perhaps we could proceed with the fix, while we defer the matter of keeping/removing XDG_DATA_HOME of the app. In that case, if it's Google Drive, Instagram, or similar app, you documents/pictures are safe even if you remove the app, assuming the app has stored them in the right location. The things you loose are things like cached image thumbnails (which usually can be re-generated), custom layout options, ringtone selection and similar settings, which does not have any serious impact on user's app data, that is being removed. All of these are easy to recreate, assuming we leave the XDG_DATA_HOME, until we have a real UI/fix for it.

[John Lea (johnlea)]

Bug description updated with desired resolution from design, following in design and also with Thomas Voss.

[Jamie Strandboge (jdstrand)]

John Lea's update to the bug description does not match Michał's comment #9. Specifically, "100% of files associated with the app should be deleted" sounds like the user's data files will be removed. We cannot assume that the app will store its data in the cloud somewhere. I mentioned videos, pictures and music in a previous comment and this was addressed with the concession that core apps can't be removed. However, we can expect alternatives to these core apps in the app store, or apps that are different from the core apps with overlapping functionality. Ubuntu does not currently have a backup solution for the phone and people make mistakes-- as a user I would be livid if I or a child I let use the phone accidentally removed an application containing unreproducible personal, aka precious, data. The mental model may be simple that if you remove an app, you remove all of its data-- the problem is, this model is contrary to what is done in Ubuntu now. Uninstall is different than delete. If I uninstall libreoffice, I don't expect my Documents folder to be wiped. This simple mental model may only be fully understood after the user uninstalls an app and loses unrecoverable data. We can't just delete people's data.

Implementation-wise, the security team's concerns still stand regarding the dangers of automatically deleting user data and would like to be involved in the review process.

[Michael Vogt (mvo)]

The attached branch is now in line with Michał's comment #9 - it will delete the $XDG_{CONFIG,CACHE}_DIR but leave the data dir.

[John Lea (johnlea)]

@jdstrand yes, when a user deletes an app, all of the data stored within that app *including the user's data* should be removed.

This is the same mental model as iOS user are used to. Also many apps these days have a server side component where the user's data is stored. In these cases deleting the app would not delete the user's account with the service provider. But this is of course up to app authors.

What we can do is display a warning when a user goes to delete an app informing them that this action will also delete all of their data within the app.

[Michał Karnicki (karni)]

Do we want to include this for RTM? Do we have capacity to do so?

[Jamie Strandboge (jdstrand)]

If this is pursued, please make sure this is implemented: "What we can do is display a warning when a user goes to delete an app informing them that this action will also delete all of their data within the app."

[Michael Vogt (mvo)]

Is there consensus on this now? I'm happy to land the branch, is Jamies comment in #15 addressed?

[Michael Vogt (mvo)]

It seems from #13 that this is addressed so unless I hear differently I will land my branch.

[Davide Alberelli (dadexix86)]

I personally don't agree with this decision and I think think it is highly inconsistent with the desktop experience.
The users' .config, .cache, .local folders should not be removed unless specifically asked by the user itself (or at least give us the possibility to choose to not remove them).

If I remove an app, so to say, to make space to try a bigger app that does not fit on my device, I would like to be able to recover the previous configurations without needing to setting it up again once I remove the second one (this is what happens by default on Android's systems - at least the ones that I tried).

The best option for me would be to have global Settings triggers "Actions to do when uninstalling apps" with user-specified configurations
"delete personal configurations" (not selected)
"delete cached data" (selected)
"delete other local data" (not selected)
and a warning when uninstalling saying "you are going to remove this, this and that data", depending on the chosen configurations.

[dobey (dobey)]

I must strongly disagree with the design decision as noted by John here. While it makes sense to delete some things (notably, cached data) upon removal of an app, it does not make sense to delete all things.

I'm inclined to mark this bug as won't fix. There is no good precedent or evidence that suggests this action would be beneficial to the user, yet there is plenty of evidence to suggest it would be harmful to the user. I would be ok with a change that deletes the cache only (the point of cache is that it should be recoverable and not harmful to the user to remove it), but any of the user's personal data or configuration for an app, should not be removed. Instead, there should be some other UI to allow a user to fully remove the data associated with an application package, if the user CHOOSES EXPLICITLY to do so. Implicit destruction of data is harmful, and it has been shown to be harmful countless times throughout the industry.

I would also suggest that system-settings storage info has UI to delete all current cached application data, similar to what Android has.

[John Lea (johnlea)]

@dobey this needs to be discussed with tvoss - will happily update this change request, but only with tvoss's agreement

[Alberto Mardegan (mardy)]

Just adding my 2 cents.

One possibility would be to let the application specify a text message to be displayed when the app is going to be removed. For instance, a photo manager app could want to tell the user "All of your photos currently stored in CoolGallery will also be removed."

So, if the user does not agree with this, he can cancel the application removal, copy his data to another app and then remove the first app again.

[Dubstar_04 (dubstar-04)]

I keep getting an out of storage message in the camera app. The photos are saved to the SD Card that has over 10GB available.
The storage ui shows that 'other files' make up the majority of the internal storage consumption. The 'other files' seems to be folders for apps that are no longer installed and there is no method to delete.

These folders should be deleted with the apps.

[Pat McGowan (pat-mcgowan)]

Checking the Other files storage usage on two long used and never wiped phones

Nexus 4 2.0GB
MX4 1.6GB

While these phones were not treated 100% as a consumer might those sizes are a bit disconcerting. I do not yet now what is included there.

It would be nice to present an interface to manage files that are no longer needed, such as for uninstalled applications, or logs
We should also do an audit to ensure we are not leaking storage unnecessarily, such as through unrestricted log files.

the Storage panel could be updated to separate out "Uninstalled apps" with an option to delete these on a per app basis

[Krzysztof Tataradziński (ktatar156)]

In Ubuntu System Settings should be also option to remove only app data and config files (without deleting app itself). It can be useful if we want to have 'clean' install of an app. Similar thing is implemented in Android:
http://www.download.net.pl/upload/News%20January%202015/AndroidUsuwanieAplikacji/usuwanie-2.jpg

[Michael Zanetti (mzanetti)]

Reopening this. The current situation is just too much of a mess.

Currently online accounts are deleted when an app associated with it is uninstalled. While all other data is preserved for good reasons.

I think we can't just delete user data when an app is deleted. The current situation that accounts are deleted is already too much of an issue in various situations. Also, I'm arguing that we just can't delete for instance the user's authenticator tokens just because he temporarily uninstalls the app for whatever reason.

We might want to offer a possibility for the user to delete an app's data, but just deleting it without asking is not an option.

[Krzysztof Tataradziński (ktatar156)]

@Michael
That solution seems to be good,
https://launchpadlibrarian.net/223769885/_1499971.png
asking if we want to delete them or not.

[Alberto Mardegan (mardy)]

Michael, can you please detail what are these "various situations", where you'd want online accounts to persist? I know of bug 1454210, which will eventually be fixed, but what else?
I cannot think of any reason why you would not want your account deleted, especially given that it doesn't store any user data -- it can be recreated by just typing a username and a password.

[Julia Palandri (julia-palandri)]

Why not ask the user if they want to delete the account settings, the app data or both? One might uninstall apps for a variety of reasons, so maybe the best approach is to let the user decide what to do (maybe even adding some info about how much storage the app data is using could be useful when prompted for a decision)

[SB (emehntehtt)]

As a user I support the option where OS asks the user whether he wants to uninstall application and keep the data or uninstall the application and wipe all data, showing how much disk space application is taking up would be a bonus, but prompting for uninstallation or full removal is ok too. Wipe all data option should wipe the whole application folder if possible so there are no leftovers like empty com.ubuntu.application folder or anything else like that.

[dobey (dobey)]

Added ubuntu-system-settings as really, the settings page which shows how much data is being used by apps, is what should implement a way to delete that data.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Norbert (nrbrtx)]

Confirmed on BQ M10, OTA-13. Folder in ~/.local/share/... persists after app remove.