chromium-browser does not follow XDG base directory specification

[Avamander]

Currently Chromium does not follow the XDG base directory specification which means that the cache and configuration folder aren't properly used.

This causes the issue that it's nearly impossible to properly migrate or back up Chromium configuration (between computers) and it makes for example mounting cache as tmpfs or wiping it very very difficult.

I felt those both immensely when trying to migrate my home folder, gigabytes of cache in ~/snap/chromium/[id]/.config/chromium/[folder] (and ~/snap itself is also not XDG base directory specification compliant, shame!).

There's also the issue that the previous configuration that is located properly, isn't migrated by the migrational package.

In order to fix this bug, it would require making the snap package follow the XDG base directory specification.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Olivier Tilloy (osomon)]

The chromium snap is strictly confined, and that means that it cannot read/write to hidden directories inside the user's home folder (such as $HOME/.config or $HOME/.cache). That's a security feature to prevent snap applications from reading other applications' data.

Instead, as you've found out, the app gets its own home directory located at $HOME/snap/chromium/current. The application itself does respect the XDG base directory specification, but with a rewritten $HOME.

Now, you mention the cache data. That one is stored in $HOME/chromium/common/.cache/. The default profile directory is stored in $HOME/chromium/common/chromium/.

The transitional package does copy over an existing profile from $HOME/.config/chromium if it exists the first time the snap is being run.

Given all this, I'm not really sure what the bug really is about? Can you share concrete use cases?

[Avamander (avamander)]

> The chromium snap is strictly confined, and that means that it cannot read/write to hidden directories inside the user's home folder (such as $HOME/.config or $HOME/.cache).

Change the folder it's confined to.

> That one is stored in $HOME/chromium/common/.cache/

No, not really. The profile folder contains gigabytes of cache files.

> The default profile directory is stored in $HOME/chromium/common/chromium/.

No, not really. Copying it over resulted in a utterly broken profile, some vital files are not in the common folder.

> Given all this, I'm not really sure what the bug really is about? Can you share concrete use cases?

I'm not sure what's unclear, XDG base directory specification is NOT followed, my environment variables are ignored and the wrong folder is used for everything. That has multiple negative effects, as I listed.

[Olivier Tilloy (osomon)]

Please do not confirm your own bugs.

> Change the folder it's confined to.

As I explained earlier it's a security feature implemented by snapd, the chromium snap doesn't get to choose where its data is stored. I just found bug #1575053 which looks very similar to your issue, so I'm going to mark it as duplicate, and I encourage you to read all the comments to understand the rationale.

> No, not really. The profile folder contains gigabytes of cache files.

You're right, it does contain cache data. That would be an upstream bug, can you please file it at https://bugs.chromium.org/p/chromium/issues/entry ?

[Avamander (avamander)]

> You're right, it does contain cache data. That would be an upstream bug, can you please file it at https://bugs.chromium.org/p/chromium/issues/entry ?

I will

> I just found bug #1575053 which looks very similar to your issue, so I'm going to mark it as duplicate, and I encourage you to read all the comments to understand the rationale.

They're not duplicates because that one is about moving ~/snap, this one is making the package actually XDG base directory specification compliant. If Snap does not offer basic mount features, then this is also a bug in snapd.

[Avamander (avamander)]

I also have to repeat that currently, copying over the folder contents you mentioned did NOT result in a working instance of Chromium. All of my extensions broke, databases failed to open, the common folder is just incomplete and needs attention. Please do try it yourself.

That absolutely 100% NOT a duplicate of the general snap lunacy.

[Avamander (avamander)]

Oh and I didn't confirm my own bug, that was done by the janitor because someone else felt that this affects them, I just restored the previous status.

[Avamander (avamander)]

Upstream bug report that would resolve a half of this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=1106754

Just to summarize what's left broken by this specific Ubuntu package:

  * Config and cache are not in the folders defined by the base directory specification

  * Current `common`/configuration folder is incomplete - can't be backed up and restored

[Olivier Tilloy (osomon)]

Thanks for the upstream bug, I'll be following it closely.

Regarding the other half of the problem (XDG base directory specification compliance), bug #1575053 does say this in its description:

    Therefore snap packages should base the value of SNAP_USER_DATA on the value of XDG_DATA_HOME (or its specified default). For the "notes" snap, this would be "$HOME/.local/share/snap/notes/1".

So I think that bug and the problem you're describing are essentially the same.

As to comment #6, can you elaborate on how you copied the existing profile? This code path has been exercised quite a bit by all users who migrated from the deb to the snap (the majority probably didn't even notice), and I haven't seen any complaint yet.

[Avamander (avamander)]

> Therefore snap packages should base the value of SNAP_USER_DATA on the value of XDG_DATA_HOME (or its specified default). For the "notes" snap, this would be "$HOME/.local/share/snap/notes/1".

That's still incorrect.

> As to comment #6, can you elaborate on how you copied the existing profile?

Literally just did that, copied the folder over. It didn't work.

[Olivier Tilloy (osomon)]

> That's still incorrect.

How so?

[Avamander (avamander)]

> How so?

The actual software of which the configuration belongs to is chromium (or "notes"), not snap, snap is "just there" in the middle.

Say I want to wipe snap's configuration, I should be able to wipe `$XDG_DATA_HOME/snap` without losing my n+1 snap application's data. Very simply put, `$XDG_DATA_HOME/snap` is for snap, `$XDG_DATA_HOME/chromium` is for chromium. It would be equally silly to have i386 software use ``$XDG_DATA_HOME/i386/software`.

Thirdly, it creates a discrepancy in configuration folder location between distributions, which is pointlessly annoying and hinders distro-hopping.

[Olivier Tilloy (osomon)]

I don't think the specification¹ mandates how folders should be structured under $XDG_DATA_HOME and $XDG_CONFIG_HOME. Or is there a follow-up specification that does that?

Hopefully snapd behaves (mostly) the same on all supported distributions, so distro-hopping shouldn't be a concern either.

¹ https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

[Avamander (avamander)]

> I don't think the specification¹ mandates how folders should be structured under $XDG_DATA_HOME and $XDG_CONFIG_HOME.

`$XDG_CONFIG_HOME/$SOFTWARE_NAME` is the de facto way, snap ignores it. Technically, yes, you're absolutely free to use hashes instead of your software's name when you store things in the configuration folder, really doesn't mean it's a good idea.

> Hopefully snapd behaves (mostly) the same on all supported distributions, so distro-hopping shouldn't be a concern either.

You're making three assumptions. That snap will behave the same across distributions, that snap is supported on different distributions at all, and that someone even wants to install the snap package. All of them are frankly really bad assumptions and just cause people pain.

[Olivier Tilloy (osomon)]

> `$XDG_CONFIG_HOME/$SOFTWARE_NAME` is the de facto way

That's different from "does not follow the XDG base directory specification".
A specification (hopefully) is unambiguous. Anything else has little objective value.

> That snap will behave the same across distributions

That sounds like a reasonable assumption to me.

> that snap is supported on different distributions at all

That's not an assumption, it's a fact: https://snapcraft.io/docs/installing-snapd.

> that someone even wants to install the snap package

Most (average) users don't care about packaging formats at all (nor do they care where software stores their cache/config, for that matter).

> All of them are frankly really bad assumptions and just cause people pain.

That's just your opinion/perception.
I'm marking again this bug as a duplicate of bug #1575053. Please file a new bug for the migration of existing profiles that doesn't work for you (with details on your profiles layout), so we can track the issue separately.

[Avamander (avamander)]

> Anything else has little objective value.

That's just your opinion.

> That's not an assumption, it's a fact: https://snapcraft.io/docs/installing-snapd.

It's an assumption, there are a lot of distributions that stay away from snap.

> Most (average) users don't care about packaging formats at all (nor do they care where software stores their cache/config, for that matter).

That's just your opinion and really not up to you to decide.

> That's just your opinion/perception.

That's really just your opinion.

> I'm marking again this bug as a duplicate of bug #1575053.

It's not! The location of the snap directory is totally separate from where chromium's configuration is stored.

[Avamander (avamander)]

Just to be sure you didn't miss a few facts here as you really like to rub it under people's noses:

* Snap mangles Chromium's configuration directory

* Snap-mangled Chromium configuration directory is not backupable

* Snap-mangled Chromium configuration directory is not compliant with the de facto interpretation of XDG-BD specification

* Snap-mangled Chromium configuration directory is not compliant with the spirit of the XDG-BD specification

* Snap-mangled Chromium configuration directory is unnecessarily not portable between distributions because it assumes the existence of snap

I see a common denominator here - snap's mangling that should really not be done.

[Sebastien Bacher (seb128)]

Thank you for your bug report but to maintain a respectful atmosphere, please follow the code of conduct - http://www.ubuntu.com/project/about-ubuntu/conduct. Bug reports are handled by humans, the majority of whom are volunteers, so please bear this in mind.

Bug reports should stick to fact, using wording like 'Snap-mangled' or doing statement as 'as you really like to rub it under people's noses' isn't appropriate and not a basis for any discussion.

I'm closing that as Opinion, you might disagree with how snapd is designed but you made enough of your point here, if you are interested into a discussion about snapd data handling you should rather do that on the snapd forum, https://forum.snapcraft.io/ . It's worth noting that other modern app format are doing similar choices, see e.g https://www.ctrl.blog/entry/backup-flatpak.html

[Avamander (avamander)]

> Bug reports should stick to fact, using wording like 'Snap-mangled'

I'm sorry, that's factually what happened. Should I have worded it Snap-disfigured, Snap-distorted, Snap-impaired or Snap-wrecked instead?

> isn't appropriate and not a basis for any discussion.

I have **never** had any software on Linux in the past ten years that has caused this many issues because of the stuff it does, how that behaviour is forced on me and which's developers are **this** dismissive and inappropriate. Issues, some major, have been open for years.

You should have enough empathy to understand that users don't take that behaviour very kindly for any significant period of time.

[Bast1aan (bast1aan)]

Also run into this issue. I used to exclude cache data from backups by simply excluding $HOME/.cache . Since the introduction of Snap apps, I have the troublesome task to exclude every .cache folder per snap app separately in the backup script. Especially with apps like Spotify it is necessary for it will otherwise include many GBs of cache data in my backups I really don't want.

It would be great if indeed Snap would comply to at least one base XDG_CACHE_HOME for all apps somehow, for instance by putting subdirectories for cache data per app,like ~/.cache/snap/chromium ~/.cache/snap/spotify etc.

[Jeff (malakai1197)]

Please see comment #20.

My online backup set specifically excludes $HOME/.cache (the default or unset value of $XDG_CACHE_HOME) to avoid churning backup snapshots with transient application cache data.

I don't know if chromium ever respected $XDG_CACHE_HOME (it would appear from the bug report mentioned in comment #8 that it does not), but in previous packages, it used to store cached data under $HOME/.cache/chromium and now it no longer does. Although it could have been a change in chromium's behavior, the change of location appears to be co-incidental with the transition from deb to snap packaging.

Please note that in the snap packaged version of chromium (build 88.0.4324.96), there is a $HOME/snap/chromium/common/.cache directory, but chromium does not store the bulk of its cache data in that location. Most of the cached data is found in 86 or so various directories under $HOME/snap/chromium/common/chromium/Default .

It is not clear to me that I can reasonably exclude either or both of $HOME/snap/chromium/common/.cache/ or $HOME/snap/chromium/common/chromium/Default/ to achieve the desired result of limiting transient disk cached data from entering the backup snapshots while keeping chromium user configurations backed up.

Since the previous deb packaged version of chromium did store its cached data in the XDG default location (although perhaps only by accident? maybe the deb package maintainer configured it?), and the current snap packaged version does not, I am looking for current package maintainer's guidance on the best way for our backup system to handle this change in behavior.

[Avamander (avamander)]

@~malakai1197

You're out of luck really, switch distributions that don't use half-baked snaps. Maintainers think that this usability regression is an "opinion" and that tells really more than enough.

[Olivier Tilloy (osomon)]

As previously discussed, this issue is twofold:

 - upstream chromium stores parts of its cache in the profile folder (that's https://bugs.chromium.org/p/chromium/issues/detail?id=1106754)

 - the snap format enforces that apps' cache folder is under $HOME/snap/$SNAP_NAME/common/.cache

The first point equally affects all packaging formats (including deb). The second one is a design decision for improved security that has its downsides, particularly (as pointed out in the last few comments) with regards to backup solutions that want to exclude one single cache folder in the user's home directory. As a mitigation, can your backup tool exclude folders including wildcards, e.g. -$HOME/snap/*/common/.cache ?

[Jeff (malakai1197)]

Thank you, Olivier. After spending an afternoon analyzing backups, I arrived at a similar conclusion and decided that adding an exclude pattern

*/chromium/*/*Cache

in addition to the one that excludes */.cache is going to be the right thing to do to solve our immediate problem.

[Mike Carifio (carifio)]

I'm also confused about the expected settings of XDG_{CACHE,CONFIG,DATA}_HOME in a snap and their relationship(s) to SNAP_USER_{COMMON,DATA}. Unfortunately, chromium isn't a good example to disentagle them. Chromium (or chrome) doesn't seem to respect these values either.

Rather than state what chromium uses (and yes, it's an important example), can you @osoman state what they _should_ be? My understanding is that XDG_CACHE_HOME defaults to $HOME/.cache, XDG_CONFIG_HOME defaults to $HOME/.config and XDG_DATA_HOME defaults to $HOME/.local/share. And it would appear that well-behaved snaps do that. However, since $HOME in a snap includes it's revision (not classic containment), that means that a new revision starts those locations afresh (or the first start of the new revision copies over the previous directories). I find this behavior a little surprising, even given the "security benefit" that confinement brings. In a sense, "my" data in .local/**, .config/** and ./cache/** is really now less mine and I have to know a little bit more about the directory tree at ~mcarifio/snap/${snap}/**. And I have to pay attention to when snaps update because my configuration(s) might disappear for a misbehaving snap.

It appears that the Intellij tools use classic confinement for this reason, since a new major revision will "copy over" the previous configuration when it first starts. But the app does that, not the underlying snap machinery.

xdg also adds to the potential confusion with .config/user-dirs.dirs, used when XDG_* isn't exported. And unfortunately SNAP_USER_{COMMON, DATA} seem to actually be independent of XDG_* although the names suggest (to me anyways) that SNAP_USER_DATA might play a role similar to XDG_DATA_HOME. Yes, that's because I can't read :-), not because that's stated at https://snapcraft.io/docs/environment-variables. Nonetheless, the descriptions are terse. Perhaps a sentence like "not be confused with XDG_DATA_HOME" would help.

I understand I'm recapitulating some of the thread above with less ummm enthusiasm and this is better discussed in the snapcraft forum. I think snaps bring a lot to the table, but when something breaks or is confusing, then I think we (the snap consumers) suddenly become very interested in the details of the packaging. And not by choice. Snaps are different. They make different assumptions (sandboxing, interfaces, multiple versions). And "personal configuration" e.g. .config/${sofware} and .local/share/${software}, is in that gray area between what's "mine" and what's "the app's". I don't have an answer to that. Just look at all the github repos for dotfiles. There are even directories of dotfiles https://github.com/webpro/awesome-dotfiles and dotfile managers like stow. But I would like to understand a little better how the "snap game" is played so I can leverage it's strengths and avoid its pitfalls. I do work to incorporate a tool into my workflow and XDG tells me to put that work in certain locations e.g. $HOME/.config/${pkg} and $HOME/.local/share/${pkg} for XDG compliant apps. HOME can't just move around willynilly for security's sake. It's nice to "know where to go" and go to one place. Snap's packa...