upstream use of build-time defined PREFIX or DATADIR incompatible with snaps relocation

Bug #1583250 reported by Didier Roche-Tolomelli
36
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Snapcraft
Fix Released
High
Unassigned
snapd
Triaged
Undecided
Unassigned
snapd (Ubuntu)
Fix Released
Undecided
Cris Dywan

Bug Description

A lot of projects are using --datadir= to ship ui data dir path. Those paths are store in a generated config.h at compile time most of the time.

However, using snapcraft autotools plugins, you can't get around it. Modifying --datadir (like settings it to /snap/<snap_name>/current, what you shouldn't do as the path can change) will make it installed by the build system under /snap/<snap_name>/current/snap/<snap_name>/current). The only way for now is to patch the world, what we want to avoid to be able to get to latest upstream directly, or convince the world to change their whole build system to have relocatable path, one project after another.

This is typically the case for all gettext based project:
any gettext program calls
 bindtextdomain ("domain", LOCALEDIR)'
to read translations

where LOCALEDIR is $prefix/usr/locale (set in a config.h)

tags: added: snap-desktop-issue
Changed in snapcraft (Ubuntu):
importance: Undecided → High
Revision history for this message
Sebastien Bacher (seb128) wrote :

That's an issue for most desktop (and probably non desktop ones as well) and not easy to workaround.

Typically autotools define a "datadir" based on the build prefix, if you prefix=/usr then datas go to /usr/share/<name>/<somefile> and then you get code doing things like

" if (!gtk_builder_add_from_file (builder,
                                  DATADIR "/gnome-dictionary-menus.ui",
                                  &error))"

There seems to be no easy way to deal with that at the moment and it's blocking snappy trivial desktop softwares, some guidance on what to do would be useful.

Some possible options we can see

* have some patch over the upstream code that is applied before the build

Trevinho has an example with a custom snapcraft plugin to do that, it works but

- it means applications have to be "fixed" to be snap-ed which might be non-trivial/lead to bugs
- it requires maintaining a patch over the upstream code (until the change are upstreamed/merged which is likely to be rejected by upstream if the changes are snap specific

Note that encoding a runtime path at buildtime (/snap/<name>/current/datadir/filename) means the snap might stop working if the snap runtime changes in the futur, the alternative is to check for the env at runtime and have a dynamic path, but that's more code change

* find a way to define the right paths at buildtime

We didn't find a working solution so far since changing the prefix impacts on where the make install is putting files, also that means relying on a runtime behaviour that has no stability garanty (futur snap version might mount the fs in another location than /snap/name/current)

* divert syscalls with some sort of wrapper to append the snap prefix

that seems like non trivial work, hackish and not reliable

* get some part of the snap accessible under the snap "/" rather than "/snap..."

that would probably be the simplest/more efficient way but I guess there is probably some rational behind the design choice of having the squashfs showing under another subdir?

* other idea/solution?

Since that's blocking packaging on several applications having some recommendation of workaround/solution would be really nice

summary: - No way for dealing without distro patching configure DATADIR (and alike)
- path
+ upstream use of build-time defined DATADIR incompatible with snaps
+ relocation
Revision history for this message
Launchpad Janitor (janitor) wrote : Re: upstream use of build-time defined DATADIR incompatible with snaps relocation

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in snapcraft (Ubuntu):
status: New → Confirmed
Changed in snapd (Ubuntu):
status: New → Confirmed
Revision history for this message
Bill Filler (bfiller) wrote :

We are hitting this issue trying to snap EDS

Revision history for this message
Renato Araujo Oliveira Filho (renatofilho) wrote :

EDS has similar problems it uses the --prefix value to append on modules path use to load dynamic modules necessary on runtime.

For example:
<prefix>/lib/evolution-data-server/credential-modules

Another thing to keep in mind is how to allow external plugins? For example if we want to install a gedi or EDS plugin from another snap package. How this will work together?

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

For now I'd focus on just getting EDS as a snap to work. Having plug-ins from other snaps is a feature for another day. I suspect that you can configure EDS to use /snap/eds/current/usr as prefix and use organize in snapcraft but you may want to inspect code changes to understand $SNAP natively.

Revision history for this message
Kyle Fazzari (kyrofa) wrote :

Ideally upstreams would be made more relocatable, but that shouldn't be a requirement to get it snapped.

The short-term workaround is as Zygmunt mentioned-- use /snap/<name>/current in the prefix and organize. This is not a long-term solution as this path may not be consistent long-term, either due to future snapd developments or even snap renames.

On the snapcraft side we can improve things by creating a part that utilizes LD_PRELOAD and can be used by such unrelocatable projects. This is not a silver bullet as it doesn't work in all cases, but it will be an improvement. We'll try to re-prioritize our tasking to get to this sooner rather than later.

It may also be possible to deal with some of this pain with bind mounts in snap-confine, but that's more Zygmunt's area of expertise.

Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

We got multiple other examples: nginx, alsa…

summary: - upstream use of build-time defined DATADIR incompatible with snaps
- relocation
+ upstream use of build-time defined PREFIX or DATADIR incompatible with
+ snaps relocation
Revision history for this message
Sergio Schvezov (sergiusens) wrote :

After some discussion with Zyga and sabfdl we agreed on allowing /snap/<snap_name>/current so we can adapt the plugins make it so. I am curious about /snap/<snap_name>/current/snap/<snap_name>/current though. Any ideas how this happens?

Changed in snapcraft:
status: New → Triaged
no longer affects: snapcraft (Ubuntu)
Changed in snapcraft:
importance: Undecided → Medium
importance: Medium → High
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

/snap/<snap_name>/current/snap/<snap_name>/current is quite easy: when run make install, it will install assets at DESTDIR/PREFIX (or DESTDIR/DATADIR for instance).
So you end up with: parts/<part_name>/install/snap/<snap_name>/current, which, once the snap is installed is at $SNAP/snap/<snap_name>/current if you didn't do any workaround in your parts.

Revision history for this message
Kyle Fazzari (kyrofa) wrote :

> I am curious about /snap/<snap_name>/current/snap/<snap_name>/current though. Any ideas how this happens?

Yeah, using `./configure --prefix=/snap/<snap_name>/current` gets the right path into the binaries, but then it still needs to be installed with `make install DESTDIR=<installdir>` to get it into the snap. That combination turns into an install location of DESTDIR/PREFIX, or <installdir>/snap/<snap_name>/current in this case.

Cris Dywan (kalikiana)
Changed in snapd (Ubuntu):
assignee: nobody → Christian Dywan (kalikiana)
Revision history for this message
Cris Dywan (kalikiana) wrote :

See bug 1648431 for a potential solution via mounts.

Revision history for this message
John Lenton (chipaca) wrote :

From snapd's point of view the need for this is addressed with layouts in strictly confined snaps, and via fixed paths in classic snaps, so I'm marking it fixed there.

Changed in snapd (Ubuntu):
status: Confirmed → Fix Released
Changed in snappy:
status: New → Triaged
Revision history for this message
Sergio Schvezov (sergiusens) wrote :

core20 based plugins solve the --prefix issue (up to the publisher), with layouts the story should round up. Extensions in Snapcraft take care of most of the scaffolding required as well.

Changed in snapcraft:
status: Triaged → Fix Released
Michael Vogt (mvo)
affects: snappy → snapd
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.