libmirclient in snaps gets out of sync with archive

[Pat McGowan]

Because we directly include mir client libraries in the platform (due to a dependency of qtubuntu), when the mir interface is revised we get out of sync. The mir team publishes a mir-libs snap that tracks these changes and exposes a mir-libs content interface. We should investigate if that can be used transparently by apps connected to the platform.

as of this report they are out of sync.

[Timo Jyrinki (timo-jyrinki)]

There is a build from 2017-02-08 in edge channel that is after the latest Mir landing.

However, that is blocked in QA queue from being promoted to stable channel due to bug #1662145.

[Pat McGowan (pat-mcgowan)]

The conclusion on the email thread seemed to be to enforce the use of mir-libs by any applications that want to run that way, and for unity8 to also use mir-libs.

per mterry:

One additional wrinkle here is that you probably also want to recommend to clients that they put the mir-libs path first in LD_LIBRARY_PATH.

It's easy to accidentally include libmirclientX in a snap through stage-packages dependencies. So whatever documentation you have around "you must use mir-libs" (which should already mention the need to include the mount path from mir-libs in LD_LIBRARY_PATH at all) should also mention that it should probably go first in that list.

The trick then is that mir-libs is now overriding all the mir-dependant libraries from the client snap. Things like glib, boost, etc. So your documentation should probably also include that possibility and mention that they can instead put mir-libs paths at the END of LD_LIBRARY_PATH, but they have to then add some logic to snapcraft.yaml to exclude the mir libraries out of their snap, in case they have them there.

This can be somewhat mitigated by making the mir-libs content snap as minimal and isolated as possible

[Pat McGowan (pat-mcgowan)]

The mir debs updated again in the xenial overlay earlier this week, so the latest platform snap published yesterday to edge is again incompatible with the mir-libs snap on edge.

Also the platform snap was published to every channel, so the previous working snap is not available except via a local revert.

[Cris Dywan (kalikiana)]

FYI for now I disabled automatic updates from https://code.launchpad.net/~ubuntu-sdk-team/+snap/ubuntu-app-platform - snaps can still be installed by hand for testing purposes, and we can publish on demand. The latest update was due to adding new API needed by the Ubuntu browser.

I was thinking we could use a branch to get mir into ubuntu-app-platform, that would be the same Unity/apps would use.
As per the suggestion in this report, we could also to try and make ubuntu-app-platform use mir-libs. Not sure if that's generally possible or apps would have to use it explicitly - this will need a proof of concept before we know what's the best approach.

[Timo Jyrinki (timo-jyrinki)]

Right. Indeed we used to auto-publish to all channels except stable. We can re-enable auto-uploads later but I now configured it so that it'd be only to edge from now on. Stable promotions via QA as usual, maybe in case of UAP we might need QA team for other channels too, there are so many use cases using different channels.

[Michael Terry (mterry)]

There was a meeting today with the Mir team and some stakeholders and I believe the outcome was that mir-libs will continue to be the only supported source for Mir libraries.

So we should probably do the following:

- Drop Mir libraries from the ubuntu-app-platform snap.

- Update snapcraft-desktop-helpers to stop including Mir libraries in consuming snaps and to point at mir-libs's libraries if they are being used.

- Update all Mir-consuming snaps to plug into mir-libs and to include the path in their LD_LIBRARY_PATH. And to remove any accidentally included duplicate libraries in their own snap.

I'm working on #2 there (snapcraft-desktop-helpers). I'll add a task here. I'll also work on converting the unity8 snap to use mir-libs.

[Michael Terry (mterry)]

Adding snapd task to get the mir-libs content interface auto-connected.

[Michael Terry (mterry)]

Bug 1665775 is the request for mir-libs to be auto-connected.

[Michael Terry (mterry)]

I've got a branch up for changing desktop-launch to point at mir-libs [1]. I also adjust LD_LIBRARY_PATH, so apps won't have to manually add that. They just need to plug into mir-libs.

[1] https://github.com/ubuntu/snapcraft-desktop-helpers/pull/54

[Didier Roche-Tolomelli (didrocks)]

I'm not really found of this decision from a pure developer experience perspective.
What would make Mir special compared to other system libraries, like libsystemd, pulseaudio, evolution-data-server, your email service libs, and so on and so on?

Sure, we can separate all of them, but we are back in the deb world, which is exactly what we want to avoid with snap. Also, changing all existing snaps for that change really shows something isn't coorrect.
It will mean that every applications that depends on mir-libs needs to declare an additional plug, create an empty directory and so on…

I guess a better solution to pave the way forward is either:
- get a version and stable ABI/API between the libs and the server (or this won't work on the long term anyway).
- get that transparently transition for applications. Meaning, the content ubuntu-app-platform interfaces still ship the Mir functionality. However, this could be done either directly or via another content interface snap. It may needs some upstream development.

Please discuss those changes on the snapcraft mailing list to not only base on my opinion.

[Timo Jyrinki (timo-jyrinki)]

FWIW, the UAP branch for removing mir libs is available and linked now. Since we agreed not to publish anything new to even edge for now (not to mention beta or candidate), I'm not merging it yet or uploading anything to the store.

A manually built snap can be downloaded from https://code.launchpad.net/~timo-jyrinki/+snap/ubuntu-app-platform-nomir

[Michael Terry (mterry)]

Didier, I mostly agree with you, but you are missing the context and meetings that led to this approach. I'm just the messenger here, and anyone please correct me if I mess something up.

- The Mir team is actively working on a stable ABI 1.0 release. That's coming in short term (but will involve an ABI change when it lands).

- The Mir team is so-far uninterested in providing a stable socket protocol, certainly in the short or medium term. (Preferring instead to say the library is the interface.) This isn't a very snappy approach, granted. But it's been the approach they've used in the archive for a while.

- *IF* the Mir team says "using a content snap is the only supported way", we really don't want to make that content snap be u-a-p. Partly because there are Mir clients that don't care about the rest of u-a-p (GTK has a Mir backend for example). And partly because all the issues around LD_LIBRARY_PATH and library content snaps (which is sort of a nightmare that just hasn't happened to bite us yet) make it advantageous to strongly limit the scope of an always-required content snap.

- I agree in theory Mir is not different than those other system libraries. Except that those system libraries tend to have a more stable library and protocol interface policies. Policies that mean the interface can survive the lifetime of a Core series (which is all that we care about, right?). But Mir is not able to make such promises yet.

[Didier Roche-Tolomelli (didrocks)]

I still think that this proposal creates a real burden, ABI breakage to developers. This isn't part of the desktop helper for sure as it doesn't follow any of the snappy best practices.

To compare paradigms, libwayland has a versioned protocol which is part of the GNOME and KDE runtime in flatpak. I don't see this concept being different from us and make it more difficult to package desktop snap running on Mir than desktop flatpacks running on wayland.

I guess if the Mir team doesn't want to work on some stability, maybe get the Mir team to document how to do it, what they should depend on, and adding their own env variable to every single snap in the store (as this needs rebuilding them all already)?

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity8-desktop-session - 1.0.13+17.04.20170221-0ubuntu1

---------------
unity8-desktop-session (1.0.13+17.04.20170221-0ubuntu1) zesty; urgency=medium

  * Support connecting to mir-libs and unity8 when using unity8-snap-
    install. (LP: #1663048)

 -- Michael Terry <email address hidden> Tue, 21 Feb 2017 16:33:17 +0000

[Michael Terry (mterry)]

Didier and I settled on a version of desktop-launch that tries to use mir-libs first, falling back to ubuntu-app-platform (while the libs are in there), and if not, then falling back to an internal copy of the libraries.

That should help the transition and doesn't encode a requirement on mir-libs inside of desktop-launch. But should enable the use of them if the snap chooses.

This has landed and now also sets LD_LIBRARY_PATH to point inside mir-libs for your snap, if mir-libs is connected.

So go ahead and update your snaps. All you need to do is define the plug and create the $SNAP/mir-libs folder. Everything else should be done for you.

[Michael Terry (mterry)]

mir-libs is only 0.26.0, but xenial overlay has 0.26.1. I can't fix unity8-session to use mir-libs until that's updated.

[Timo Jyrinki (timo-jyrinki)]

UAP would be ready, but should an upload without mir-libs to edge be delayed to a later point or could it be done eg in the beginning of next week?

[Michael Terry (mterry)]

From my end, I don't know of blockers for the UAP-without-Mir upload.

However, I would like to just take this moment to note that I haven't heard any talk of actually bumping the UAP interface number. Which means that by removing libraries, we're breaking our snap interface contract -- one we've released into the stable channel.

Now that's fine, teething pains and all that. And maybe we don't expect anyone to seriously be using the Mir libraries from UAP. (Though someone might be...)

Has the UAP team considered what the Right Thing would look like here? At least as a thought experiment for next time. But maybe even if it's easy, we can do it here too.

[Michael Terry (mterry)]

Hmm, unity8-session runs into bug 1665123 when trying to use mir-libs. I'll continue to bundle for now.

[Michael Terry (mterry)]

OK, I found a way to work around that with symlinks, so unity8-session is now using mir-libs.

Though don't rush to upgrade; something (else?) broke, so edge isn't rendering on screen. I think it's a gsettings issue, digging into it.

[Timo Jyrinki (timo-jyrinki)]

UAP would probably like to move to the new tracks, like 1/stable, 2/stable etc. At the Hague sprint it was recommended to use what we currently use, ie have "content" field with ubuntu-app-platform1, where that 1 could be bumped to 2.

Tracks add another piece to the puzzle, where the latest stable content: ubuntu-app-platform1 providing snap could continue to be available in 1/stable track. However I'm not sure if everything would work out yet in practice if we'd have 2/stable with ubuntu-app-platform2 and apps defining to plug into that - could ubuntu-app-platform1, ubuntu-app-platform2 and apps using either co-exist fully with snap pulling updates correctly to everything.

For now, and the mirlibs case, we might not want to go to that though yet, but it's good that features needed are appearing.

[Pat McGowan (pat-mcgowan)]

@timo can we go ahead and get tracks allocated for this?

In hindsight we likely should not have published to stable but oh well

[Timo Jyrinki (timo-jyrinki)]

(requested on Rocket, no answer yet)