provide and support a top-level library package for libmozjs (Was: Unable to use libmozjs.so in an application, because of library path problem.)

Bug #286906 reported by James Su on 2008-10-21
64
This bug affects 9 people
Affects Status Importance Assigned to Milestone
XULRunner
Confirmed
Medium
mozjs (Ubuntu)
Wishlist
Unassigned

Bug Description

Binary package hint: xulrunner-1.9

source package: xulrunner-1.9
version: 1.9.0.1
ubuntu 8.04 and 8.10

I'm working on a project which depends on xulrunner and spidermonkey (libmozjs). I use ubuntu 8.04 and xulrunner 1.9 as my development environment. The problem is, libmozjs.so is in /usr/lib/xulrunner-1.9.0.x/ subdir, the application can't find it if rpath is not used. But if rpath is used, it'll be bound to a specific version of xulrunner, which is not what we want. And there is no way to get the correct library path of libmozjs.so by pkg-config --libs mozilla-js.pc, which returns /usr/lib/xulrunner-devel-1.9.0.x/lib as the library path, which is only available in xulrunner-1.9-dev package.

The content of mozilla-js.pc provided by xulrunner-1.9-dev package:
----8<--------------------------------------------------------
prefix=/usr
sdkdir=/usr/lib/xulrunner-devel-1.9.0.1
includedir=/usr/include/xulrunner-1.9.0.1

Name: JavaScript
Description: The Mozilla JavaScript Library
Version: 1.9.0.1
Requires: nspr >= 3.12.0
Libs: -L${sdkdir}/lib -lmozjs
Cflags: -I${includedir}/stable -DXP_UNIX -DJS_THREADSAFE
----8<--------------------------------------------------------

You can see that there is no libdir definition in this file, and the -L parameter points to /usr/lib/xulrunner-devel-1.9.0.1/lib, which is only available when xulrunner-1.9-dev is installed.

Below is mozilla-js.pc from openSUSE 11:
----8<--------------------------------------------------------
prefix=/usr
sdkdir=/usr/lib64/xulrunner-devel-1.9.0.3
libdir=/usr/lib64/xulrunner-1.9
includedir=/usr/include/xulrunner-1.9.0.3

Name: JavaScript
Description: The Mozilla JavaScript Library
Version: 1.9.0.3
Requires: nspr >= 4.7.1
Libs: -L${sdkdir}/lib -lmozjs
Cflags: -I${includedir}/unstable -DXP_UNIX -DJS_THREADSAFE
----8<--------------------------------------------------------

You can see that libdir is provided in this file and "/usr/lib64/xulrunner-1.9" is a symbol link to the real xulrunner path(/usr/lib64/xulrunner-1.9.0.3).
Then I can use the libdir as rpath in my application and is still ok after upgrading xulrunner to a newer version.

Besides the libdir issue, the Cflags in ubuntu's mozilla-js.pc is incorrect, which should be -I${includedir}/unstable instead of -I${includedir}/stable

Both ubuntu 8.04 and 8.10 have this issue.

James Su (james-su) on 2008-10-21
Changed in xulrunner-1.9:
assignee: nobody → asac
Alexander Sack (asac) wrote :

xulrunner doesnt really ship a library in the "old" sense. xulrunner is more a sdk + a runner.

What is your use case? How do you currently try to link against it?

Changed in xulrunner-1.9:
status: New → Incomplete
assignee: asac → nobody
James Su (james-su) wrote :

The project I'm working on is: http://code.google.com/p/google-gadgets-for-linux, which has two modules that relies on xulrunner:
1. A script runtime module based on SpiderMonkey library(libmozjs.so), taking charge of javascript interpretation.
2. A browser module based on Gecko(gtkmozembed) and SpiderMonkey library, taking charge of html rendering. It needs inject methods into html window and run some javascript code, so only Gecko is not enough.

xulrunner is the only package which provides both Gecko and SpiderMonkey. Though Gecko can be accessed via xpcomglue, SpiderMonkey can only be used as a shared library. So in this point of view, xulrunner ships not only a sdk + runner, but also a shared library.

Dan Winship (danw-gnome) wrote :

This currently breaks gnome-shell as well, because gjs uses spidermonkey. bgo bug is http://bugzilla.gnome.org/show_bug.cgi?id=573413. (I was going to link that to this bug, but the "Also affects project" link only talks about *upstream* bugs, and this is downstream.)

FWIW, removing libmozjs-dev and installing xulrunner-1.9-dev in karmic allowed me to build gnome-shell.

I work on the ELinks Web browser (<http://elinks.cz>). ELinks has (presently very minimal) support for ECMAScript via SpiderMonkey, so this issue affects ELinks as well.

Until recently, I had been building ELinks with libmozjs1d and libmozjs-dev. However, xulrunner-1.9.1, on which firefox-3.5 depends, conflicts with libmozjs-dev. In order to be able to update to the latest release of Firefox, I have changed ELinks to be compatible with the new interfaces in the new xulrunner, and I have manually added some symlinks (/usr/lib/libmozjs.so -> xulrunner-1.9.1.1/libmozjs.so and /usr/include/mozjs -> xulrunner-1.9.1.1/unstable).

ELinks's ECMAScript support is optional, and Ubuntu's ELinks package does not enable it, although Debian's does, and my own build does, so this issue with ELinks does not necessarily cause another bug in Ubuntu, but it is inconvenient for me and anybody else who wants to enable ECMAScript in ELinks on Ubuntu.

Please let me know how the ELinks project should proceed to maintain compability with Ubuntu. Many thanks!

Ubuntu wants to ship libmozjs as a system library. I think the main thing they want is just for us to put a version number in the libmozjs.so filename.

It seems to me that we should be able to do that on Unix-y platforms. I think I'll just try and do it and ask Jim the questions as they arise. (Jim, it would be helpful if you could hop on IRC and ping me though, so I can do it while you're available to answer my dumb questions.)

We should not do this for 1.9.2, and should do it for 1.9.3 (on all platforms I think... no reason for Windows to be different from unixy platforms in this respect).

Slightly related: bug 469654 will probably change js3250.dll -> js.dll on Windows.

The main issue here is one of policy.

In short, people developing apps for Ubuntu want SpiderMonkey to grow up and start acting like a real library. That means we commit to keeping libmozjs's ABI backward compatible within each stable release branch (such as mozilla-1.9.1).

Either these users would be a net win for us or a net distraction. I encourage discussion on this point. On the whole I think diverse use of libmozjs is a net win, potentially a huge win.

What is the cost of having such a compatibility policy? Two things.

1. Mainly, everyone on the JS team would just need to be aware of it when writing patches that might be backported to a stable branch. Branch owners/release drivers should agree to the policy too. ABI backward compatibility means we can't delete a "friend API" function or change its signature or semantics incompatibly in a stable branch.

2. Second, there's the cost of actually doing releases, which seems small but deserves some consideration given what happened with 1.8 (in short, with tremendous effort I managed to shove a release candidate out the door, but never managed a final release). I think we can minimize this cost just by saying "whatever ships in gecko is an official SpiderMonkey release".

Please comment.

As someone who has installed SpiderMonkey separately (because of CouchDB), it would be very nice if SpiderMonkey acted (more) like a real library.

We're already changing API incompatibly:

http://bit.ly/3mJBqR

Other possible changes include OOM handling, but this one looks like it won't change for the API users:

http://bit.ly/1AjSUj

We definitely want to change the GC rooting API, and add a write barrier:

https://wiki.mozilla.org/JavaScript:SpiderMonkey:GC_Futures

We have had a pretty stable JS API for over a decade, with few (hardly any -- I am having a hard time recalling the last one) compatibility breaks, only some additions. But now we are in a tight competitive race and old API design flaws are restricting how quickly we can optimize.

We could #ifdef JS_BETTER_GC_API, etc. but I would prefer to ifdef the old stuff we're trying to get rid of. It happens that mark/sweep global GC will be needed anyway for the old generation, or the worst case, for any future copying generational GC. So we can probably support bad old API at some not too high cost. In this case!

In general, I'd rather not pay the price any longer. We've done it, it paid in ways we can't measure, there are "SpiderMonkey dark matter" embeddings all over the place (tellme.com and danger.com, now part of Microsoft!), but they are all way downrev AFAICT.

In light of all this, is now the time to be conservative with the JS API? I think not.

/be

(In reply to comment #5)
> We're already changing API incompatibly:
>
> http://bit.ly/3mJBqR
>
> Other possible changes include OOM handling, but this one looks like it won't
> change for the API users:
>
> http://bit.ly/1AjSUj
>
> We definitely want to change the GC rooting API, and add a write barrier:
>
> https://wiki.mozilla.org/JavaScript:SpiderMonkey:GC_Futures

All these things happened or will happen in trunk. I don't think any of them are likely to be backported to a stable branch.

If we ever do want to backport a breaking change to a stable branch, we could just bump the major version number. But that has not happened much in the past and I don't expect it to happen much in the future.

Ok, if we think we can manage major version changes and not change them so fast the downstreams object, then the only issue is the cost of maintaining the stable versions. We don't currently do much to maintain released spidermonkey tarballs. What expectations do system library users bring for maintenance?

/be

Whatever those expectations are, they bring them to their distributors, which in this case is Ubuntu/Canonical. I'm more than fine leaving the packagers to decide how/what to backport as needed, though our experience with embedders indicates that most likely people will just stay downrev and then update their app when they want fixes or features from the new version.

Sounds like a free lunch -- why don't I trust those?

/be

It's not a free lunch, it's a lunch that Ubuntu can buy at a reasonable price if they want to ship the library with different support/backport provisions than upstream will provide. I don't think that we should do backport or other multi-stream support work here. If it's not important enough to Ubuntu's customers for _Ubuntu_ to do it, I don't know why it would be important enough to us!

(In reply to comment #10)
> It's not a free lunch, it's a lunch that Ubuntu can buy at a reasonable price
> if they want to ship the library with different support/backport provisions
> than upstream will provide.

This begs the question of whose reputation gets damaged if the stable version is not well-maintained. We don't have resources to keep a stable version patched for security bugs, perhaps Ubuntu doesn't either -- then what?

/be

I don't think it begs that question, but it does raise it. It's not something we can do anything about -- we can't prevent people from shipping our code as part of their distribution, obviously, so unless we want to grant everyone in the world the right to allocate our time to their pet fixes by simply packaging up our work, I think we're best to just ignore this situation entirely, TBH. In the unlikely event that it causes a problem with noise in the community or press, we should simply point them to this bug, in which we indicated that we weren't going to backport fixes to standalone library, and so people shouldn't distribute it unless they're comfortable with that situation.

(In reply to comment #12)
> I don't think it begs that question, but it does raise it.

Ok, "raises" not "begs". But what is the answer?

> It's not something
> we can do anything about -- we can't prevent people from shipping our code as
> part of their distribution, obviously, so unless we want to grant everyone in
> the world the right to allocate our time to their pet fixes by simply packaging
> up our work,

This is off the mark again, no one is proposing that we enslave ourselves that way.

The issue is what happens if we fix this bug (marking it FIXED) so Ubuntu can and does provide a stable .so as a system library. Sure, they could ship a .so anyway (some do already, IIRC).

But we have a choice: fix this bug, or WONTFIX it (or stall, pretty much the same thing).

> I think we're best to just ignore this situation entirely, TBH.

Do you mean we should fix this bug and ignore the added risk of bad outcomes in the stable .so damaging our rep? Or WONTFIX/stall?

If this bug doesn't matter, then it's INVALID. I think it does matter. The way it matters is that we increase the good rep effects of a useful and safe-enough system lib, and the bad rep effects of the opposite kind of outcome, based on our supporting ("Make it possible") the system library with stable version.

It would be cool to have done this already for the years when our ABI was very stable. Now that we are destabilizing it, I'm a bit concerned about (a) rate at which we thrash it forcing too many stable major numbers downstream; (b) risk inherent in our seeming to leave (or be a party to leaving) unmaintained stable libs downstream.

/be

As far as this bug is concerned: if we're going to provide ABI version numbers, we should try to make them mean what they're supposed to mean. We're not willing to keep the ABI stable, and we don't want the bad PR of frequently ABI number changes. That sounds like WONTFIX to me.

Of course, Ubuntu and everyone else is free to add promises of their own, but it sounds like it supporting that effort would tie us down in ways we can't afford.

(In reply to comment #6)
> All these things happened or will happen in trunk. I don't think any of them
> are likely to be backported to a stable branch.
>
> If we ever do want to backport a breaking change to a stable branch, we could
> just bump the major version number. But that has not happened much in the past
> and I don't expect it to happen much in the future.

I guess I'm not sure we should be in the business of providing a stable branch. It doesn't seem to fit our priorities (speed and language leadership).

I concur completely with comment 15 here: I think we should WONTFIX the request in comment 0, but since it's already possible for Ubuntu to provide libmozjs however they choose, the resolution of the summary would be WORKSFORME?

Comment 5 was arguing based on JS API needing to change rapidly that we cannot afford to be in the business of providing a stable branch, but there are other reasons too. I agree with the conclusion in comment 15. Want to make sure we all agree on the reasoning.

/be

Changed in xulrunner-1.9 (Ubuntu):
status: Incomplete → Confirmed
Bastl (s-menge) wrote :

Same problem here when linking a haskell program that uses gtk2hs and gtkmozembed.

Micah Gersten (micahg) wrote :

Moving to xulrunner-1.9.1
We now have an upstream request for this.

summary: + provide and support a top-level library package for libmozjs (Was:
Unable to use libmozjs.so in an application, because of library path
- problem.
+ problem.)
affects: xulrunner-1.9 (Ubuntu) → xulrunner-1.9.1 (Ubuntu)
Changed in xulrunner-1.9.1 (Ubuntu):
importance: Undecided → Wishlist
status: Confirmed → Triaged
Changed in xulrunner:
status: Unknown → Confirmed
Filippo Giunchedi (filippo) wrote :

Hi,
judging from the upstream bug looks like it is not going to happen, would it be possible to ship libmozjs plus the soname like e.g. what is being done in debian? (libmozjs2d for xulrunner 1.9.1)

thanks

Joshua Timberman (jtimberman) wrote :

Bug #418762 was marked as a duplicate of this bug.

What can be done to remove the X11 requirements of xulrunner-1.9, particularly in regard to CouchDB's dependency? CouchDB is desirable on server installations (such as systems running chef-server, which uses CouchDB as a storage engine), and X11 is not desirable on servers. Can a xulrunner-nox packcage be created that only includes enough of xulrunner to get the spidermonkey requirement for CouchDB?

Darren Salt (dsalt) wrote :

This prevents people from building gxine, or at least makes it unnecessarily hard.

libmozjs0d is old and should long since have been replaced with libmozjs2d (and the corresponding libmozjs-dev), built from xulrunner 1.9.*. Debian has this right…

Micah Gersten (micahg) wrote :

Moving to xulrunner-1.9.2 since 1.9.1 is no longer in the devel release.

affects: xulrunner-1.9.1 (Ubuntu) → xulrunner-1.9.2 (Ubuntu)
Philippe Coval (rzr) wrote :

Hi, this bug is also affecting freewrl (see url at bottom)
I am about to package it for debian but can't be run on lucid (unless i use a rpath workaround) ...

Please build upgraded version of libmozjs-dev , if possible.

For info :

apt-cache rdepends libmozjs-dev
libmozjs-dev
Reverse Depends:
  xulrunner-1.9.1
  libxul-dev
  libsablot0-dev
  xulrunner-1.9.1

--
http://rzr.online.fr/q/vrml

Changed in xulrunner:
importance: Unknown → Medium
Jim Blandy (jimb) wrote :

What is it about Debian's strategy that is 'right'? Are they simply packing up successive versions under different package names that can be installed in parallel, and killing off versions as soon as practical? What in the world do they do for maintenance? Cherry-picking patches from modern SpiderMonkey and adapting them to the sources from even a year ago would be heroic.

I think a fundamental thing here is that there's a large world between:

[API changes all the time] -- ... -- [ stable API/ABI ]

In particular, in the middle area, there are small things that the SpiderMonkey consuming community would greatly benefit from that don't restrict SpiderMonkey really. See https://bugzilla.mozilla.org/show_bug.cgi?id=618381 for an example.

Another one is build support code for an independent-of-firefox SpiderMonkey release could be upstream, and it could have an independent version.

So, let me lay out some tasks:

* Land some version of 618318 so that embedders can adapt more easily when API changes
* Move xulrunner/installer/mozilla-js.pc.in into js/src/
* Add a version to the libmozjs.so
* Remove the Firefox configure flag --enable-shared-js
* Add a "make dist" target that generates spidermonkey-X.tar.gz to js/src/Makefile.in

(In reply to comment #18)
> * Remove the Firefox configure flag --enable-shared-js

Why would we do this?

(In reply to comment #19)
> (In reply to comment #18)
> > * Remove the Firefox configure flag --enable-shared-js
>
> Why would we do this?

Because it conceptually conflicts with a separate spidermonkey, assuming we keep the same .so and .pc names. And if you were just interested in consuming spidermonkey, why wouldn't you use /usr/lib/libmozjs.so.42 instead of being coupled to Firefox's /usr/lib/xulrunner-1.9.x/libmozjs.so?

(In reply to comment #20)
> Because it conceptually conflicts with a separate spidermonkey, assuming we
> keep the same .so and .pc names. And if you were just interested in consuming
> spidermonkey, why wouldn't you use /usr/lib/libmozjs.so.42 instead of being
> coupled to Firefox's /usr/lib/xulrunner-1.9.x/libmozjs.so?

Nothing prevents you to build spidermonkey from firefox and ship it in /usr/lib. That's what we do in Debian, and it has worked for a quite long time.

(In reply to comment #18)
> * Add a "make dist" target that generates spidermonkey-X.tar.gz to
> js/src/Makefile.in

AFAIK, there is no make dist target to generate source tarballs for other products. And there used to be spidermonkey releases, though it's been a while:
ftp://ftp.mozilla.org/pub/mozilla.org/js/

(In reply to comment #21)

> Nothing prevents you to build spidermonkey from firefox and ship it in
> /usr/lib. That's what we do in Debian, and it has worked for a quite long time.

Sure, though clearly separate tarballs are better. The larger question is what you do about the mozilla-js.pc file; in this case then, xulrunner-dev wouldn't be providing it, right? Or do you have separate .pc files?

(In reply to comment #20)
> Because it conceptually conflicts with a separate spidermonkey, assuming we
> keep the same .so and .pc names. And if you were just interested in consuming
> spidermonkey, why wouldn't you use /usr/lib/libmozjs.so.42 instead of being
> coupled to Firefox's /usr/lib/xulrunner-1.9.x/libmozjs.so?

Note that conceptually, you'd want a --with-system-js, but I'm already not convinced bug 618318 will actually work out, so I'm doubtful about having an up-to-date configure test.

(In reply to comment #23)
> Sure, though clearly separate tarballs are better. The larger question is what
> you do about the mozilla-js.pc file; in this case then, xulrunner-dev wouldn't
> be providing it, right? Or do you have separate .pc files?

$ apt-file show libmozjs-dev | grep pkgconfig
libmozjs-dev: /usr/lib/pkgconfig/mozilla-js.pc
$ apt-file show xulrunner-dev | grep pkgconfig
xulrunner-dev: /usr/lib/pkgconfig/libxul-embedding-unstable.pc
xulrunner-dev: /usr/lib/pkgconfig/libxul-embedding.pc
xulrunner-dev: /usr/lib/pkgconfig/libxul-unstable.pc
xulrunner-dev: /usr/lib/pkgconfig/libxul.pc
xulrunner-dev: /usr/lib/pkgconfig/mozilla-gtkmozembed-embedding.pc
xulrunner-dev: /usr/lib/pkgconfig/mozilla-gtkmozembed.pc
xulrunner-dev: /usr/lib/pkgconfig/mozilla-plugin.pc

(In reply to comment #23)
> (In reply to comment #21)
>
> > Nothing prevents you to build spidermonkey from firefox and ship it in
> > /usr/lib. That's what we do in Debian, and it has worked for a quite long time.
>
> Sure, though clearly separate tarballs are better.

They also complicate things. See the latest security release (3.5.16/3.6.13), for instance: spidermonkey adds new symbols that xpconnect (in firefox) uses.

(In reply to comment #25)
>
> $ apt-file show libmozjs-dev | grep pkgconfig
> libmozjs-dev: /usr/lib/pkgconfig/mozilla-js.pc

Ok. Does your libxul.so (xulrunner package) depend on libmozjs.so (libmozjs package), or are they on potentially separate schedules?

(In reply to comment #27)
> Ok. Does your libxul.so (xulrunner package) depend on libmozjs.so (libmozjs
> package), or are they on potentially separate schedules?

Package: xulrunner-1.9.2
Depends: (...) libmozjs3d (= 1.9.2.13-1) (...)

(I even force a strict dependency, version wise, though not strictly necessary)

(In reply to comment #28)
> Package: xulrunner-1.9.2
> Depends: (...) libmozjs3d (= 1.9.2.13-1) (...)
>
> (I even force a strict dependency, version wise, though not strictly necessary)

(Both packages come from the same source (iceweasel) package)

Bug 618318 is a little bit of a different bug. It doesn't make us 'grow up and become a ral library', but it goes about halfway there in a way which is useful for quite a number of users. Specifically, it:

- packages spidermonkey separately to FF and xulrunner.
- adds a major version number to the .so file, which is updated on API (and ABI as much as we can) changes
- allows mozilla to keep making incompatible changes, with a new .so number
- allows distributions to backport changes if they wish (in which case they would update the minor version number).
- only adds a tiny burden to mozilla (remember to increase the number of API changes, and aspirationally ABI changes).

It doesn't do everything distros would like (too high a burden on us), but it makes the world nicer for users.

I propose we WONTFIX this bug, and move our attention to bug 618318.

(In reply to comment #31)
> Wrong bug#?

My bad, bug 618381.

Micah Gersten (micahg) wrote :

We added a mozjs source in natty which provides libmozjs185-1.0.

affects: xulrunner-1.9.2 (Ubuntu) → mozjs (Ubuntu)
Changed in mozjs (Ubuntu):
status: Triaged → Fix Released

Excerpts from Micah Gersten's message of Sun May 22 16:52:47 UTC 2011:
> We added a mozjs source in natty which provides libmozjs185-1.0.

*WOOT!*

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.