[inkscape-quartz] gtk-mac-bundler: include python modules (and runtime) for extensions

[su_v]

Follow-up to
Bug #738973 “Issues with inkscape-quartz blueprint”

From comment 21:
> 1) The proposed solution for python extensions can't work (same issue
> was encountered with old launcher…)
>
> The install_names of the two *.so libs from lxml are changed to be
> relative to the executable of the app bundle. The extensions however
> launch a separate binary (/usr/bin/python) which can't find the required
> libraries since '@executable_path/../Resources/lib' no is relative to
> the path of the python binary.
>
> AFAIU either you need to include a copy of a python installation (or
> a virtualenv (?)) into the app bundle, or build and link the required
> lxml module statically using the system python version (and not the
> one from MacPorts), and not run install_name_tool on its *.so libs.
>
> ('osx-lxml.sh' script in my branch compiles the python module for the
> system python (without any references to MacPorts), installs lxml
> directly into the app bundle skeleton and then only needs to be
> copied along with the rest into the final app bundle).
<https://bugs.launchpad.net/inkscape/+bug/738973/comments/21>

Application bundle built based on r11625 of lp:~inkscape.dev/inkscape/dev-osx,
built on OS X 10.7.4 (Xcode 4.3.2)

Python-based extensions fail, apparently due to version incompatibility of shared libraries:

Error message from debug extension (which basically imports lxml, etree from lxml and numpy, and outputs the version and paths of the used python binary and loaded module):

Traceback (most recent call last):
  File "debug.py", line 5, in <module>
    from lxml import etree
ImportError: dlopen(/Volumes/cyan/src/inkscape/inkscape-dev-osx/packaging/macosx/inkscape.app/Contents/Resources/python/site-package/lxml/etree.so, 2): Symbol not found: _exsltDateXpathCtxtRegister
  Referenced from: /Volumes/cyan/src/inkscape/inkscape-dev-osx/packaging/macosx/inkscape.app/Contents/Resources/python/site-package/lxml/etree.so
  Expected in: /usr/lib/libexslt.0.dylib
 in /Volumes/cyan/src/inkscape/inkscape-dev-osx/packaging/macosx/inkscape.app/Contents/Resources/python/site-package/lxml/etree.so

Comparing libexslt loaded by system python with the one installed in MacPorts:
Chillida:macosx su_v$ otool -L /usr/lib/libexslt.0.dylib
/usr/lib/libexslt.0.dylib:
 /usr/lib/libexslt.0.dylib (compatibility version 9.0.0, current version 9.13.0)
 /usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.24.0)
 /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.3.0)
 /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
 /usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 46.1.0)
Chillida:macosx su_v$ otool -L /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libexslt.0.dylib
/Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libexslt.0.dylib:
 /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libexslt.0.dylib (compatibility version 9.0.0, current version 9.15.0)
 /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
 /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.8.0)
 /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.6)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
 /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0)
Chillida:macosx su_v$

$ port provides /Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libexslt.0.dylib
/Volumes/cyan/mp-quartz/with-a-long-long-long-directory-name/lib/libexslt.0.dylib is provided by: libxslt

AFAIK if the versions of libexslt don't match, extensions fail - we had similar issues with the old packaging script too (when it still used DYLD_LIBRARY_PATH instead of rewriting the install names).

AFAIU it's not save to build the module(s) in MacPorts, add the files from MacPorts into the app bundle, and at runtime load the bundled module(s) with the system python version. It might work on Mountain Lion for now, but fail again if MacPorts updates libxslt to a newer version not compatible with the installed system version.

[Gellule (gellule-xg)]

Maybe we should just package python2.7, numpy, and lxml altogether, and not rely on the version provided by OS X.

Was there any problem with lxml being compiled for a range of python version and architecture, and the actual version to use determined at run time?

[su_v (suv-lp)]

> Maybe we should just package python2.7, numpy, and lxml altogether, and not rely
> on the version provided by OS X.

A downside of including python runtime in the application bundle is download/file size: new native GIMP.app for example has additional ~94MB just for Python 2.7 in 'Contents/lib/python-2.7' (possibly this can be reduced e.g. by omitting all *.pyc files).

An upside of including a python runtime would be better support for external extensions which use pygtk to display an additional GUI (e.g. very popluar 'sozi' and 'textext' extensions). Once the app bundle includes a python runtime, it should be easy (?) to include py-gtk modules as well. Currently support e.g. for the 'Sozi' extension to create SVG presentations with the official Inkscape.app is only doable with a hack (<https://answers.launchpad.net/inkscape/+question/179289>).

> Was there any problem with lxml being compiled for a range of python version
> and architecture, and the actual version to use determined at run time?

Mainly two (IIRC):
a) old packaging relied on $DYLD_LIBRARY_PATH instead of rewriting install names
Caused system python to load libs from the app bundle not compatible with other libs loaded from the system.
b) new OS X versions provide newer Python version:
- Snow Leopard: 32bit/64bit conflict (Python 2.6)
- Lion/Mountain Lion: missing support for newer default python version (Python 2.7)
c) implementing optional support for custom python installs
With current official package, users have to edit the launcher script inside the app bundle to use a custom python install e.g. from MacPorts.

See also:
Discussion of b) in bug #482993 (Snow Leopard) and bug #819209 (Lion).

[su_v (suv-lp)]

Closing - out-of-date (the report was based on the branch 'osx-dev' which hasn't seen any activity in the past years).

Bug #738947 tracks the proposal to use gtk-mac-bundler for packaging (GTK+/Quartz based builds only).