inkex.py not found for extensions in user's directory (regression)

[su_v]

Inkscape 0.47+devel r9249 on OS X 10.5.8,
still present in r9251 and r9257

Regression: Extensions installed in '~/.config/inkscape/extensions' no longer can find the modules in the system extension directory (e.g. 'inkex.py'). This worked without problems in previous revisions - without explicitly adding the path in the python script.

originally reported in comments (16-25) to bug #505107.

related:
bug #197475 “inkex.py not found for extensions in ~/.inkscape” (fixed in 0.47)

[su_v (suv-lp)]

console message (r9257):

PYTHONPATH = Contents/Resources/extensions:/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/python/site-packages/i386/2.5

The relative path is partially 'correct' as it points to the location of the extension directory inside the application package (which corresponds to '/usr/share/inkscape/extensions' on linux systems), but previous revisions have the full path added to PYTHONPATH.

e.g. Inkscape 0.47 reports this debug info:

['/Volumes/blue/src/Inkscape/Inkscape-0.47-1.LEOPARD+/Inkscape-047-1.app/Contents/Resources/python/site-packages/i386/2.5/lxml']
------------
lxml.etree: (1, 3, 6, 0)
libxml used: (2, 6, 16)
libxml compiled: (2, 6, 30)
libxslt used: (1, 1, 12)
libxslt compiled: (1, 1, 22)
------------
['/Volumes/blue/src/Inkscape/Inkscape-0.47-1.LEOPARD+/Inkscape-047-1.app/Contents/Resources/python/site-packages/i386/2.5/numpy']
------------
numpy version: 1.0.4
------------
sys.path: ['/Users/suv/.config/inkscape.extensions/LeWitt', '/Volumes/blue/src/Inkscape/Inkscape-0.47-1.LEOPARD+/Inkscape-047-1.app/Contents/Resources/extensions', '/Volumes/blue/src/Inkscape/Inkscape-0.47-1.LEOPARD+/Inkscape-047-1.app/Contents/Resources/python/site-packages/i386/2.5', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload', '/Library/Python/2.5/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC']
------------

[su_v (suv-lp)]

... extension script used to test.

[su_v (suv-lp)]

output of above script from Inkscape 0.47+devel r9257, after uncommenting the current path in debug.py:

 sys.path.append('/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/extensions')

Note: the relative path (added in main.cpp?) is incorrectly appended to the user's extensions directory, not to the location of the shared resources in the application package. All other resources (icons, tutorials, etc) are located and found by inkscape. I don't know if inkscape uses $INKSCAPE_SHAREDIR to find the resources - the variable is set in the launcher script (among others).

['/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/python/site-packages/i386/2.5/lxml']
------------
lxml.etree: (1, 3, 6, 0)
libxml used: (2, 6, 16)
libxml compiled: (2, 6, 30)
libxslt used: (1, 1, 12)
libxslt compiled: (1, 1, 22)
------------
['/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/python/site-packages/i386/2.5/numpy']
------------
numpy version: 1.0.4
------------
sys.path: ['/Users/suv/.config/inkscape.extensions/LeWitt', '/Users/suv/.config/inkscape.extensions/LeWitt/Contents/Resources/extensions', '/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/python/site-packages/i386/2.5', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload', '/Library/Python/2.5/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC', '/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/extensions']
------------

[su_v (suv-lp)]

in my current setup
  ~/.config/inkscape/extensions
is a symlink to
  ~/.config/inkscape.extensions/LeWitt

this has never caused issues with the user-installed extensions.

[su_v (suv-lp)]

> I don't know if inkscape uses $INKSCAPE_SHAREDIR
> to find the resources
$INKSCAPE_SHAREDIR is not relevant for this bug:

Setting $PYTHONPATH to 'Contents/Resources/extensions' is relative to the working directory from which the sequence of launcher shell scripts execs the inkscape binary.
<http://bazaar.launchpad.net/~inkscape.dev/inkscape/trunk/annotate/head%3A/packaging/macosx/Resources/script#L47>

AFAIU this worked prior to r9248:
1) `pwd` is '/path/to/Inkscape.app'
2) set environment and launch inkscape-bin
3) $PYTHONPATH is set by inkscape-bin to "Contents/Resources/extensions:$PYTHONPATH"
4) full path of the shared extensions directory thus is
   '/path/to/Inkscape.app/Contents/Resources/extensions'

Could it be that in recent revisions the spawned python process changes the working directory and starts from '~/.config/inkscape/extensions' and not in the directory that the inkscape-bin process was started from?

[Krzysztof Kosinski (tweenk)]

"Could it be that in recent revisions the spawned python process changes the working directory and starts from '~/.config/inkscape/extensions' and not in the directory that the inkscape-bin process was started from?"

Yes, it's the exact cause. I did this to work around an issue on Windows where Python 2.x can't load any script from an Unicode directory. I assumed that INKSCAPE_EXTENSIONDIR and other defines from prefix.h are always absolute paths.

[Krzysztof Kosinski (tweenk)]

Tell me whether 9261 works correctly. I made the extension directory in PYTHONPATH absolute.

[su_v (suv-lp)]

Inkscape 0.47+devel r9261 on OS X 10.5.8

PYTHONPATH now has the full path to the shared extensions directory prepended and extensions installed in '~/.config/inkscape/extensions' successfully import modules like 'inkex.py' without adding the path explicitly in the python script.

console message:
PYTHONPATH = /Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/extensions:/Volumes/blue/src/Inkscape/Inkscape-BZR/Inkscape.app/Contents/Resources/python/site-packages/i386/2.5

I also tested some of the bundled extensions:
a) Some extensions call inkscape in a new process (e.g. restack.py uses 'inkscape --query-all …') - so far I haven't noticed other issues related to the changed working directory of the spawned python process.
b) Some extensions depend on accessing subdirectories in the shared extensions directory (e.g. Barcode, Alphabet soup, 3d Polyhedron …) - so far those that I tested seem to work as expected.

thx for following up on this issue!

[su_v (suv-lp)]

JFTR - retesting with previous revision r9257 shows that even some of the bundled extensions failed: for example the barcode extension has python scripts in a subdirectory which fail to import 'inkex.py'.

[su_v (suv-lp)]

@Krzysztof - can you verify on win32 and linux with above attached debug extension installed in the user extensions directory that changing the working directory of the spawned python process doesn't cause the same regressions as on osx?

Another correction: the barcode extensions bundled with inkscape failed in earlier revisions (see comment #10) because I still had a test version installed in my user extension folder which took precedence over the one in the shared extensions directory. Sorry for the mistake, usually I am careful to rename both the inx file and the associated python script and change the id as well as the menu entry to avoid such (hidden) conflicts.

[Krzysztof Kosinski (tweenk)]

How about this solution to the "spawning Inkscape from Python" problem?
1. When started, get argv[0]
2. If it's not an absolute path, resolve it relative to the working directory
3. if such a file exists, add the basedir of the resolved name to PATH

[su_v (suv-lp)]

> How about this solution to the "spawning Inkscape from Python" problem?
What problem? Does it fail on Windows or Linux? Sorry if my wording in comment #9 was unclear: it works on OS X [1] and is not influenced by your changes to fix the win32 unicode issue.

> 1. When started, get argv[0]
> 2. If it's not an absolute path, resolve it …
> 3. if such a file exists, add the basedir …
Where would you want to change that? On OS X the current sequence of launcher scripts handles it and does not need to be 'fixed'.

[1] the second launcher script (named 'inkscape') adds the absolute path of its location to $PATH so that an executable named 'inkscape' can be found by the Popen() or os.popen3() commands used in extension scripts calling inkscape to query selected objects.

[su_v (suv-lp)]

To summarize my previous comments: as far as I could test it, r2961 has fixed the problem I initially reported for OS X.

Whether some extension scripts internally depend on the working directory being the same as 'inkscape' was being launched from I don't know and have not systematically tested. There are scripts that may use the working directory as default path for creating files (bitmap images, zip archives, …):

e.g. 'Extract Images…' now apparently defaults to extract the images into the shared extensions folder, certainly not a location one would expect to search for the extracted bitmap images (if the writing of the file doesn't fail because the user has not the permission to write in that directory).

[su_v (suv-lp)]

Maybe we should open a separate report about issues with extensions themselves?

Your printf() statement in line 578 of src/main.cpp breaks some extensions (Perspective, Voronoi Pattern, possibly others). IIRC we had a similar problem with output of a launcher script sent to stdout instead of stderr (see bug #168336 comment 16-22).

[Krzysztof Kosinski (tweenk)]

The printf bug is fixed in 9278. I'm marking this as fix committed - feel free to repoen if you find any related issue.

The Extract Images bug should be reported separately, because the behavior was wrong even before my change.