Wishlist: Automated method for converting objects to paths before exporting (plt, hpgl, dxf, win32-vector-print).

Bug #1662531 reported by Alvin Penner on 2017-02-07
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Inkscape
Wishlist
Alvin Penner

Bug Description

There are four output formats that are sometimes used to send output to laser cutters/plotters. They are .hpgl, .plt, .dxf, and the (Windows only) extension Export->Win32 Vector Print. These were all originally designed to support only the <svg:path> element. Most or all of these formats will not output objects of the type <line>, <circle>, <ellipse>, rounded <rect>, or <text>. This deficiency has been known for some time, see https://bugs.launchpad.net/inkscape/+bug/192923/comments/4. Traditionally, the suggested work-around in these cases has been to first select the objects and then use the menu item Path->Object to Path. However, this has the disadvantage that it destroys the original form of the object so that it is no longer editable in the usual way. Requests for an automated, non-destructive way, of doing this have been made, see: https://bugs.launchpad.net/inkscape/+bug/600472/comments/5.
Attached is a proposal for such an extension, which will perform these operations automatically.

Alvin Penner (apenner) wrote :
Alvin Penner (apenner) wrote :

Attached is a pair of Python extensions, which will appear on the Extensions->Export menu. The extension "Pre-Process File Save As..." will perform the operations:
- save a copy of the file in a temporary location
- open a new instance of Inkscape and load the new file
- perform "Edit->Select All" and "Path->Object to Path"
- load the "File->Save As..." dialog to allow saving as .plt, .hpgl, or .dxf
- close the new file silently and return to the original file, which is unchanged.

Alvin Penner (apenner) wrote :

The extension "Pre-Process Win32 Vector Print" will do the same, but will execute "Win32 Vector Print" instead of "Save As...", to allow printing to a laser.

Alvin Penner (apenner) wrote :

The extensions are posted here for testing purposes. If there is no adverse response within a week or so, then I will commit them.
It is a pleasure to acknowledge helpful discussions with TylerDurden, as well as the code by Ryan Lerch and Moini.

su_v (suv-lp) on 2017-02-07
tags: added: extensions-plugins
jazzynico (jazzynico) wrote :

From a user point of view, it would be easier to add an option to the existing dialogs (and create a dialog for the extensions that don't have one), but I understand it would be more difficult to achieve.

That said, I managed to run prepare_print_win32_vector on Windows XP (32-bit), with lp:inkscape/0.92.x rev. 15364, but not the prepare_file_save_as script. The -prepare file is correctly created, but the Save as... dialog doesn't open and Inkscape switches back to the original document.

Changed in inkscape:
assignee: nobody → Alvin Penner (apenner)
importance: Undecided → Wishlist
milestone: none → 0.93
status: New → In Progress
Alvin Penner (apenner) wrote :

thanks for testing! apologies in advance for the lengthy response:

1. With respect to the File->Save As... dialog not appearing. I wonder if you would be willing to run the prepare_file_save_as extension again, just in case it works the second time. I have a vague memory that I encountered the same problem myself and then it went away. I know that there is a difference between the very first run and subsequent runs, because the default save-as directory is different. The first time I run it, the default directory is the directory of the temporary file, which is quite inappropriate. The second time I run it, it will use whatever directory I used the first time, which is hopefully more accessible. fwiw, I have run this successfully on:
Windows XP, Inkscape 0.92pre4 (32 bit)
Windows 10, Inkscape 0.92.0 r15299 (32 bit)
both of which were 32 bit builds.

2. With respect to the File->Save As..., again, I get a crash with a runtime error dialog when I run it on Windows 10, Inkscape 0.92+devel 15458 (64 bit) trunk. I will investigate that.

3. With respect to adding an option to the existing save as dialog, I am not entirely sure that this would work. The extension relies on the idea that a second instance of Inkscape will be run in order to prevent corruption of the original file, and it is this second instance that calls up the save as dialog, so the first save as dialog is no longer needed in this case.

4. With respect to adding an option to a print_win32_vector dialog, it currently does not have a dialog, it just calls up a printer dialog. I would like to make the original "Win32 Vector Print" extension invisible on the menu, but don't know how to do that. I would very much also like to merge the extensions "Pre-Process Win32 Vector Print" extension with the original "Win32 Vector Print" extension. However I don't know how to do that. When I merge them I end up in an infinite loop which is very difficult to recover from, because one instance of Inkscape is forever calling up a new instance.

I'll report back if I fix the 64 bit crash.

Alvin Penner (apenner) wrote :

on the 64 bit build, Windows 10, Inkscape 0.92+devel 15458
when I run gdb and run the extension "Pre-Process File Save As...", the crash is caused by the following message (popup dialog):

The fantastic lxml wrapper for libxml2 is required by inkex.py and therefore this extension.Please download and install the latest version from http://cheeseshop.python.org/pypi/lxml/, or install it through your package manager by a command like: sudo apt-get install python-lxml

Technical details:
No module named lxml

................................................

I am not sure why this is happening since I can successfully save the original file as dxf which also uses lxml. I assume this message is coming from the second instance of Inkscape, not the first one. Will investigate some more.

Alvin Penner (apenner) wrote :

the crash in trunk was diagnosed further by printing the output coming from the Popen command. The following code was used to examine the output:
    f = p.communicate()
    err.close()
    inkex.errormsg(str(f))

When this code was run on 0.92.x, Inkscape 0.92.0 r15299
the result was f = ('', '') as expected

When this code was run on trunk, Inkscape 0.92+devel 15458
the result was:
f = ('', "terminate called after throwing an instance of 'std::runtime_error'\r\n what(): locale::facet::_S_create_c_locale name not valid\r\n\r\nEmergency save activated!\r\nEmergency save completed. Inkscape will close now.\r\nIf you can reproduce this crash, please file a bug at www.inkscape.org\r\nwith a detailed description of the steps leading to the crash, so we can fix it.\r\nGtk-Message: GtkDialog mapped without a transient parent. This is discouraged.\r\n")

it appears that the crash is caused by a gtk message

Alvin Penner (apenner) wrote :

attached is a modified version of prepare_file_save_as.py, to be used just for testing purposes. This contains an extra, dummy, command line option, --query-x, which helps to avoid the crash because it operates successfully without a gui. The code also contains some printout of the stderr device.

running this on Windows 10, trunk, Inkscape 0.92+devel 15458, I get the output:
this output does not occur in the 0.92.x branch.

............................................................................
WARNING: ignoring verb EditSelectAllInAllLayers - GUI required for this verb.

** (inkscape.exe:10928): CRITICAL **: static void Inkscape::EditVerb::perform(SPAction*, void*): assertion 'ensure_desktop_valid(action)' failed

WARNING: ignoring verb EditUnlinkClone - GUI required for this verb.

** (inkscape.exe:10928): CRITICAL **: static void Inkscape::EditVerb::perform(SPAction*, void*): assertion 'ensure_desktop_valid(action)' failed

WARNING: ignoring verb FileSaveACopy - GUI required for this verb.

** (inkscape.exe:10928): CRITICAL **: static void Inkscape::FileVerb::perform(SPAction*, void*): assertion 'ensure_desktop_valid(action)' failed

WARNING: ignoring verb FileSave - GUI required for this verb.

su_v (suv-lp) wrote :

Attached is the output of the extension 'Export > Pre-Process File Save As...' with the (replaced) test version of the script ('prepare_file_save_as - Copy.py') run with lp:inkscape r15500 on Ubuntu 14.04.5.

This test version of 'prepare_file_save_as.py' crashes the spawned inkscape process (not the one the extension was launched from). The copy of the original file which is left behind in $TEMPDIR (like with the boolean extensions) contains the original content.

Note that the original 'prepare_file_save_as.py' from comment 2 does not trigger the crash of the externally spawned inkscape process (AFAICT it opens the modified document in second instance, prompts for saving and saves as with the selected output format as expected). The two other crashes reported separately (bug #1663585, bug #1663697) do not reproduce with trunk on Ubuntu 14.04 either.

su_v (suv-lp) wrote :

With trunk on Ubuntu, the original version of 'Pre-Process File Save As...' failed (spawned inkscape process crashes) with a file which contained a group with a rect and circle, and two clones of that group, one on an additional layer. If all clones are unlinked first, everything ungrouped and moved to a single layer, the extension works as expected. There was no error message returned to the main inkscape process, only the dialog from the second instance notifying about the crash:

«Inkscape encountered an internal error and will close now.
Automatic backups of unsaved documents were done to the following locations:
        /tmp/ink_ext_XXXXXX-prepare.svg.2017_02_10_20_24_44.0.svg»

Attempting to run the same command as the extension script in the terminal directly produced also a crash (backtrace attached), so it is not related to how the second instance is spawned via Popen:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7573d46 in SPLPEItem::removeAllPathEffects(bool) () from /home/su-v/local/src/inkscape/build/lib/libinkscape_base.so

su_v (suv-lp) wrote :

<off-topic>
The SVG file used for the last test (I'm aware that shapes and clones inside groups are not going to get 'prepared' based on the verbs used in the extension, this was just for testing). The mentioned crash can also be reproduced in the GUI following the same steps (select all across all layers, unlink clone, object to path). I'll further investigate and then file a separate report about this particular crash.
</off-topic>

Alvin Penner (apenner) wrote :

wow, thanks for testing!

with respect to comment 10:
>>This test version of 'prepare_file_save_as.py' crashes the spawned inkscape process (not the >>one the extension was launched from). The copy of the original file which is left behind in >>$TEMPDIR (like with the boolean extensions) contains the original content.

it is possible that the additional error dialog may have artificially introduced a crash by interfering with the timing or the communication. Would have been preferable to dump the error messages to a terminal/DOS/file but I did not know quite how to do that.

On Windows I still have a problem in that there is a distinct difference between 0.92.x and trunk, not sure why.

su_v (suv-lp) wrote :

Attached is the output of the extension 'Export > Pre-Process File Save As...' with the (replaced) test version of the script ('prepare_file_save_as - Copy.py') run with lp:inkscape/0.92.x r15365 on OS X 10.7.5 (same crash as with Inkscape trunk on Ubuntu).

The attachment also contains the backtrace of the spawned inkscape process as tracked by the native OS crash reporter (likely the crash is related to mixing without-gui (query-x) and with-gui (verbs) command line arguments (?)).

The original 'prepare_file_save_as.py' from comment 2 does not trigger the crash of the externally spawned inkscape process (AFAICT it opens the modified document in second instance, prompts for saving and saves as with the selected output format as expected).

Both tests had been run separately, with a new document containing a simple stroked and filled rect with rounded corners.

Alvin Penner (apenner) wrote :

so, if you run the original script with no debugging output, can you tell if the Object to Path has been successfully implemented? If I understand correctly, this does not crash. If you save this new file as normal svg, does it appear that the conversion to path is done?

On 2017-02-10 22:18 (+0100), su_v wrote:
> (likely the crash is related to mixing without-gui (query-x) and
> with-gui (verbs) command line arguments (?)).

Actually, the crash can be reproduced without extension directly on the
command line (using the same command line arguments as the test version
of the script), and it seems to be triggered by the last verb
'--verb=FileQuit'.

su_v (suv-lp) wrote :

On 2017-02-10 22:27 (+0100), Alvin Penner wrote:
> so, if you run the original script with no debugging output, can you
> tell if the Object to Path has been successfully implemented? If I
> understand correctly, this does not crash. If you save this new file as
> normal svg, does it appear that the conversion to path is done?

Yes - I tested on Ubuntu (trunk) with saving as 'Compressed Inkscape
SVG', and on OS X (stable) with saving as 'Inkscape SVG'. In both cases,
the rectangle from the original drawing had been converted to path as
expected.

Alvin Penner (apenner) wrote :

are you able to successfully execute the original sequence, which was:

p = Popen('inkscape --verb=EditSelectAllInAllLayers --verb=EditUnlinkClone --verb=ObjectToPath --verb=FileSaveACopy --verb=FileSave --verb=FileQuit '+tempfile, shell=True, stdout=PIPE, stderr=PIPE)

?

Alvin Penner (apenner) wrote :

sorry, I think our messages crossed each other in the dark, glad to hear it worked!

su_v (suv-lp) wrote :

JFTR - the results (with either version of the 'prepare_file_save_as.py' script) with lp:inkscape/0.92.x r15367 on Ubuntu 14.04 are the same as with lp:inkscape r15500 (trunk) on Ubuntu and with lp:inkscape/0.92.x r15365 on OS X. Based on the available platforms I can test on, there is no difference in this regard between stable and trunk.

Alvin Penner (apenner) wrote :

thanks for testing! I think I am going to try to backtrack in trunk to see if I can find out where the divergence between 0.92.x and trunk started (on Windows)

su_v (suv-lp) wrote :

Possibly the timing of individual page reloads, and lp's delays in
posting replies via email, causes some confusion in the timeline of the
final posts as seen on the bug report's page? ;)

On 2017-02-10 22:41 (+0100), Alvin Penner wrote:
> are you able to successfully execute the original sequence, which was:
>
> p = Popen('inkscape --verb=EditSelectAllInAllLayers
> --verb=EditUnlinkClone --verb=ObjectToPath --verb=FileSaveACopy
> --verb=FileSave --verb=FileQuit '+tempfile, shell=True, stdout=PIPE,
> stderr=PIPE)
>
> ?

I see no point in further testing this, since that same sequence of
command line arguments also crashes inkscape without being spawned from
python (i.e. it is not related to Popen()). See also comment 16.

Alvin Penner (apenner) wrote :

- the crash mentioned in comment 8, in current trunk, on Windows, is not reproduced on Windows 64 bit build Inkscape 0.92+devel 15113
- both the "Save As" and the "Win32 Vector Print" work normally with the Pre-Process extensions.
- This is the latest 64 bit trunk build at the website:
http://download.tuxfamily.org/inkscape/win64/

Alvin Penner (apenner) wrote :

- crash not reproduced in trunk rev 15443
- crash reproduced on Windows in trunk rev 15444
- appears to have been introduced in:
merge lp:~inkscape.dev/inkscape/doc_rotate
http://bazaar.launchpad.net/~inkscape.dev/inkscape/trunk/revision/15444

su_v (suv-lp) wrote :

Just curious: the prepare script uses '--verb=FileSaveACopy' which opens the file chooser dialog. If your current prefs are set to use GTK open/save dialogs, this would trigger bug #1663884. Could you test the behavior with native file dialogs?

Alvin Penner (apenner) wrote :

I normally use native Windows dialogs, so, yes, these tests were performed with Windows dialogs.

I just re-tested this with gdb.
attempting to perform a backtrace using rev15444 and using the file synfig_test.svg and using the extension "Pre-Process File Save As", I get the same "fantastic lxml wrapper" error and the same backtrace as in Bug 1663697
https://bugs.launchpad.net/inkscape/+bug/1663697
https://bugs.launchpad.net/inkscape/+bug/1663697/comments/7

Alvin Penner (apenner) wrote :

with respect to the error message obtained in comment 8:
this message comes from libstdc++-6.dll

.............................................
terminate called after throwing an instance of 'std::runtime_error'

  what(): locale::facet::_S_create_c_locale name not valid

Alvin Penner (apenner) wrote :

attached is a proposed patch for the crash that occurred in comments 7 and 8. The crash is apparently caused by the fact that the spawned instance of Inkscape does not have a valid locale associated with it. This problem applies only to the spawned instance, not the original.
   The patch uses an alternative definition of the locale, which does exist, and which avoids the crash.
   Any comments would be welcome, since I do not know what the recommended way of defining a locale is.

Alvin Penner (apenner) wrote :

attached are new versions of the extensions 'prepare_file_save_as.py' and 'prepare_print_win32_vector.py'. With these new versions it is not necessary to apply the patch from comment 28. The new versions avoid the crash (on Windows) by passing the environment variable 'LC_ALL' to the spawned version of Inkscape.

I will leave these versions here for about a week for testing purposes. If there is no adverse reaction within a week or so, then I will commit them.

Alvin Penner (apenner) wrote :

and the other one

Hachmann (marenhachmann) wrote :

Has there been any feedback (maybe via other channels than this bug report) on the patch proposed for Inkscape proper?

To my understanding, it appears that the crash concerns many/all? extensions that spawn a second Inkscape instance, and also, and more importantly, concerns normal usage of Inkscape via command line (on Windows).

So wouldn't this rather require a fix in Inkscape, and not in any extensions (of course, I don't understand the origin of the issue - why is there no locale?) ?

su_v (suv-lp) wrote :

In addition to Hachmann's comment, I would recommend to limit the scope of the change(s) - proposed to avoid the crash of inkscape instances spawned via Popen() in python-based extension scripts - to the affected platform(s), instead of forcing falling back to the C locale entirely on all platforms (even for those not affected by the symptom as exposed with trunk builds for Windows). Changes can be ifdef'd in C/C++ code, and wrapped in a conditional in python (e.g. "if os.name == 'nt':" as used elsewhere). If needed, the conditions can be later expanded to include other platforms.

Questions one could ask:
- what environment is inherited by inkscape processes spawned via Popen() on Windows
- did the environment for such processes change in GTK3-based trunk builds for Windows using current devlibs64
- what changes had been introduced with the merge of the document-rotation branch that affect such spawned processes [1]
- what is the minimal change to solve this for affected platforms

[1] Clue mentioned elsewhere:
«(...) because this Inkscape rev introduced a degree symbol (°) in a new spin box»
https://github.com/Moini/inkscape-extensions-multi-bool/issues/2#issuecomment-280758987
The '°' symbol for degree is already used elsewhere in Inkscape's GUI (via unit menu for angles) - why does the newly added one have such side-effects on Windows? Note also that rev 15444 does not use 'U+00B0' (DEGREE SIGN) but instead AFAICT adds multiple uses of 'U+00BA' (MASCULINE ORDINAL INDICATOR) in literal strings used for canvas text (via cairo) as well as in entries and labels (via Gtk).

Alvin Penner (apenner) wrote :

Hi,
    some responses, in no particular order
1. no other feedback, other than this bug report
2
. this concerns only those instances of Inkscape that are spawned by Python, not the command line, and also only on Windows
3. with respect to the question of fixing it in Inkscape, versus Python, I think it would be better to do it in Python. The Inkscape code that triggers the issue is: std::locale(""). This code will cause a crash if triggered from a Python Popen command, but not if Inkscape is called from the command line. As I understand it, this code tells Inkscape to inherit its locale from the parent that called it, which is a reasonable thing to do, and it would not be wise to arbitrarily override it.
4. wrt the question: 'what environment is inherited by inkscape processes spawned via Popen() on Windows'. To the best of my knowledge the answer is 'null'. Any use of the statement std::locale(""), even in just a print statement, generates an instantaneous crash if triggered by Popen. If triggered by a DOS run command, the answer is 'C'.
5. As far as the impact on Python extensions is concerned: to the best of my knowledge, the only affected extensions would be Synfig output, and the multi-bool extensions, and this current bug report. When I first learned of this problem, I generated output from every single Inkscape output extension I could find, and only Synfig raised a flag. In order for this problem to occur, you need to, first, be on Windows, secondly use Popen from Python, and thirdly use a verb that involves the gui. For example, commands like the commonly-used '--query-x' will not cause a problem because they apparently do not involve the gui. (It is kind of a sad commentary to realize how little these gui verbs are used. Given the potential power of the verb systen, one might have hoped that there would be many more examples of extensions that would be affected, but there are not.)
6. limiting the Python code change to Windows-only seems like a good idea. It could be done with a line of code like this:
if inkex.sys.platform.startswith('win'):
    inkex.os.environ['LC_ALL'] = 'C' # pass this to the spawned Inkscape

My proposal would be to patch the three known examples with code of this type, in Python.

Patrick Storz (ede123) wrote :

> 4. wrt the question: 'what environment is inherited by inkscape processes spawned via Popen() on Windows'. To the best of my knowledge the answer is 'null'.

That should *not* be the case!

We use "Glib::spawn_async_with_pipes()" [1] to launch python (see extension/implementation/script.cpp) and the documentations says "Like the main spawn_async_with_pipes() method, but inheriting the parent's environment."

The documentation on "Popen()" [2] says "If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of inheriting the current process’ environment, which is the default behavior."

So if "env" is not set in the call to Popen() the environment of the parent Inkscape process should propagate all the way through.

[1] https://developer.gnome.org/glibmm/stable/group__Spawn.html#ga6d494f70dd5b914102c89083f7996486
[2] https://docs.python.org/2.7/library/subprocess.html#popen-constructor

Alvin Penner (apenner) wrote :

I am probably going to display my ignorance here, but there appears to be a discrepancy between these two calls.
The glibmm documentation shows 9 arguments, as in:

void Glib::spawn_async_with_pipes ( const std::string & working_directory,
  const Glib::ArrayHandle< std::string >& argv,
  const Glib::ArrayHandle< std::string >& envp,
  SpawnFlags flags = SPAWN_DEFAULT,
  const SlotSpawnChildSetup& child_setup = SlotSpawnChildSetup(),
  Pid* child_pid = nullptr,
  int * standard_input = nullptr,
  int * standard_output = nullptr,
  int * standard_error = nullptr
 )

The Inkscape call in script.cpp shows 8 arguments, as in:

Glib::spawn_async_with_pipes(working_directory, // working directory
                             argv, // arg v
                             static_cast<Glib::SpawnFlags>(0), // no flags
                             sigc::slot<void>(),
                             &_pid, // Pid
                             NULL, // STDIN
                             &stdout_pipe, // STDOUT
                             &stderr_pipe); // STDERR

The argument that is supposed to be the environment, envp, appears to be 0 (no flags)

Alvin Penner (apenner) wrote :

okay, scratch that, I see that there are 2 versions, one with 8 args, and one with 9, charming...

Patrick Storz (ede123) wrote :

Look at the overloaded function call I linked to...
Omitting "envp" and using the environment from the parent instead is the intended difference between those two calls.

Alvin Penner (apenner) wrote :

okay, I think I have confirmed that Python indeed does inherit the Windows environment automatically. I ran a Python script from inside Inkscape and used the Python command:
fout.write(str(inkex.os.environ) + "\n\n")
to write out the entire environment. The environment was a duplicate of the normal Windows environment except for three new variables, which were 'LANG', 'PYTHONPATH', and 'INKSCAPELOCALE'. All three of these variables had reasonable values, as far as I can tell.
    So my question is, how come the variable 'LC_ALL' was not present?
    Is this variable present in non-Windows systems?
    Setting this variable fixed the crash, for reasons that I do not fully understand.

Alvin Penner (apenner) wrote :

The reason that I am obsessing about the variable LC_ALL is because this is the variable that keeps on coming up if you google the error message:
'_S_create_c_locale name not valid'
which was the first error message that actually made any sense in this crash.

Hachmann (marenhachmann) wrote :

On Linux Mint:
$ echo $LC_ALL

$ echo $LANG
de_DE.UTF-8

Alvin Penner (apenner) wrote :

Thanks for testing. I wonder if you would be willing to try this output test in Python.
In a Python extension I add the code at the start:

if (inkex.os.environ.has_key('LC_ALL')):
    inkex.errormsg('LC_ALL = ' + inkex.os.environ['LC_ALL'])
if (inkex.os.environ.has_key('LANG')):
    inkex.errormsg('LANG = ' + inkex.os.environ['LANG'])

and I get the output:
LANG = en_US
so there is no variable LC_ALL present.

Hachmann (marenhachmann) wrote :

I don't have a trunk build installed at the moment, Alvin, so probably this will be useless to you.
In 0.91 and 0.92.1, I get only

LANG = de_DE.UTF-8

Alvin Penner (apenner) wrote :

Thanks. In actual fact, I do not think that the output of that question will have changed recently. I mean, it is not the environment that has changed in trunk, it is the manner in which the environment is being interrogated that has changed.
    So I wonder if you would be willing to do one final test, which is to dump out the entire contents of the environment on your machine, from Python from Inkscape.
    In my case I used the following commands in Python to do this:

def effect(self):
    fout = open('\Windows\Temp\save_as_err.txt', 'wt')
    fout.write(str(inkex.os.environ))
    fout.close()

I executed this from inside a standard Inkscape extension and got the following result, attached here. You'll need to modify the file/path appropriately, but hopefully it should work.
    I guess what I'm hoping to see is any kind of variable that starts with the three letters LC_, since there seem to be a number of variables of this type, none of which exist in Windows.
    There must be something that distinguishes a Windows environment set from a non-Windows set.

su_v (suv-lp) wrote :

Quick summary of locale-related environment variables:
https://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html#Locale-Environment-Variables

Note that LC_ALL overrides all other more fine-grained localization variables defined in the user's context. There is usually no need to use it. Setting it to 'C' "unsets" all localization aspects, including numeric and time.

A spawned inkscape process with LC_ALL=C will not use translations in the GUI (defaults to en_US), and not know about other local time and numeric conventions.

The question AFAIU is why $LANG is unset on some platforms in inkscape processes spawned via Popen()?

Alvin Penner (apenner) wrote :

would it be possible for you to perform the test mentioned in comment 43?
we need to find out for sure whether there is a distinguishable difference between the environment that exists on a non-Windows platform, in a Python extension, run from Inkscape, versus a Windows platform, so that we can explain why this is an issue on Windows and is not an issue on other platforms.

su_v (suv-lp) wrote :

On 2017-02-19 10:09 (+0100), su_v wrote:
> The question AFAIU is why $LANG is unset on some platforms in
> inkscape processes spawned via Popen()?

Ignore that - it is not unset, at least not with Inkscape 0.92.1 and
inkscape_trunk_r15113 (both 64bit, 7z), but it is also not the same as
the one defined in the environment of the parent inkscape process, as
tested on Windows 10:

The current user's system language is set to en_US as default (via
Settings). In the CMD shell, I override it by setting $LANG before
launching inkscape:

> set LANG=de_CH
> inkscape

First instance:
- uses German as interface language
- Python: LANG=de_CH
Second instance (launched via extension):
- uses English as interface language
- Python: LANG=en_US
- tends to hang after quitting (probably bug #1655619)

--
Extensions used for tests (work-in-progress):
https://gitlab.com/su-v/lp1662531 (spawn_inkscape.inx, spawn_inkscape.py)
https://gitlab.com/su-v/inx-debug (pyinfo.inx, pyinfo.py)

Alvin Penner (apenner) wrote :

the variable LANG is not the issue. The issue is variables that begin with the prefix LC_.
would it be possible for you to perform the test in comment 43?

su_v (suv-lp) wrote :

On 2017-02-19 12:29 (+0100), Alvin Penner wrote:
> the variable LANG is not the issue. The issue is variables that begin
> with the prefix LC_.

I disagree - if $LANG is set, the LC_* variables will be adjusted based
on $LANG (see earlier link).

su_v (suv-lp) wrote :

Comment #46 continued:
If %LANG% (aka $LANG) is not set in the CMD shell, then the first instance has no LANG in the environment (as read by Python), the second inkscape instance launched from within Inkscape does have $LANG set to the system language (tested with en_US as well as with de_CH as current user's default language via Settings dialog).

Alvin Penner (apenner) wrote :

the variables LC_ do not even exist in Windows.
see comment 43
if they do not exist, then they cannot be set

su_v (suv-lp) wrote :

On 2017-02-19 12:29 (+0100), Alvin Penner wrote:
> would it be possible for you to perform the test in comment 43?

On Linux (and OS X), the python process and processes spawned via python from Inkscape inherit the same environment variables as present in the parent process (all of them). Whether or not the environment defines specific LC_* variables, or just $LANG, or $LC_ALL (to override any specific defined LC_* aspects) is up to the current user's context.

If you ask me specifically: with my local test builds on Mac OS X (regular, linux-style command line installations into a fixed prefix [1], launched via Terminal), the environment for inkscape usually only has $LANG defined (my default is LANG="en_US.UTF-8"), and none of the specific LC_* variables (because they are automatically defined based on $LANG). Only if I need to test a specific issue (or bug), I might override a single LC_* variable (e.g. LC_NUMERIC), or force an entirely different locale (via LC_ALL, e.g. LC_ALL="fr_FR.UTF-8") for the current session of inkscape:

$ LC_NUMERIC=de_DE.UTF-8 inkscape

Alvin Penner (apenner) wrote :

> would it be possible for you to perform the test in comment 43?

it is essential that the test be performed starting from inside Inkscape, and then running a Python extension and then dumping the environment directly from the Python extension spawned by Inkscape, so that the environment will be an exact duplicate of what the second instance of Inkscape is receiving from the first instance of Inkscape.

su_v (suv-lp) wrote :

Alvin wrote:
> it is essential that (...)

I'm fully aware of that.

<off-topic>
Since the comments section in this report is now so long, I hesitate to add further noise about a detail I recall when testing a branch of Jabiertxof (the initial implementation of the Measure LPE), where inkscape locally immediately crashes when adding the Measure path effect unless I explicitly launch it with LC_ALL=C (which I do not like, and which I think is wrong). IIRC it also has to do with the usage of std::locale(), and produces the same error as you describe in comment 27:

(inkscape:52522): glibmm-ERROR **:
unhandled exception (type std::exception) in signal handler:
what: locale::facet::_S_create_c_locale name not valid

The issue was never further investigated because that branch was merged into trunk later at a time where I no longer could compile it locally (due to C++11 requirement).
</off-topic>

su_v (suv-lp) wrote :

On 2017-02-19 12:29 (+0100), Alvin Penner wrote:
> would it be possible for you to perform the test in comment 43?

As requested, dumped env (via python-based extension [1]) from initial inkscape instance, as well as from a second inkscape instance (launched via python-based extension from within the first instance [2]) attached.

--
[1] https://gitlab.com/su-v/lp1662531/blob/master/src/dump_env.py#L34
[2] https://gitlab.com/su-v/lp1662531/blob/master/src/spawn_inkscape.py#L140

su_v (suv-lp) wrote :

Dumped environment as seen with trunk on Ubuntu 14.04.5 LTS.

su_v (suv-lp) wrote :

Dumped environment as seen with 0.92.x on Mac OS X 10.7.5 (updated with output of `locale`).

su_v (suv-lp) wrote :

On 2017-02-19 13:25 (+0100), su_v wrote:
> inkscape locally immediately crashes when adding the Measure path
> effect unless I explicitly launch it with LC_ALL=C

The crash in that other case (comment 53) is also prevented with LANG=C
(i.e. in that case it is not the explicit presence / definition of
LC_ALL or other specific LC_* variables in the environment, but the
value of the locale itself).

Alvin Penner (apenner) wrote :

    thanks for the output, it will be very helpful. However, I have to confess to a certain amount of confusion. Where did this information come from, did it come from the terminal, or Inkscape, or did it come from Python?
    Ideally what we need is information that comes directly from Python, that is, from a Python instance that was initiated by Inscape and dumped just prior to a Popen command. The dump code should look something like this, in Python:
def effect(self):
     fout = open('\Windows\Temp\save_as_err.txt', 'wt')
     fout.write(str(inkex.os.environ))
     fout.close()

or some equivalent of that, written in Python. That way we will know what has been inherited and is being transmitted by the Popen command.
 In any event, this is very helpful because it indicates that you have a large number of variables that begin with LC_. None of these variables exist in Windows. Or to be more precise none of these variables exist in Python just before the Popen command is executed. That partially explains why Windows is having a problem with this issue.
     I will do some further experimentation with this later in the day to see if some of these 'LC_' variables can be used in Windows.

Alvin Penner (apenner) wrote :

okay, scratch that comment, I just checked out your gitlab repository and I think I get it.
So I will investigate to see if I can use some of that info.

su_v (suv-lp) wrote :

Just a quick comment - I'm working on an update for the two extensions in the gitlab repo, to avoid executing affect() from inkex.Effect(): AFAICT it does call localize() now, and changes the environment on Windows. I'll comment later tonight (CET) about the findings.

su_v (suv-lp) wrote :

@Alvin - could you please pull the latest changes from the gitlab repo? Then try running
Extensions > Debug > Spawn Inkscape' with these two options:
1) 2nd Inkscape instance: Launch GUI; Use localize(): Off
2) 2nd Inkscape instance: Launch GUI; Use localize(): On

Do both crash with current trunk (GTK3) on Windows, or does only the second one crash?

Alvin Penner (apenner) wrote :

only the second one crashes

Alvin Penner (apenner) wrote :

totally by accident, I happened to have some debugging code in the routine desktop-widget.cpp. The debugging code looks like this:
    std::cout << "sp_dtw_rotation_output before locale()" << std::endl;
 s.imbue(std::locale(""));;

it was intended to track occurrences of the offending call. When I run your debugging code with localize off, I get the following Python window with data:

sp_dtw_rotation_output before locale()

sp_dtw_rotation_output before locale()

sp_dtw_rotation_output before locale()

sp_dtw_rotation_output before locale()

so, four calls to the offending code, but no crash.

when I run your code with localize on, I get the following output:
sp_dtw_rotation_output before locale()

sp_dtw_rotation_output before locale()

so, two calls, plus a crash.

su_v (suv-lp) wrote :

On 2017-02-19 23:21 (+0100), Alvin Penner wrote:
> only the second one crashes

Thank you for testing. The issue seems two-fold:

1) localize() in inkex.py may set unexpected (or invalid) LANG env on Windows.

2) internal code for some document rotation widget(s) does not guard against unexpected (or invalid) locale value.

Re 1): I don't know why that happens on your system, but not on the Windows10 system I currently have available for testing. A possible fix for localize() might be to check current_locale for 'None' (see attached), or whatever other value locale.getdefaultlocale()[0] returns on your system that later triggers the crash of the second instance.

Re 2): probably best tracked in a separate report.

Alvin Penner (apenner) wrote :

the localize() code appears to be working on my machine. I insert the following code into inkex.py

def localize():
    domain = 'inkscape'
    if sys.platform.startswith('win'):
        import locale
        floc = open('\Windows\Temp\save_locale.txt', 'wt')
        floc.write("'" + str(locale.getdefaultlocale()) + "'\n")
        floc.close()
        current_locale, encoding = locale.getdefaultlocale()

then I execute prepare_file_save_as.py, and I get a file with the entries:

'('en_US', 'cp1252')'

after that I get a crash

su_v (suv-lp) wrote :

The only difference between the two tests asked for in comment 61 is whether or not inkex.localize() is run before the second Inkscape instance is spawned via Popen().

inkex.localize() modifies the environment on Windows (sets/changes $LANG), which seems to affect the spawned second Inkscape (trunk) instance. You could test (re-read and write to file) the value of os.environ['LANG'] at the end of localize(), or right before calling Popen(), maybe that gives another clue as to what happens.

I'm running out of ideas what else to test, sorry, and due to the lack of current devel snapshot builds for Windows I can't reproduce the crash locally either ...

su_v (suv-lp) wrote :

Oh, one other test you could try: in a new CMD shell, set LANG to the same value as happens in localize(), and then launch inkscape (from the same shell).

C:\path\to\inkscape> set LANG=en_US
C:\path\to\inkscape> inkscape

Does that open the GUI of current trunk without crash?

Alvin Penner (apenner) wrote :

I doubt if this is related to the variable LANG. I monitored the variable LANG in the prepare_file_save_as extension using code like this:

inkex.errormsg('LANG = ' + inkex.os.environ.get('LANG', '-1'))
p = Popen('inkscape --verb=EditSelectAllInAllLayers etc etc

The output I got was 'LANG = en_US' followed by a crash. The LANG value was correct.
I believe the variables of the type LC_ are implicated. These variables simply do not exist on Windows at all. The only thing that makes sense in this crash is the error message:
what(): locale::facet::_S_create_c_locale name not valid

If you google this error message, virtually every hit you get will have two characteristics. Firstly it will probably refer to a non-Windows OS, and secondly it will almost certainly contain a reference to a variable of the general type LC_*, which do not exist on my machine.

In any event I will play with it some more over the next week or so. The error is not worth the bandwidth being given to it, so frankly I think almost any patch would be acceptable.

Alvin Penner (apenner) wrote :

Well, I am going to have to reverse my position once again. Here is an improved patch which works well for me, on Windows: Rewrite comment 6 to read:

6. limiting the Python code change to Windows-only seems like a good idea. It could be done with a line of code like this:
 if inkex.sys.platform.startswith('win'):
     inkex.os.environ['LANG'] = '' # pass this to the spawned Inkscape

This patch avoids the crash for me.

Alvin Penner (apenner) wrote :

sorry, that should read comment 33, not 6:
https://bugs.launchpad.net/inkscape/+bug/1662531/comments/33

The reason for doing this is as follows. If I monitor the environment variables from inside c in the routine main.cpp, I find that the LANG variable is not set (if Inkscape was started from DOS).

If I monitor the environment variables from inside Python as called by Inkscape, I find that LANG is set.

If I monitor again in Inkscape c code called by Python called by Inkscape, it is still set.

As soon as I unset it, the crash disappears. This appears to be the difference between running from DOS versus spawning from Python.

su_v (suv-lp) wrote :

On 2017-02-20 13:51 (+0100), su_v wrote:
> Oh, one other test you could try: in a new CMD shell, set LANG to the
> same value as happens in localize(), and then launch inkscape (from the
> same shell).
>
> C:\path\to\inkscape> set LANG=en_US
> C:\path\to\inkscape> inkscape
>
> Does that open the GUI of current trunk without crash?

Tested with Inkscape 0.92+devel 15534 on Windows 10: current trunk
crashes on Windows if launched from an environment which has LANG set to
anything other than C or POSIX - this can be from a python process
(inkscape with GUI launched via Popen()), or from the CMD shell (aka
DOS) with LANG set e.g. to en_US or de_CH.

Inkscape 0.91, 0.92.0 and 0.92.1 on Windows (32bit, 64bit, installed
from 7z into custom location) do not expose this crash, and set the GUI
language according to LANG as defined in the environment.

I'll file a separate report about this crash (it is not related to
running extensions, the nested calls to Inkscape with GUI (e.g. for
verbs) in combination with the localize() function of inkex.py simply
exposed it).

Alvin Penner (apenner) wrote :

committed to rev 15591
(without the changes in 'LC_ALL')

Changed in inkscape:
status: In Progress → Fix Committed
AMcBain (mcbain-asm) wrote :

Hi, as this tangentially affects me I think I have a proposal that may be better (at least for Print Win32 Vector). I've been working the last few days to enhance it and add support for all the elements that were previously not handled as indicated in the first post.

I even have better support for line caps, joins, and miter limit and more. This would negate the need for a prepare_print_win32_vector.py once I fix the issues outlined below. You can view the changes here: https://bitbucket.org/AMcBain/print-win32-vector-plus/

I was planning to submit it back for merging once I fixed that project's issues 1, 2, and 3 but hopefully also 6. (tl;dc: 1 -> fix polyline support, 2 -> fix start/end gap with closed shapes <rect> (no rx or ry) and <polygon>, 3 -> let users choose cut stroke width cutoff value), 6 -> turn text into a path before printing) This ticket gives me ideas on how to do 6 at least.

I contacted a few of the people related to laser cutting and Print Win32 Vector here on Launchpad to see if I could get other testers and input. Myself I've tested these changes with an Epilog Mini 24 to good success.

AMcBain (mcbain-asm) wrote :

I made those changes before I found this ticket and did them for myself and others locally. I did a quick scan of the tickets to make sure my work wouldn't run into any issues when I eventually tried to submit it back.

To post a comment you must log in.