Simulator freezes until return is pressed in terminal

[Jonatan Liljedahl]

Most often (but sometimes not), running the simulator freezes the app, until one switches to the terminal where kicad was started[1] and press the return key a number of times, then the simulation completes. There's no errors or Y/n questions in the simulator log or in the terminal.

[1] - At least on OSX, kicad must be started from the terminal with proper environment variable set to make it find the needed ngspice init file and codemodels.

[Jonatan Liljedahl (kymatica)]

This *might* be a fix: add "set nomoremode" to your spinit file. At least after doing that I haven't seen any freezes during the last minutes..

It would be great if kicad could provide (or even generate on-the-fly) a proper spinit file.

[Maciej Suminski (orsonmmz)]

Hi Jonatan,

I am truly grateful for your findings. We have noticed the ngspice init file problem occuring on Windows as well, and seemingly the best way to deal with it is to send commands setting up the simulator environment before a simulation starts, so we can include 'set nomoremode' there as well. I hope I will find some spare time this week to fix it.

[Wayne Stambaugh (stambaughw)]

@Orson,

Please do not hard code any spice init options and make them
configurable because there are many options that are going to be
platform and system dependent. I would still like to pursue the
packaging solution before we put too much effort and add potentially
unnecessary code. On windows it was a simple configuration issue and it
appears that this may also be the case on osx.

Thanks,

Wayne

On 10/5/2016 5:12 PM, Maciej Sumiński wrote:
> Hi Jonatan,
>
> I am truly grateful for your findings. We have noticed the ngspice init
> file problem occuring on Windows as well, and seemingly the best way to
> deal with it is to send commands setting up the simulator environment
> before a simulation starts, so we can include 'set nomoremode' there as
> well. I hope I will find some spare time this week to fix it.
>
> ** Tags added: sim
>

[Jonatan Liljedahl (kymatica)]

Great!

The issue is how to make the bundled ngspice find the init files. Maybe the best would be if kicad could package/bundle a default init file, and then the user can override it with ~/.spiceinit as expected. (Currently ~/.spiceinit is ignored)

If a default/bundled init file is used (if you find a way to make libngspice use it), it must also know about the location of code models. Maybe it makes sense to bundle these and all the ngspice stuff too in the package, not only the libngspice?

It might be good to at least hardcode 'set nomoremode' (or 'pre_set nomoremode') however, we don't want the app to freeze!

Good idea to include a configuration UI, at least a built in text editor for the ~/.spiceinit (and perhaps a per-project init file as well?) but it could be made very user friendly with some common options and settings as UI widgets/fields.

[Maciej Suminski (orsonmmz)]

Hi Wayne,

I see the following options, both start with forcing ngspice to ignore the default spinit file (e.g. by setting SPICE_SCRIPTS env var to a not existing directory):
- Manually issue commands to load .cm files and enable nomoremode (having it disabled apparently can freeze the simulator and gives no benefits here). This way we can have a list of directories to be checked and pick the one which in fact contains .cm files.
- Load the spinit file in KiCad and execute the stored commands with adjusted paths.

Alternatively, we can override the SPICE_SCRIPTS env variable to contain a relative path to spinit, and make it contain relative paths as well. It will solve the problem for KiCad installers/packagers, but not for the others.

[Wayne Stambaugh (stambaughw)]

Hey Orson,

I see a problem with your solution on Linux. On linux, the default
spinit file will work out without any changes. Ignoring it may cause
issues for advanced users who use custom settings. I know I wouldn't be
very happy if my settings that I spent a lot of time researching were
ignored by kicad.

My preference is to see if you can get the relative path solution
working so that a reasonable default can be provided in the installer
which would be a better all around solution. This would also give
advanced users an option to configure ngspice to suit their needs.

There is a lot of individual knowledge acquired while trying to figure
out some of the issues with the simulation. I'm assuming you are going
to provide documentation for the simulator so please make sure this
local knowledge makes it's way into that documentation. I'm sure these
issues will pop up once more users start using the simulator. I would
rather not have to keep rehashing this issue.

Thanks,

Wayne

On 10/6/2016 3:42 AM, Maciej Sumiński wrote:
> Hi Wayne,
>
> I see the following options, both start with forcing ngspice to ignore the default spinit file (e.g. by setting SPICE_SCRIPTS env var to a not existing directory):
> - Manually issue commands to load .cm files and enable nomoremode (having it disabled apparently can freeze the simulator and gives no benefits here). This way we can have a list of directories to be checked and pick the one which in fact contains .cm files.
> - Load the spinit file in KiCad and execute the stored commands with adjusted paths.
>
> Alternatively, we can override the SPICE_SCRIPTS env variable to contain
> a relative path to spinit, and make it contain relative paths as well.
> It will solve the problem for KiCad installers/packagers, but not for
> the others.
>

[Jonatan Liljedahl (kymatica)]

Also on OSX the default spinit file works fine, except that 'nomoremode' must be set. But that could be done from kicad automatically (add "pre_set nomoremode" command before running sim).

However, as you know, the default spinit file (or the other files) or not found automatically on OSX, probably because the bundled libngspice has some other path hardcoded in?

[Wayne Stambaugh (stambaughw)]

On 10/6/2016 1:10 PM, Jonatan Liljedahl wrote:
> Also on OSX the default spinit file works fine, except that 'nomoremode'
> must be set. But that could be done from kicad automatically (add
> "pre_set nomoremode" command before running sim).

I would prefer users not have to set this in their simulation files.
This is something that should work correctly with a normal install.
Customizing ngspice config settings in a simulation file is fine for
advanced uses who know what they are doing.

>
> However, as you know, the default spinit file (or the other files) or
> not found automatically on OSX, probably because the bundled libngspice
> has some other path hardcoded in?
>

I don't know if the osx package manager has any tools for creating text
files at install time if the install path is changed by the user. I'm
not sure if sed is installed on osx by default. If it is, the osx
install could run a post install script to update the spinit file
accordingly.

[Jonatan Liljedahl (kymatica)]

Exactly, 'nomoremode' should not be optional, it must be hard coded. For example by sending "pre_set nomoremode" command before sending the input circuit to spice.

Regarding packaging and paths, I think there are two options:

1) Distribute the full ngspice inside the kicad app bundle, including a default spinit file etc. Then tell ngspice the location of this. Also make sure it reads ~/.spiceinit

2) Require users to install ngspice separately. Add a preferences UI in kicad to tell it the location. This could have sane default values depending on platform (/Applications/ngspice/share/ngspice/ on OSX). Also make sure it reads ~/.spiceinit

I haven't looked at libngspice API so I don't know how to tell ngspice about this location (except setting the SPICE_LIB_DIR env var).

One potential issue with option 2 on OSX is that the bundled libngspice might be from a different version than the user-installed ngspice distribution. But I guess it should be compatible unless they make some changes in syntax and codemodel format, etc.. so probably not a problem?

[Jonatan Liljedahl (kymatica)]

Oh, forgot to add:

For option 1, it's important that the paths in the spinit file are relative. I don't know if ngspice supports relative paths for codemodels in its spinit file. (An app bundle on OSX can be moved around freely by the user.)

I think the best is probably option 2, require the user to install ngspice. It could even allow to open the download page URL for ngspice if user tries to run simulator without it finding ngspice.

[Jonatan Liljedahl (kymatica)]

Hi Wayne, I saw your post in the dev mailing list:

> Placing a spinit file in the path where the ngspice and libngspice
> binaries reside works with no need to set any environment variables.
>
> Option 2 could be used by the installer. The installer itself would
> have to create the spinit file by substituting the install path for the
> path of the .cm files. I'm not sure if this would work on osx. Maybe
> one of our osx devs could test this. If it does, than that would
> resolve the simulation init issues.

On OSX, the bundled libngspice resides in /Applications/Kicad/kicad.app/Contents/Frameworks/

I tried copying spinit there, but it didn't work - it was not used.

The bundled libngspice has a hardcoded path in it:
/Users/kicad/homebrew/Cellar/libngspice/26/share/ngspice

Which of course works fine on the build machine, but not for the end user unless they also installed ngspice via homebrew.

Would it work to simply let kicad set SPICE_LIB_DIR (using putenv()) and libngspice would see that?

[Holger Vogt (h+vogt)]

Hello KiCad developers,

I have uploaded a new branch sh-kicad to the ngspice git repository. It is based on the actual master branch, adding some features to introduce relative paths in shared ngspice compilation.

With mingw there is a new configure flag --enable-relpath enabling relative paths.

For this compilation there is now the following (relative to bin) directory structure established:
bin
   ngspice.dll
lib
   ngspice
      analog.cm
      digital.cm
      ...
share
   ngspice
      scripts
         spinit

There is a script compile_min_shared.sh available that shows the compile and install commands. Installing still requires an absolut path. In the example a direct installation is made into the Windows install paths for KiCad, if run with admin priviledges. There are however some redundant files installed not needed for KiCad. I have tested the script under Windows 7, TDM-MINGW and MSYS2.

Compilation with MS Visual Studio 2015 Community (my preferred method, resulting in the fastest executable) is possible as well, however no installation is provided and thus has to be done manually.

I have no OSX, so no tests available. Not tested with LINUX as well, but there should be no influence by the changes made.

Please check if this is a solution to the KiCad install procedures.

Best regards

Holger

[Holger Vogt (h+vogt)]

Concerning .spinit:

ngspice is looking for it in the current directory (should be /bin in the Windows install) and in the directory with the "HOME" environmental variable.

On my professional Windows system I do not have HOME, but HOMEDRIVE and LOCALAPPDATA. LINUX has HOME, I guess, what about OSX and others?

I could add search paths if we agree on some common directories.

Holger

[Holger Vogt (h+vogt)]

just a correction:

Concernimg .spiceinit of course

Holger

[Jonatan Liljedahl (kymatica)]

Hi Holger,

OSX has "HOME" env variable as well. However, it's also common to use
~/Library/Application Support/TheAppName/ as a folder for app-specific
stuff. For example, Kicad is supposed to look in ~/Library/Application
Support/kicad/scripting/plugins/ for python footprint wizards (but
some other bug make it not see it currently).

Perhaps it makes sense to use ${HOME}/.spiceinit for ngspice, since
that's how it works on other platforms.

On Thu, Oct 13, 2016 at 1:36 PM, Holger Vogt <email address hidden> wrote:
> just a correction:
>
> Concernimg .spiceinit of course
>
> Holger
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1630676
>
> Title:
> Simulator freezes until return is pressed in terminal
>
> Status in KiCad:
> New
>
> Bug description:
> Most often (but sometimes not), running the simulator freezes the app,
> until one switches to the terminal where kicad was started[1] and
> press the return key a number of times, then the simulation completes.
> There's no errors or Y/n questions in the simulator log or in the
> terminal.
>
> [1] - At least on OSX, kicad must be started from the terminal with
> proper environment variable set to make it find the needed ngspice
> init file and codemodels.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/kicad/+bug/1630676/+subscriptions

--
/Jonatan
http://kymatica.com

[Wayne Stambaugh (stambaughw)]

While I applaud your effort, I'm not thrilled with the prospect of using
the development version of ngspice let alone using a third party repo of
a the development version of ngspice with custom changes. This is not
something I can endorse. As I have already explained, the issue can be
resolved by copying a properly constructed spinit file in
${install_path}/bin on windows. I still think this is a packaging
configuration issue. My guess it that a similar concept would work on
osx but I don't have access to an osx system to confirm this. I haven't
seen any feedback from the osx devs on this. I'm leaving this fix for
this up to developer who made ngspice a dependency. He knows who he is. ;)

On 10/13/2016 9:16 AM, Jonatan Liljedahl wrote:
> Hi Holger,
>
> OSX has "HOME" env variable as well. However, it's also common to use
> ~/Library/Application Support/TheAppName/ as a folder for app-specific
> stuff. For example, Kicad is supposed to look in ~/Library/Application
> Support/kicad/scripting/plugins/ for python footprint wizards (but
> some other bug make it not see it currently).
>
> Perhaps it makes sense to use ${HOME}/.spiceinit for ngspice, since
> that's how it works on other platforms.
>
> On Thu, Oct 13, 2016 at 1:36 PM, Holger Vogt <email address hidden> wrote:
>> just a correction:
>>
>> Concernimg .spiceinit of course
>>
>> Holger
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1630676
>>
>> Title:
>> Simulator freezes until return is pressed in terminal
>>
>> Status in KiCad:
>> New
>>
>> Bug description:
>> Most often (but sometimes not), running the simulator freezes the app,
>> until one switches to the terminal where kicad was started[1] and
>> press the return key a number of times, then the simulation completes.
>> There's no errors or Y/n questions in the simulator log or in the
>> terminal.
>>
>> [1] - At least on OSX, kicad must be started from the terminal with
>> proper environment variable set to make it find the needed ngspice
>> init file and codemodels.
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/kicad/+bug/1630676/+subscriptions
>
>

[Holger Vogt (h+vogt)]

Wayne,

to introduce myself:

I am ngspice maintainer for more than 15 years now.

I have created shared ngspice.

Your activities seem to be the first serious effort using this development. I appreciate this very much, because it may open up a large new user group for ngspice.

I am interested in learning about proper usage of ngspice.dll and improve it if suitable.

Please let me know if any support is required.

Best regards

Holger

[Maciej Suminski (orsonmmz)]

Hi Holger,

We really appreciate your assistance here and your efforts on providing ngspice as a shared library. Now we have implemented a few workarounds to fix all the problems we have faced using libngspice, but perhaps you have other ideas how to solve the problems in a more efficient way. This is what we fixed:

- Ngspice in certain situations (IIRC an error during param value evaluation) asked user for the input, but there was no way to answer, so in the end the whole application hung up. It is fixed in the git repository (and requires unsetting interactive mode), though it is not available in the current stable release, hence we use a script to build the right version [1].
- There is a problem with codemodel libraries on platforms other than Linux. Ngspice during the configuration step hardcodes the path to spinit file directory, which works fine for Linux packages. For Windows/msys2, it stores the absolute unix-style path (/path/to/spinit), but it is not possible to open a path like this in Windows environment (even when running in mingw shell). Due to this, we need to load spinit files by hand.
- The other consequence of above is that spinit contains the absolute unix-style paths to codemodel libraries, so even we load the spinit file manually, it still points to a file that cannot be loaded due to the path style.

The two points above currently have a workaround that is likely to be applied soon in KiCad [2]. I do not blame ngspice here, as I do not have any ideas how to solve the problem in an elegant way, neither on KiCad nor ngspice side. Perhaps relative paths in spinit file or using a ngspice variable, as we did in the described workaround [2]?

Regarding the shared library, I think it would be helpful if there was a versioning scheme for the library. Currently it is built as libngspice.so.0.0.0 or libngspice-0.dll. One day the interface may change, but there will be no way to determine the library version used in the system.

Regards,
Orson

1. https://orson.net.pl/pub/libngspice/get_libngspice_so.sh
2. https://lists.launchpad.net/kicad-developers/msg26647.html

[Wayne Stambaugh (stambaughw)]

Holger,

Thank you for supporting KiCad. One of our goals with the introduction
of simulation in KiCad is to match the feature set of LTSpice. I know
that is a pretty ambitious goal but it's something a lot of our users
have been asking for. It's good to know we have some support from the
ngspice maintainer. I wish we had this kind of support from our other
dependency library maintainers. Orson is the developer who wrote the
simulator so I appreciate you working with him to help improve our
simulator.

I discovered the configuration issue on windows. Currently when on
building ngspice on windows with msys2, the system spinit file is not
read from its default location (${install_path}/share/ngspice/scripts)
and even if it did, the paths in the default spinit file are unix paths
not windows paths so the .cm files cannot be found. I resolved this
issue by copying a modified spinit file with the correct windows paths
to the same folder where the libngspice.dll file resides and that fixed
the issue. The problem is this requires the user to configure this by
hand which I would rather they didn't have to. Any help in resolving
this would be helpful.

One other minor off topic issue is that by default, ngspice installs the
dev header files in $prefix/share/ngspice/include/ngspcine instead of
$prefix/include/ngspice so packagers have to patch the header
Makefile.am file to install the headers in the proper folder. Fixing
that would be helpful as well.

I've only just started converting some of my ltspice simulations over to
KiCad. So far it's mostly just test fixtures for op-amps but hopefully
of the next few months I will convert so of my more complex simulations
to start pushing our simulator capabilities.

Thank you for efforts with the ngspice project.

Cheers,

Wayne

On 10/14/2016 2:29 AM, Holger Vogt wrote:
> Wayne,
>
> to introduce myself:
>
> I am ngspice maintainer for more than 15 years now.
>
> I have created shared ngspice.
>
> Your activities seem to be the first serious effort using this
> development. I appreciate this very much, because it may open up a large
> new user group for ngspice.
>
> I am interested in learning about proper usage of ngspice.dll and
> improve it if suitable.
>
> Please let me know if any support is required.
>
> Best regards
>
> Holger
>

[Holger Vogt (h+vogt)]

Concerning .spiceinit, a correction:
In actual release 26 of shared ngspice .spiceinit is only looked for in the 'current directory'.

Holger

[Holger Vogt (h+vogt)]

Concerning ngspice release 27, to come soon, some action items connected
with KiCad:

install dev headers
status:
install path $prefix/include/ngspice
ngspice repository, master branch

library versioning
status: t.b.d.

user input required
status:
user input only upon having variable 'interactive' set.
variable 'nomoremode' replaced by 'moremode', more mode is deactivated by default.
ngspice repository, master branch

spinit
status:
with the additional configure flag --enable-relpath
installation in a path relative to ngspice library (located in bin directory).
code models read from a path relative to ngspice library
(see my mail from Oct. 13).
variable 'interactive' only set if ngspice executable is used (not set with ngspice library).
Without this flag: satndard behavior as used for LINUX.
ngspice repository, sh-kicad branch

.spiceinit
status
search paths now are
current directory, HOME environment, USERPROFILE environment
ngspice repository, sh-kicad branch

Tested with TDM MINGW and MSYS2
Please give the additions in sh-kicad a try and tell me, if o.k..

Holger

[Holger Vogt (h+vogt)]

library versioning
status:
libtool versioning system installed
Input is via configure flag LIB_VERSION=current:revision:age
ngspice repository, sh-kicad branch

I just have to understand the versioning policy :)

Holger

[Maciej Suminski (orsonmmz)]

Hi Holger,

I still need to test the new branch more, but I am already grateful for your changes.

Please do not consider me capricious, but what do you think about following the ngspice version? So the current library would be v26, and the next ngspice release v27? Just an idea to consider.

Previously I have misunderstood the --enable-relpath switch, but it might be a good idea. We still need to find out the shared library path and use it as the current working directory. One idea is dladdr(), but it is not very portable.

Also, we still need to convince ngspice to read the right spinit file, though using an environmental variable SPICE_SCRIPTS would work here. Perhaps it could also try a relative path when looking for the spinit file.

Once again, I am truly thankful for your support.

[Holger Vogt (h+vogt)]

An update:

spinit

ngspice.dll will find its own location (using dladdr() or the corresponding Windows API function), if --enable-relpath is set in ./configure.

spinit is searched for in ../share/ngspice/scripts/ relative to this location.

A relative path is enterd into spinit to find the code models in a path ../lib/ngspice, if --enable-relpath is set in ./configure, independently from the prefix="some_directory".
However this search path is relative to the calling program (eeschema.exe), not relative to ngspice.dll. If both eeschema.exe and ngspice.dll are in the same directory (e.g. in KiCad/bin as in the nightly Windows builds), then everything is ok..

The changes are established in branch sh-kicad, but also in the main development branch scope-inpcom-7.

Holger

[Maciej Suminski (orsonmmz)]

Hi Holger,

It would be great if your changes were available in ngspice v27. It seems that all the problems are solved now (or at least I am not aware of any others). Thank you for your help, it is the first time when I see library developers lend a helping hand to a project. We really appreciate it.

Regards,
Orson

[Holger Vogt (h+vogt)]

ngspice-27 is finally available.

What we have for the dll:

No user input required (Variables moremode, interactive are deactivated by default).

--enable-relpath will yield a relative path for spinit and the code models (relative to the caller, not the dll).

Search path for .spiceinit (the user configurable file) is current directory, HOME environment, USERPROFILE environment.

Install path for headers

What we do not have (yet)
Versioning

Holger

[Maciej Suminski (orsonmmz)]

Hi Holger,

Thank you for the update, I look forward to testing v27. I think it will take some time for the major distros to update the packages, so for now we will keep the workarounds. Dynamic library versioning would really help us to decide whether we need to apply the workarounds, but I understand there are always too many things to be done in every project.

I appreciate your support, I rarely observe such high level of commitment.

[Holger Vogt (h+vogt)]

ngspice-27 is finally available.

What we have for the dll:

No user input required (Variables moremode, interactive are deactivated by default).

--enable-relpath will yield a relative path for spinit and the code models (relative to the caller, not the dll).

Search path for .spiceinit (the user configurable file) is current directory, HOME environment, USERPROFILE environment.

Install path for headers

What we do not have (yet)
Versioning

Holger

[Nick Østergaard (nickoe)]

What is the status of this bug? Any actions outstanding?

[Jonatan Liljedahl (kymatica)]

I think this is all well and fixed now? At least with latest Kicad official release, "it just works" :)

[Wayne Stambaugh (stambaughw)]

It seem like we can set this to fix release. Please change it back as required.