EESchema - none of the BOM python plugins work anymore

[Art]

When I try to generate BOM using any of the BOM plug-ings with KiCad several versions back everything works fine. If I use the latest version, I get the following error message:

Command error. Return code 3
Error messages:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00002340 (most recent call first):

The command I run:

python "C:\Program Files\KiCad\bin\scripting\plugins/bom_csv_grouped_by_value_with_fp.py" "C:\Users\***\file_name.xml" "C:/Users/***/file_name"

Application: KiCad
Version: (5.99.0-398-g3be1862b0), release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.66.0 OpenSSL/1.1.1d (Schannel) zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.1.1) nghttp2/1.39.2
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    Build date: Nov 20 2019 22:10:18
    wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.71.0
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.66.0
    Compiler: GCC 9.2.0 with C++ ABI 1013

Build settings:
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=OFF
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

[Ionel (ionelcompact)]

This also present in the latest stable release 5.1.5

Command error. Return code 3
Error messages:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00003ed8 (most recent call first):

Application: KiCad
Version: (5.1.5)-2, release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.66.0 OpenSSL/1.1.1d (Schannel) zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.1.1) nghttp2/1.39.2
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.71.0
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.66.0
    Compiler: GCC 9.2.0 with C++ ABI 1013

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=OFF
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

[Rene Poeschl (poeschlr)]

See also https://forum.kicad.info/t/bom-export-python-error-code-3/19665

[nico (zeroisnan)]

Providing personal experience with this issue using a fresh install
Version information at the bottom of the message

I believe the title of this issue is misleading
You can simply reproduce the error by invoking the interpreter from the command line:
/c/Program\ Files/KiCad/bin/python

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00003db8 (most recent call first):

Setting PYTHONHOME and PYTHONPATH does not resolve the issue
KICAD_HOME="/c/Program\ Files/KiCad"
export PYTHONHOME="$KICAD_HOME/lib/python2.7"
export PYTHONPATH="$PYTHONHOME:$PYTHONHOME/lib-dynload:$PYTHONHOME/site-packages"

Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00003174 (most recent call first):

Note the python version reported:
/c/Program\ Files/KiCad/bin/python --version
3.7.5

Is this expected? The site-package within KiCAD installation refers to python 2.7 instead

This issue does not only affect EEschema, it prevents any standalone usage of the pcbnew python module.

Application: Pcbnew
Version: (5.1.5)-2, release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.66.0 OpenSSL/1.1.1d (Schannel) zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.1.1) nghttp2/1.39.2
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.71.0
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.66.0
    Compiler: GCC 9.2.0 with C++ ABI 1013

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=OFF
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

Thanks,
nico

[Ian McInerney (imcinerney)]

@nickoe has anything changed in the Windows packaging recently that could cause this?

[Art (diametrix)]

@Ian

I’m not a python expert, but I don’t think it’s related to Windows. If I go back an older Kicad versions then all the scripts work fine.

[nico (zeroisnan)]

Ian was not suggesting this is related to Windows, but to 'Windows packaging', i.e. the act of pulling together all the files (including anything python related) to create the installer. The fact that things work right when you go back to previous KiCAD releases proves that something incorrect was made with 5.1.5

Can anyone at least confirm my finding on the shipped python version? From a Windows command prompt:
"C:\Program Files\KiCad\bin\python.exe" --version

The issue with EEschema can be easily worked around by pointing to an external python installation.
However I am struggling to workaround the same issue to use the pcbnew python module standalone - I can get as far as:

    import pcbnew
  File "C:\Program Files\KiCad\lib\python2.7\site-packages\pcbnew.py", line 15, in <module>
    import _pcbnew
ImportError: DLL load failed: The specified module could not be found.

Even though I have appended to the python path every single path I could query from PCBNew embedded python shell:

sys.path.append('C:\\Program Files\\KiCad\\lib\\python27.zip')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\plat-win32')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\lib-tk')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\lib-old')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\lib-dynload')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\site-packages')
sys.path.append('C:\\Program Files\\KiCad\\lib\\python2.7\\site-packages\\wx-3.0-msw')
sys.path.append('C:\\Program Files\\KiCad\\share\\kicad\\scripting')
sys.path.append('C:\\Program Files\\KiCad\\share\\kicad\\scripting\\plugins')
sys.path.append('C:\\Program Files\\KiCad\\bin')
sys.path.append('C:\\Program Files\\KiCad\\bin\\plugins\\3d')

I have a sign-off flow which performs a number of steps, some of which requires the pcbnew python module, so moving completely inside pcbnew is not an option for me.

Can the windows builder be cross checked?
Also, seeing that the whole depot is migrating to gitlab, should this issue be re-opened there?

Thanks,
nico

[Mitch Kapa (mitchkapa)]

I do not have any insight as to the cause of this problem. However I discovered a temporary workaround that has allowed me to continue using the BOM creation tool with version 5.1.5. I went back to my last installed version of KiCad (5.1.0) and copied the \bin\python.exe file to the \bin folder of the 5.1.5 and once I did that the BOM creation scripts were working for me once again. I renamed the python.exe file that installed with version 5.1.5 rather than copying over it.

Thanks,
Mitch Kapa

[Nick Østergaard (nickoe)]

A fix has already been released, use build 5.1.5_3.