eeschema loading symbol libraries everytime

[Radovan Blažek]

Every time you want to place a component, the symbol libraries are loading, I guess they should load only once. With all libraries active it takes a few seconds.

Application: kicad
Version: (2017-11-27 revision 0acdd9f02)-master, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.56.1 OpenSSL/1.1.0g zlib/1.2.11 libpsl/0.18.0 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.27.0
Platform: Linux 4.13.12-1-ARCH x86_64, 64 bit, Little endian, wxGTK
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
    Boost: 1.65.1
    Curl: 7.56.0
    Compiler: GCC 7.2.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Wayne Stambaugh (stambaughw)]

Unless someone broke something, the symbol libraries do not get loaded
every time the chooser dialog is opened. The issue is repopulating and
redrawing the symbol library tree control each time the symbol chooser
dialog is opened. The new symbol library table includes all of the
available symbol libraries of which there are over 90 where as the old
symbol library list only included a minimal number of about 15
libraries. If the old list had included all of the same libraries, my
guess is that you would have seen the same issue. I'm not sure if there
is a good solution for this other than removing unnecessary libraries
from the symbol library table.

On 11/27/2017 4:58 PM, Radovan Blažek wrote:
> Public bug reported:
>
> Every time you want to place a component, the symbol libraries are
> loading, I guess they should load only once. With all libraries active
> it takes a few seconds.
>
> Application: kicad
> Version: (2017-11-27 revision 0acdd9f02)-master, release build
> Libraries:
> wxWidgets 3.0.3
> libcurl/7.56.1 OpenSSL/1.1.0g zlib/1.2.11 libpsl/0.18.0 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.27.0
> Platform: Linux 4.13.12-1-ARCH x86_64, 64 bit, Little endian, wxGTK
> Build Info:
> wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
> Boost: 1.65.1
> Curl: 7.56.0
> Compiler: GCC 7.2.0 with C++ ABI 1011
>
> Build settings:
> USE_WX_GRAPHICS_CONTEXT=OFF
> USE_WX_OVERLAY=OFF
> KICAD_SCRIPTING=ON
> KICAD_SCRIPTING_MODULES=ON
> KICAD_SCRIPTING_WXPYTHON=OFF
> KICAD_SCRIPTING_ACTION_MENU=ON
> BUILD_GITHUB_PLUGIN=ON
> KICAD_USE_OCE=ON
> KICAD_SPICE=ON
>
> ** Affects: kicad
> Importance: Undecided
> Status: New
>
>
> ** Tags: eeshema
>

[Radovan Blažek (blazra)]

I have disabled most of the libraries and left active only 9 and it's still slow as you can see from the screen recording in the attachment. The second loading doesn't take as much time as the first one, but it's not a huge difference. Also the filtering is really slow - I don't know if that's somehow connected.

[Patrik Bachan (xorly)]

I can confirm change of behavior in symbol adding to schematic.
Version 7e7d3004f
- first time, symbol tree dialog open takes some time
- next time dialog pops up almost immediately (below 500ms)

In version 48a37c299
- first time, symbol tree dialog open takes some time
- next time dialog pops up with significant delay (~6s), it takes suspiciously similar time as first dialog open

Progress bar of library loading was added between those 2 versions, so maybe library reload happened with older version too. This is still rather fast. Once the progress bar disappears, nothing (visually) happens for next several seconds.

Application: kicad
Version: (2017-11-12 revision 7e7d3004f)-master, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.56.1 OpenSSL/1.1.0g zlib/1.2.11 libpsl/0.18.0 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.27.0
Platform: Linux 4.13.11-1-ARCH x86_64, 64 bit, Little endian, wxGTK
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
    Boost: 1.65.1
    Curl: 7.56.1
    Compiler: GCC 7.2.0 with C++ ABI 1011
Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=OFF
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=OFF

Version: (2017-11-26 revision 48a37c299)-master, release build
rest is same

[Patrik Bachan (xorly)]

891cf783fbef33af27fa82e034fe7bb26e39ee72
- last working commit with fast schematic symbol tree dialog pop up

db4bd0c2db504323580e680973a50085c04fa989
- next commit, still loads fast, but tree is empty

e0e4e5f1be647e950b7ef865306d6a9e17c6609c
- next commit, problem started, repeated loading is slow as first one

[Maciej Suminski (orsonmmz)]

I have just checked and cannot reproduce the problem. It takes some time to load the libraries when you open either the symbol editor or the component selector the first time, but later they are not loaded again.

Just to be sure that we are seeing the same thing, please add many libraries to the symbol library table and compare the load times.

[Maciej Suminski (orsonmmz)]

Video in message #2 gives me a hint. Most likely it takes time to populate the component selector widget tree, not to reload the libraries. I will have a look at that.

[Radovan Blažek (blazra)]

With all 100 libraries active it takes 10 seconds the first time and 8 seconds the second time (SSD and 1st gen. i5). The loading progress bar is actually pretty fast, most of the time is spent after the loading progress bar had gone to 100%.

[Patrik Bachan (xorly)]

When filtering is triggered (character changed in search box), it takes approximately same time as that delay after progress bar hits 100%. Even CPU load seems to be same.

891cf783 has blazing fast filtering. Two commits later, in e0e4e5f1, filtering time increased to ~6s on my 3rd gen i7.

[Wayne Stambaugh (stambaughw)]

I don't know if it's related to this change or not but now the symbol
library tree is sorted from Z-A instead of A-Z in the symbol chooser dialog.

On 11/29/2017 4:14 AM, Maciej Suminski wrote:
> Video in message #2 gives me a hint. Most likely it takes time to
> populate the component selector widget tree, not to reload the
> libraries. I will have a look at that.
>

[Oivind Toien (otoien)]

I really hope the sorting will be fixed back to A..Z and not permanently reversed. When personal libraries have been named in a way to get to the top of the list, it gets troublesome when it then ends up at the bottom. It is also an old issue that it is alphabetized as per the ASCII table, not on uppercase characters. (The KiCad libraries do not seem to be consequent here yet. The human brain does not make that distinction as easy as computers so it gets confusing.)

[Oivind Toien (otoien)]

I forgot to add that tested with the Nov 25 nightly under Win7.

[Maciej Suminski (orsonmmz)]

I attach a possible fix. I do not dare to push it to the master branch, as I might have no Internet access during the weekend and I could not handle problems if any arise.

[Maciej Suminski (orsonmmz)]

I attach a possible fix. I do not dare to push it to the master branch, as I might have no Internet access during the weekend and I could not handle problems if any arise.

[Radovan Blažek (blazra)]

Awesome! Works a treat for me. First time loading is still with the delay after the progress bar finishes, but second time there is no delay and loading takes only like a second with all libraries enabled. Also filtering is instantaneous. Thanks for the fix Maciej.

[Wayne Stambaugh (stambaughw)]

This seems to work for me and thankfully the sort ordering has been
restored as well. Thanks.

On 12/01/2017 10:41 AM, Maciej Suminski wrote:
> I attach a possible fix. I do not dare to push it to the master branch,
> as I might have no Internet access during the weekend and I could not
> handle problems if any arise.
>
> ** Patch added: "0001-Restoring-the-previous-performance-of-COMPONENT_TREE.patch"
> https://bugs.launchpad.net/kicad/+bug/1734773/+attachment/5017093/+files/0001-Restoring-the-previous-performance-of-COMPONENT_TREE.patch
>

[KiCad Janitor (kicad-janitor)]

Fixed in revision 45bf919923272a814be5cb5b4a245950d8bc6706
https://git.launchpad.net/kicad/patch/?id=45bf919923272a814be5cb5b4a245950d8bc6706

[Bogdan Șerban (moonfire-bogdan)]

I can confirm this issue is still present in the latest nightly for Windows.

[Patrick EGLOFF (tk5ep)]

Same for me. Unactivating libraries does'nt change anything, all libraries are still loaded each time.

[Oivind Toien (otoien)]

There are two distinct phases here as seen with Win64 nightly 2017-12-27 revision fab6fc69e under Win7:

1. The progress bar is on for 8-10 seconds the first time. Next times it will only be present for less than 1/2 of that time (could be due to disk caching?).

2. After the progress bar is finished the first time there is a delay of 45-60 seconds where the equivalent of an hourglass is shown and KiCad otherwise is unresponsive. This delay in not present the next times; after the progress bar one can immediately continue.

[Nick Østergaard (nickoe)]

I guess this is related to the discussion in https://<email address hidden>/msg23266.html

[Oivind Toien (otoien)]

Also related to the new discussion here: https://lists.launchpad.net/kicad-developers/msg32632.html

An added note: Delay #2 I reference above seems to depend on the number of drives connected to the computer and their responsiveness. I removed my computer from the dock with 3 external portable drives connected, to only have system drive and internal D drive (a regular spinning drive where all my libraries are located, only system drive with software is SSD) connected and the delay in #2 went down to < 10 seconds. I think a lot of the reason for the delay might also be due to the need for those external drives to wake up from sleep mode after an inactive period, as when the computer were back in the dock with freshly spinning drives the delay in #2 was still <10 seconds. The progress bar delay in #1 above was not affected by these changes, either for first time use or the shorter following use.

Reports on these delays could be affected by type of drives, those with libraries located to SSD's might have much faster response time, so perhaps best to report this information too.

[Patrick EGLOFF (tk5ep)]

@Oivind

No external devices here.

There is something really weird.

1st time :
It takes about 10 s to load. A window showing all librairies being loaded. The footprints are ready.

2nd time :
It takes about the same time to load, BUT the footprints are not available and a green progress bar + sort of hourglass takes even more time to load the footprints !!

3rd time and next :
About 7s loading time and footprints available immediately...

I tried to desactivate some librairies, but that doesn't change anything. For what is this option good for ?

Application: kicad
Version: (2017-12-27 revision fab6fc69e)-makepkg, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.60.0
    Curl: 7.54.1
    Compiler: GCC 7.1.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Oivind Toien (otoien)]

Further comment - new tests: If I deactivate libraries either by using a different sym-lib-table or by setting the libraries to inactive loading so that I only have my custom library + the official power library active, after restarting KiCad the progress bar (#1 above) is very much faster, <1 sec, while the first time load delay (#2) is about 7 sec. After reversing the changes both #1 and #2 are back to about 7 seconds. (I normally have all the official libraries + Walter Lain's libraries, a total of 132, active out of old habit, so quite a few.)

Thus I cannot confirm that loading for #1 (time of progress bar) is independent of the number of libraries loaded, quite to the contrary, while the delay after the progress bar the first time (#2) seems more independent of the number of active libraries and uncertain what causes the variation. (2017-12-28 revision 4f1b51075).

[Wayne Stambaugh (stambaughw)]

I just pushed a fix to only show the progress dialog the first time the library is loaded. I confirmed that the libraries are indeed only loaded the first time. All subsequent library loads are cached as expected unless the library file is changed so the performance it may be in the creation of the symbol choose dialog symbol tree. I suppose we could cache that as well and only update it as require but I don't think that will be trivial to implement.

[Wayne Stambaugh (stambaughw)]

On 12/29/2017 10:34 AM, Wayne Stambaugh wrote:
> I just pushed a fix to only show the progress dialog the first time the
> library is loaded. I confirmed that the libraries are indeed only
> loaded the first time. All subsequent library loads are cached as
> expected unless the library file is changed so the performance it may be
                                                                ^^issue
> in the creation of the symbol choose dialog symbol tree. I suppose we
> could cache that as well and only update it as require but I don't think
> that will be trivial to implement.
>

[Oivind Toien (otoien)]

Thanks, I can also confirm that the progress bar is not seen the second and following times. There is then about a 2-3 second delay before the choose symbol dialog shows up (when all my 132 libraries are activated).

For the record, the variation in the delay #2 I describe above for the first time load might be due to disk caching. After a fresh cold boot of the computer, delay #2 is about 45 sec (as initially described). After shutting down KiCad, restarting KiCad and loading Eeschema again, delay #2 is down to 7 sec.

[Wayne Stambaugh (stambaughw)]

I have around 95 libraries and the chooser appears in less than 1 second
after the initial load so this just may be your system that is the
issue. We could cache the tree but keeping it synced with library
changes is going to be problematic. I'm not sure what else we can do to
speed this up.

On 12/30/2017 06:03 AM, Oivind Toien wrote:
> Thanks, I can also confirm that the progress bar is not seen the second
> and following times. There is then about a 2-3 second delay before the
> choose symbol dialog shows up (when all my 132 libraries are activated).
>
> For the record, the variation in the delay #2 I describe above for the
> first time load might be due to disk caching. After a fresh cold boot of
> the computer, delay #2 is about 45 sec (as initially described). After
> shutting down KiCad, restarting KiCad and loading Eeschema again, delay
> #2 is down to 7 sec.
>

[Oivind Toien (otoien)]

I am really not complaining, just reporting, the delay on repeat insert symbol improved from 4-5 sec down to 2-3 sec when the progress bar display was removed, that is a good improvement.
(The long delay #2. after a cold reboot on first time load [I have confirmed this is repeatable on new boots] is not that problematic either, not often one do a cold reboot).

[Patrick EGLOFF (tk5ep)]

Hi,

The last build (31-12-2017) does not show the loading progress bar anymore, but it didn't change anything in the load speed.
The cursor is frozen while it loads the second time without showing the progress bar.

I can't say when this all apperead, but it worked perfectly before that and i had no hardware or configuration change...

I hope this helps.

[Chris Pavlina (pavlina-chris)]

There's a good chance that this delay is in loading footprints, not symbols. The footprint selection/preview has some known performance issues that aren't going to be fixed in time for 5.0, so I'm going to push soon a new option allowing it to be enabled or disabled (and it will default disabled). That should be in the next couple days. When I do that, please test and see if it's faster with footprints disabled.

[Wayne Stambaugh (stambaughw)]

It's definitely faster disabled. I would think the speed of the user's
computer (and internet connection if they are using the github plugin)
makes a big difference in how usable this feature is. I think the
disable option was a good idea for users with slower systems. Thanks
for the fix!

On 01/03/2018 05:04 PM, Chris Pavlina wrote:
> There's a good chance that this delay is in loading footprints, not
> symbols. The footprint selection/preview has some known performance
> issues that aren't going to be fixed in time for 5.0, so I'm going to
> push soon a new option allowing it to be enabled or disabled (and it
> will default disabled). That should be in the next couple days. When I
> do that, please test and see if it's faster with footprints disabled.
>

[Oivind Toien (otoien)]

Recalling what I wrote above for clarity,
"There are two distinct phases here as seen with Win64 nightly 2017-12-27 revision fab6fc69e under Win7:
1. The progress bar is on for 8-10 seconds the first time. Next times it will only be present for less than 1/2 of that time (could be due to disk caching?).
2. After the progress bar is finished the first time there is a delay of 45-60 seconds where the equivalent of an hourglass is shown and KiCad otherwise is unresponsive. This delay in not present the next times; after the progress bar one can immediately continue."

After the fix, without showing footprint preview in Windows nightly (2018-01-04 revision 658d181ec):

Delay #2 at first time use (which previously was quite variable, longer after a fresh reboot) is completely gone - a great improvement - so this delay seems to have been related to the footprint loading. [A note regarding github plugin: I have KIGITHUB set to the same local path as KISYSMOD to avoid going on the net. It cannot be blank; perhaps it would be better to set it to an empty folder? Could it have caused repeated footprint searches?]

Delay #1 which had already decreased to 4-5 sec for first time load and 2-3 seconds for the following loads now still remains at 4-5 sec for first time load and 2-3 seconds for following loads (the latter without progress bar showing). While the apparent freeze is slightly disconcerting, it is not too bothersome once one get used to it. As mentioned above I have 132 libraries loaded which might be above average. (I was really bothered, I could consider upgrading my data drive to an SSD.)

[Patrick EGLOFF (tk5ep)]

I uploaded the last nightly (05-01-2018).
No change for me... If there is an option somewhere to disable the footprint loading, where is it ?
I only noticed that the green loading bar in the library table disappeared...

When i click to add a new symbol, the Kicad cursor is frozen for about 6-7 seconds. And that at each time ...

Other strange behavoir, if you search for CP (polarized capacitor), the search selector lands on the W5100 Ethernet controler ... The CP symbols are on top of the list.

Thanks for your efforts,

[Nick Østergaard (nickoe)]

@Patrick, is the footprint oreview enabled?

[Chris Pavlina (pavlina-chris)]

I was just looking at this with someone in IRC, and it is definitely not fixed. Tracked it down via debugger into CMP_TREE_MODEL_ADAPTER_BASE::AddLibrariesWithProgress() - it's loading them all every time, freezing for a good sevenish seconds every time.

[Jon Evans (craftyjon)]

Chris, I just tried this as of 917804ef123293be1b93f7fe1a5f2f049b42445c and don't see this behavior. The first time I get the progress bar and a delay, and subsequent times I try to place a component, the chooser comes up instantly. (on Linux, with or without footprint preview turned on)

[Seth Hillbrand (sethh)]

This might be two different cases. The Symbol Browser is calling AddLibrariesWithProgress each time. Adding a component from Eeschema follows a different code path and doesn't.

I have a couple of patches I'm testing to address these issues. If no one has objections (don't want to step on others progress) I'll take this bug.

-S

[Chris Pavlina (pavlina-chris)]

J _only_ see this on Windows. I don't know if it actually doesn't reload the libs on Linux somehow (though I can't fathom how) or if it's just faster due to different IO caching behavior or similar...

[Seth Hillbrand (sethh)]

Chris-

Are you seeing this 7 seconds each time you add a symbol in Eeschema? Or just each time you open the symbol library editor?

[Chris Pavlina (pavlina-chris)]

Every time I open the symbol chooser from eeschema, same as you can see from the video posted in #2 except now the progress bar doesn't appear, it just freezes.

[Seth Hillbrand (sethh)]

Thanks. Which Windows do you use?

[Chris Pavlina (pavlina-chris)]

Windows 10 64-bit.

[Patrick EGLOFF (tk5ep)]

@Nick,

No, footprint was desactivated by default. Loading time is worse when activated.

Also using Windows 10 64 bit.

Will try on a 32bit tomorrow at work.

[Wayne Stambaugh (stambaughw)]

Chris,

I tested this with a debugger on windows and the library cache was not
being flushed and reloaded on 64 bit windows 7 pro. I'm not sure what
is going on here but AFAICT the library is not being reloaded from the
drive. Is it possible that building the component tree model is the
time sink culprit on windows? Seven seconds seems excessive, even on
windows which is known to lag for reasons that I cannot quite fathom.

Cheers,

Wayne

On 01/06/2018 09:46 PM, Chris Pavlina wrote:
> J _only_ see this on Windows. I don't know if it actually doesn't reload
> the libs on Linux somehow (though I can't fathom how) or if it's just
> faster due to different IO caching behavior or similar...
>

[Chris Pavlina (pavlina-chris)]

I'd be surprised if that were the case considering it was fast on the exact same test setup when I wrote the code initially, but I'll look into it a bit later.

[Wayne Stambaugh (stambaughw)]

Orson made quite a few changes to support the symbol library editor. I
know he backed some of the changes out because they were killing the
component tree adapter performance but maybe he missed something. I did
ping him about a related issue with storing LIB_PART pointers in the
tree which I eliminated due for safety reasons. Before his changes, the
performance of the component tree adapter was fine.

On 01/07/2018 02:40 PM, Chris Pavlina wrote:
> I'd be surprised if that were the case considering it was fast on the
> exact same test setup when I wrote the code initially, but I'll look
> into it a bit later.
>

[Patrick EGLOFF (tk5ep)]

Hi,

Today, i reopenend a PC i didn't use for a while and on which KICAD was installed.
I have the same loading problem, and i can't remember having suffered of it last time i used Kicad !

It's a

Application: kicad
Version: (2017-11-13 revision d98fc85a8)-makepkg, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
Platform: Windows 7 (build 7601, Service Pack 1), 32 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.60.0
    Curl: 7.54.1
    Compiler: GCC 7.1.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Patrick EGLOFF (tk5ep)]

oops ! A wrong key combination and the post has been sent before i wanted...

It's a "rather" old Kicad nightly and running on 32 bits Windows 10.

Can this problem be related to something else than the local install ? Github ?

[Patrick EGLOFF (tk5ep)]

Hi,
No progress regarding this bug... The last nightly are still suffering.
It's almost unusable and i think to downgrade to version 4.07...

I'm i the only one to suffer ... ?

[Nick Østergaard (nickoe)]

@Patrick, please add version info.

[Jeff Young (jeyjey)]

Patrick is not the only one to suffer. My version info:

Application: kicad
Version: (2018-01-22 revision 618af8738)-master, debug build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 nghttp2/1.24.0
Platform: Mac OS X (Darwin 17.3.0 x86_64), 64 bit, Little endian, wxMac
Build Info:
    wxWidgets: 3.0.4 (UTF-8,STL containers,compatible with 2.8)
    Boost: 1.66.0
    Curl: 7.57.0
    Compiler: Clang 9.0.0 with C++ ABI 1002

Build settings:
    USE_WX_GRAPHICS_CONTEXT=ON
    USE_WX_OVERLAY=ON
    KICAD_SCRIPTING=OFF
    KICAD_SCRIPTING_MODULES=OFF
    KICAD_SCRIPTING_WXPYTHON=OFF
    KICAD_SCRIPTING_ACTION_MENU=OFF
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=OFF
    KICAD_SPICE=ON

[Patrick EGLOFF (tk5ep)]

Here my version,
The 32 bits has the same problem. It's really VERY painfull to work with it ...

Application: kicad
Version: (2018-01-23 revision 975c08c46)-makepkg, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.60.0
    Curl: 7.54.1
    Compiler: GCC 7.1.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Wayne Stambaugh (stambaughw)]

Two patches ( eb7ecf1d and 2b460bc1) were just pushed to the development branch which should address most if not all of the symbol load times. Everyone who reported to this bug, please test this when you get a chance and if it resolves the problem.

[Radovan Blažek (blazra)]

The chooser dialog appears without any delay now, for me at least and after the initial load. You just click and it shows up ready.

Application: kicad
Version: (2018-01-24 revision 2b460bc1f)-master, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.57.0 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.29.0
Platform: Linux 4.14.13-1-ARCH x86_64, 64 bit, Little endian, wxGTK
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
    Boost: 1.66.0
    Curl: 7.57.0
    Compiler: GCC 7.2.1 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Oivind Toien (otoien)]

Thanks, first time load time as before (expected), after that loads instantly. So I can confirm that the fix works with the just downloaded nightly.

Application: kicad
Version: (2018-01-24 revision 7a2d9dff6)-makepkg, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
Platform: Windows 7 (build 7601, Service Pack 1), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.60.0
    Curl: 7.54.1
    Compiler: GCC 7.1.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[Patrick EGLOFF (tk5ep)]

Hi Wayne,

Everthing works fine now !
The first loading is even faster than before, and the next insertions are instantaneous.

Thanks for the great work !

[Wayne Stambaugh (stambaughw)]

@Patrick, You should thank Jeff for the majority of this speed up. My
change had minimal impact when opening the symbol chooser.

On 1/25/2018 1:18 PM, Patrick EGLOFF wrote:
> Hi Wayne,
>
> Everthing works fine now !
> The first loading is even faster than before, and the next insertions are instantaneous.
>
> Thanks for the great work !
>

[Patrick EGLOFF (tk5ep)]

OK, then i thank everybody who worked on (and solved) this nasty bug !

[Jeff Young (jeyjey)]

One last performance patch; this one lops about 1/3 of the time off of the first Place Symbol operation.

[Wayne Stambaugh (stambaughw)]

@Jeff, I merged this patch but not without some misgivings. I'm not
sure exposing LIB_PIN internals to SCH_LEGACY_PLUGIN_CACHE is a good
thing. In this case I will allow it since it presents no risk. I know
we do this elsewhere but it almost always feels like easy fix for a poor
object design. Please do not take this merge as an endorsement for this
kind of fix in the future. In the long term, we should be improving our
object designs so we don't have to expose internals to other objects.

On 1/29/2018 12:58 PM, Jeff Young wrote:
> One last performance patch; this one lops about 1/3 of the time off of
> the first Place Symbol operation.
>
> ** Patch added: "0004-More-performance-optimizations-for-symbol-libraries.patch"
> https://bugs.launchpad.net/kicad/+bug/1734773/+attachment/5045100/+files/0004-More-performance-optimizations-for-symbol-libraries.patch
>

[KiCad Janitor (kicad-janitor)]

Fixed in revision 99ad5cf394986db1e238c53de90d26560fdfefd0
https://git.launchpad.net/kicad/patch/?id=99ad5cf394986db1e238c53de90d26560fdfefd0

[Jeff Young (jeyjey)]

@Wayne, I completely agree. It's even worse that the SetXYZ() methods for the other LIB_ITEMs are low level, but the LIB_PIN ones operate at the UI level.

But the Pin stuff for multi-unit parts is so "challenged" that I wasn't about to change it. ;)