pcbnew board file loading got very slow

[Oleg Endo]

Since maybe about 2 months ago, the board file loading in pcbnew got really slow. I've got some larger designs and those would open in about 10 seconds. Now it takes several minutes!

I can't share the designs. Any suggestions how narrow it down?

Application: Pcbnew
Version: 5.99.0-unknown-6179dfd~86~ubuntu18.04.1, release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Platform: Linux 4.15.0-64-generic x86_64, 64 bit, Little endian, wxGTK
Build Info:
    Build date: Oct 2 2019 11:48:33
    wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8) GTK+ 3.22
    Boost: 1.65.1
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.58.0
    Compiler: GCC 7.4.0 with C++ ABI 1011

Build settings:
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

[Nick Østergaard (nickoe)]

Are you loading the file form SSD or a network drive?

How big is the file?

[Oleg Endo (oleg.endo)]

It's loading from an SSD. kicad_pcb file size is about 20 MByte. It's clearly CPU bound in some way because CPU load is 100% (all cores used).

[Seth Hillbrand (sethh)]

If you are unable to share the design, perhaps you can bisect the issue to determine a smaller commit range where the issue first occurs. My tests on the most complex board I have do not show substantial slow down on loading currently.

Two other options:
- If you can share the design privately, my e-mail address is available on my user page and I can take a look at it offline.
- Alternatively, there are some commercial KiCad support options that provide bug fixes for cases like this where designs may need non-disclosure agreements with a specific entity.

[Oleg Endo (oleg.endo)]

I have bisected it.

809a4e3c0e2cf528a3bf4e781f23726bf9129030 is the first bad commit
commit 809a4e3c0e2cf528a3bf4e781f23726bf9129030
Author: Jeff Young <email address hidden>
Date: Fri Aug 23 17:43:34 2019 +0100

    Improve connectivity tests for pads.

    We now test N, S, E and W as well as center.

    Fixes: lp:1840721
    * https://bugs.launchpad.net/kicad/+bug/1840721

The board design that I used for this test has
100078 pads, 3341 vias, 39500 track segments, 9835 nodes, 2025 nets. Board area is about 200 x 300 mm.

[Oleg Endo (oleg.endo)]

I poked around in the code around the problematic commit just out of curiosity...

I've spotted a few nits which could have impacts on performance. For example, using std::set to store raw pointers is not very efficient. Also, constructing std::shared_ptr from std::make_unique will result in 2 allocations, std::make_shared will do only 1 allocation and so on.

None of it had any significant impact in this case though. Without the patch (current master) it takes 4 minutes 20 secs to load the pcb file and with the patch it was 4 minutes 15 secs.

I've also tried caching the result of CN_ITEM::GetAnchor but that didn't have any effect either but instead introduced bugs.

I guess the problem is general runtime complexity of the whole connectivity stuff. In any case, I'm attaching the patch. Maybe somebody can get something out of it, even if it doesn't fix the actual problem at hand.

[Jeff Young (jeyjey)]

@Oleg, does your board contain a lot of pads which touch zones, but which don't have their centres within the zones?

If not, the performance /should/ be roughly equivalent before and after that patch....

[Oleg Endo (oleg.endo)]

As far as I can tell, most pads are completely inside of zones.

[Jeff Young (jeyjey)]

@Oleg, I've merged a change. Can you let me know if it fixes the issue?

[KiCad Janitor (kicad-janitor)]

Fixed in revision 0ad9ddb3a66861ba4309218302a518825981b9cf
https://git.launchpad.net/kicad/patch/?id=0ad9ddb3a66861ba4309218302a518825981b9cf

[Oleg Endo (oleg.endo)]

I've tried out with your latest commit. Yes, it's a significant improvement :)
Thanks!

[Jeff Young (jeyjey)]

Cool. I merged some of your fixes as well. Thanks for your contribution to Kicad!

[KiCad Janitor (kicad-janitor)]

Fixed in revision 429c7055e9968c454a8f8b5910c87ddf0c8d0a02
https://git.launchpad.net/kicad/patch/?id=429c7055e9968c454a8f8b5910c87ddf0c8d0a02