After exporting a modified footprint, KiCad must be restarted to see it

[Oivind Toien]

After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
Steps to reproduce:
1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
4. Close all KiCad windows except the KiCad main app.
5. repeat step 3. Again the exported footprint does not does not show up.
6. Close KiCad, Open KiCad, and repeat step 3. Now the exported footprint shows up and can be inserted or edited.

This is a bug that has been around at least from ver. 4.0x, however I have only recently started using the nightly again (although only the one from Sept 19 by necessity) to check for it. It makes the workflow of creating "Molecular parts" awkward, as one has to close KiCad after editing the schematic library information and creating a corresponding footprint, but before being able to enter the footprint info into the schematic symbol.

Application: kicad
Version: (2017-09-19 revision dddaa7e69)-makepkg, release build
Libraries:
    wxWidgets 3.0.3
    libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
Platform: Windows 7 (build 7601, Service Pack 1), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
    wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
    Boost: 1.60.0
    Curl: 7.54.1
    Compiler: GCC 7.1.0 with C++ ABI 1011

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_SPICE=ON

[jean-pierre charras (jp-charras)]

Export does not update library, but Save does that.
So uses Save footprint.

[Eldar Khayrullin (eldar)]

It will be good to have possibility to manual updaTe the cash. It is needed when you change footprint on github library.

[Jeff Young (jeyjey)]

The cache now updates automatically for libraries in the library table.

[Oivind Toien (otoien)]

I am afraid I do not see any improvement on this following the steps I posted above to test with the latest Windows nightly (2018-02-09 revision 07fff6ed3). I still have to restart KiCad for the renamed exported footprint to show up in a library listed in the library table. So I cannot confirm the fix.

[Jeff Young (jeyjey)]

Can anyone else reproduce this? It works fine for me on Mac.

[Wayne Stambaugh (stambaughw)]

I can reproduce this on linux. I'm not sure why the path modified time
is not being picked up the library cache flush code. I'll take a look
at it.

On 02/10/2018 12:45 PM, Jeff Young wrote:
> Can anyone else reproduce this? It works fine for me on Mac.
>

[Jeff Young (jeyjey)]

Well rats. From stack overflow:

The mtime (modification time) on the directory itself changes when a file or a subdirectory is added, removed or renamed.

Modifying the contents of a file within the directory does not change the directory itself, nor does updating the modified times of a file or a subdirectory.

[Wayne Stambaugh (stambaughw)]

Yep! I just added a path modification time check for the addition of
new files and it fixes the problem.

On 02/10/2018 04:29 PM, Jeff Young wrote:
> Well rats. From stack overflow:
>
> The mtime (modification time) on the directory itself changes when a
> file or a subdirectory is added, removed or renamed.
>
> Modifying the contents of a file within the directory does not change
> the directory itself, nor does updating the modified times of a file or
> a subdirectory.
>

[KiCad Janitor (kicad-janitor)]

Fixed in revision e832212d896db2de0103f4644182a83bc2461079
https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079

[Jeff Young (jeyjey)]

That’s showing an older version of the file. I don’t think the performance patch has been applied yet.

Can you back that out, apply the patch, and see if you’re still getting the issue?

(I’m playing with a wxFileSystemWatcher that we can use on Unix if need be.)

Cheers,
Jeff.

> On 10 Feb 2018, at 22:15, KiCad Janitor <email address hidden> wrote:
>
> Fixed in revision e832212d896db2de0103f4644182a83bc2461079
> https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079
>
> --
> You received this bug notification because you are a member of KiCad Bug
> Squad, which is subscribed to KiCad.
> https://bugs.launchpad.net/bugs/1727977
>
> Title:
> After exporting a modified footprint, KiCad must be restarted to see
> it
>
> Status in KiCad:
> Fix Committed
>
> Bug description:
> After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
> Steps to reproduce:
> 1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
> 2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
> 3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
> 4. Close all KiCad windows except the KiCad main app.
> 5. repeat step 3. Again the exported footprint does not does not show up.
> 6. Close KiCad, Open KiCad, and repeat step 3. Now the exported footprint shows up and can be inserted or edited.
>
> This is a bug that has been around at least from ver. 4.0x, however I
> have only recently started using the nightly again (although only the
> one from Sept 19 by necessity) to check for it. It makes the workflow
> of creating "Molecular parts" awkward, as one has to close KiCad after
> editing the schematic library information and creating a corresponding
> footprint, but before being able to enter the footprint info into the
> schematic symbol.
>
>
> Application: kicad
> Version: (2017-09-19 revision dddaa7e69)-makepkg, release build
> Libraries:
> wxWidgets 3.0.3
> libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
> Platform: Windows 7 (build 7601, Service Pack 1), 64-bit edition, 64 bit, Little endian, wxMSW
> Build Info:
> wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
> Boost: 1.60.0
> Curl: 7.54.1
> Compiler: GCC 7.1.0 with C++ ABI 1011
>
> Build settings:
> USE_WX_GRAPHICS_CONTEXT=OFF
> USE_WX_OVERLAY=OFF
> KICAD_SCRIPTING=ON
> KICAD_SCRIPTING_MODULES=ON
> KICAD_SCRIPTING_WXPYTHON=ON
> KICAD_SCRIPTING_ACTION_MENU=ON
> BUILD_GITHUB_PLUGIN=ON
> KICAD_USE_OCE=ON
> KICAD_SPICE=ON
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/kicad/+bug/1727977/+subscriptions

[Wayne Stambaugh (stambaughw)]

Be very careful with wxFileSystemWatcher. I didn't use it when I
implemented the plugin because the quality of wxFileSystemWatcher varies
widely across platforms. There is even a note in the documentation[1]
about this.

[1]:
http://docs.wxwidgets.org/3.0/classwx_file_system_watcher.html#af45bbabc22808e5f4a2dc560e1e30aa9

On 02/10/2018 05:26 PM, Jeff Young wrote:
> That’s showing an older version of the file. I don’t think the
> performance patch has been applied yet.
>
> Can you back that out, apply the patch, and see if you’re still getting
> the issue?
>
> (I’m playing with a wxFileSystemWatcher that we can use on Unix if need
> be.)
>
> Cheers,
> Jeff.
>
>
>> On 10 Feb 2018, at 22:15, KiCad Janitor <email address hidden> wrote:
>>
>> Fixed in revision e832212d896db2de0103f4644182a83bc2461079
>> https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079
>>
>> --
>> You received this bug notification because you are a member of KiCad Bug
>> Squad, which is subscribed to KiCad.
>> https://bugs.launchpad.net/bugs/1727977
>>
>> Title:
>> After exporting a modified footprint, KiCad must be restarted to see
>> it
>>
>> Status in KiCad:
>> Fix Committed
>>
>> Bug description:
>> After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
>> Steps to reproduce:
>> 1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
>> 2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
>> 3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
>> 4. Close all KiCad windows except the KiCad main app.
>> 5. repeat step 3. Again the exported footprint does not does not show up.
>> 6. Close KiCad, Open KiCad, and repeat step 3. Now the exported footprint shows up and can be inserted or edited.
>>
>> This is a bug that has been around at least from ver. 4.0x, however I
>> have only recently started using the nightly again (although only the
>> one from Sept 19 by necessity) to check for it. It makes the workflow
>> of creating "Molecular parts" awkward, as one has to close KiCad after
>> editing the schematic library information and creating a corresponding
>> footprint, but before being able to enter the footprint info into the
>> schematic symbol.
>>
>>
>> Application: kicad
>> Version: (2017-09-19 revision dddaa7e69)-makepkg, release build
>> Libraries:
>> wxWidgets 3.0.3
>> libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
>> Platform: Windows 7 (build 7601, Service Pack 1), 64-bit edition, 64 bit, Little endian, wxMSW
>> Build Info:
>> wxWidgets: 3.0.3 (wchar_t,wx containers,compatible with 2.8)
>> Boost: 1.60.0
>> Curl: 7.54.1
>> Compiler: GCC 7.1.0 with C++ ABI 1011
>>
>> Build settings:
>> ...

[Jeff Young (jeyjey)]

Indeed, but it’s perfect for us here. It only works well on Unix, while the existing lastModTime checking only doesn’t work on Unix. ;)

> On 10 Feb 2018, at 23:24, Wayne Stambaugh <email address hidden> wrote:
>
> Be very careful with wxFileSystemWatcher. I didn't use it when I
> implemented the plugin because the quality of wxFileSystemWatcher varies
> widely across platforms. There is even a note in the documentation[1]
> about this.
>
> [1]:
> http://docs.wxwidgets.org/3.0/classwx_file_system_watcher.html#af45bbabc22808e5f4a2dc560e1e30aa9
>
> On 02/10/2018 05:26 PM, Jeff Young wrote:
>> That’s showing an older version of the file. I don’t think the
>> performance patch has been applied yet.
>>
>> Can you back that out, apply the patch, and see if you’re still getting
>> the issue?
>>
>> (I’m playing with a wxFileSystemWatcher that we can use on Unix if need
>> be.)
>>
>> Cheers,
>> Jeff.
>>
>>
>>> On 10 Feb 2018, at 22:15, KiCad Janitor <email address hidden> wrote:
>>>
>>> Fixed in revision e832212d896db2de0103f4644182a83bc2461079
>>> https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079
>>>
>>> --
>>> You received this bug notification because you are a member of KiCad Bug
>>> Squad, which is subscribed to KiCad.
>>> https://bugs.launchpad.net/bugs/1727977
>>>
>>> Title:
>>> After exporting a modified footprint, KiCad must be restarted to see
>>> it
>>>
>>> Status in KiCad:
>>> Fix Committed
>>>
>>> Bug description:
>>> After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
>>> Steps to reproduce:
>>> 1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
>>> 2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
>>> 3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
>>> 4. Close all KiCad windows except the KiCad main app.
>>> 5. repeat step 3. Again the exported footprint does not does not show up.
>>> 6. Close KiCad, Open KiCad, and repeat step 3. Now the exported footprint shows up and can be inserted or edited.
>>>
>>> This is a bug that has been around at least from ver. 4.0x, however I
>>> have only recently started using the nightly again (although only the
>>> one from Sept 19 by necessity) to check for it. It makes the workflow
>>> of creating "Molecular parts" awkward, as one has to close KiCad after
>>> editing the schematic library information and creating a corresponding
>>> footprint, but before being able to enter the footprint info into the
>>> schematic symbol.
>>>
>>>
>>> Application: kicad
>>> Version: (2017-09-19 revision dddaa7e69)-makepkg, release build
>>> Libraries:
>>> wxWidgets 3.0.3
>>> libcurl/7.54.1 OpenSSL/1.0.2l zlib/1.2.11 libssh2/1.8.0 nghttp2/1.23.1 librtmp/2.3
>>> Platform: Windows 7 (build 76...

[Wayne Stambaugh (stambaughw)]

I developed and tested the latest patch on linux so checking the path
modification time seems to work just fine on Posix systems when adding
and removing files from the path. There is already code in place to
test for file modification time checks which the path modification time
checks do not cover. Does my last patch not work on macos?

On 02/10/2018 06:36 PM, Jeff Young wrote:
> Indeed, but it’s perfect for us here. It only works well on Unix, while
> the existing lastModTime checking only doesn’t work on Unix. ;)
>
>> On 10 Feb 2018, at 23:24, Wayne Stambaugh <email address hidden> wrote:
>>
>> Be very careful with wxFileSystemWatcher. I didn't use it when I
>> implemented the plugin because the quality of wxFileSystemWatcher varies
>> widely across platforms. There is even a note in the documentation[1]
>> about this.
>>
>> [1]:
>> http://docs.wxwidgets.org/3.0/classwx_file_system_watcher.html#af45bbabc22808e5f4a2dc560e1e30aa9
>>
>> On 02/10/2018 05:26 PM, Jeff Young wrote:
>>> That’s showing an older version of the file. I don’t think the
>>> performance patch has been applied yet.
>>>
>>> Can you back that out, apply the patch, and see if you’re still getting
>>> the issue?
>>>
>>> (I’m playing with a wxFileSystemWatcher that we can use on Unix if need
>>> be.)
>>>
>>> Cheers,
>>> Jeff.
>>>
>>>
>>>> On 10 Feb 2018, at 22:15, KiCad Janitor <email address hidden> wrote:
>>>>
>>>> Fixed in revision e832212d896db2de0103f4644182a83bc2461079
>>>> https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079
>>>>
>>>> --
>>>> You received this bug notification because you are a member of KiCad Bug
>>>> Squad, which is subscribed to KiCad.
>>>> https://bugs.launchpad.net/bugs/1727977
>>>>
>>>> Title:
>>>> After exporting a modified footprint, KiCad must be restarted to see
>>>> it
>>>>
>>>> Status in KiCad:
>>>> Fix Committed
>>>>
>>>> Bug description:
>>>> After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
>>>> Steps to reproduce:
>>>> 1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
>>>> 2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
>>>> 3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
>>>> 4. Close all KiCad windows except the KiCad main app.
>>>> 5. repeat step 3. Again the exported footprint does not does not show up.
>>>> 6. Close KiCad, Open KiCad, and repeat step 3. Now the exported footprint shows up and can be inserted or edited.
>>>>
>>>> This is a bug that has been around at least from ver. 4.0x, however I
>>>> have only recently started using the nightly again (although only the
>>>> one from Sept 19 by necessity) to check for it. It makes the workflow
>>>> of creating "Molecular parts" awkward, as one has to close KiCad aft...

[Jeff Young (jeyjey)]

Hi Wayne,

The performance optimisation pretty much removes the routine you modified entirely. All that’s left is a lastModTime check.

I think the performance patch (and your change) both cover the file added/removed, but won’t cover the file modified (in an external tool) problem on Posix. For that we probably nee the fswatcher.

Cheers,
Jeff.

> On 10 Feb 2018, at 23:56, Wayne Stambaugh <email address hidden> wrote:
>
> I developed and tested the latest patch on linux so checking the path
> modification time seems to work just fine on Posix systems when adding
> and removing files from the path. There is already code in place to
> test for file modification time checks which the path modification time
> checks do not cover. Does my last patch not work on macos?
>
> On 02/10/2018 06:36 PM, Jeff Young wrote:
>> Indeed, but it’s perfect for us here. It only works well on Unix, while
>> the existing lastModTime checking only doesn’t work on Unix. ;)
>>
>>> On 10 Feb 2018, at 23:24, Wayne Stambaugh <email address hidden> wrote:
>>>
>>> Be very careful with wxFileSystemWatcher. I didn't use it when I
>>> implemented the plugin because the quality of wxFileSystemWatcher varies
>>> widely across platforms. There is even a note in the documentation[1]
>>> about this.
>>>
>>> [1]:
>>> http://docs.wxwidgets.org/3.0/classwx_file_system_watcher.html#af45bbabc22808e5f4a2dc560e1e30aa9
>>>
>>> On 02/10/2018 05:26 PM, Jeff Young wrote:
>>>> That’s showing an older version of the file. I don’t think the
>>>> performance patch has been applied yet.
>>>>
>>>> Can you back that out, apply the patch, and see if you’re still getting
>>>> the issue?
>>>>
>>>> (I’m playing with a wxFileSystemWatcher that we can use on Unix if need
>>>> be.)
>>>>
>>>> Cheers,
>>>> Jeff.
>>>>
>>>>
>>>>> On 10 Feb 2018, at 22:15, KiCad Janitor <email address hidden> wrote:
>>>>>
>>>>> Fixed in revision e832212d896db2de0103f4644182a83bc2461079
>>>>> https://git.launchpad.net/kicad/patch/?id=e832212d896db2de0103f4644182a83bc2461079
>>>>>
>>>>> --
>>>>> You received this bug notification because you are a member of KiCad Bug
>>>>> Squad, which is subscribed to KiCad.
>>>>> https://bugs.launchpad.net/bugs/1727977
>>>>>
>>>>> Title:
>>>>> After exporting a modified footprint, KiCad must be restarted to see
>>>>> it
>>>>>
>>>>> Status in KiCad:
>>>>> Fix Committed
>>>>>
>>>>> Bug description:
>>>>> After exporting a footprint under a new name, KiCad must be restarted to refresh the information see it.
>>>>> Steps to reproduce:
>>>>> 1. In the PCB library editor, open a library and load a footprint (for instance on of the standard ones that come with KiCad. (I store the standard footprints locally under my home directory.)
>>>>> 2. Export the footprint under a new name to your own pretty folder that contains your modified footprints.
>>>>> 3. Try to open the footprint with the library browser or try to insert the footprint into a board using the browser, or search for it in the field editor of the shematics library editor. It does not show up in any of the browsers.
>>>>> 4. Close all KiCad windows except the KiCad main app.
>>>>> 5....

[Oivind Toien (otoien)]

Thanks, I tested latest Windows nightly (2018-02-12 revision ed127e866) and I can confirm that the fix works now.