That looks like it removes the file before it does the rename, so it misses the special overwrite-by-rename workaround. This is slightly unsafe on any filesystem, since you might be left with no config file with the correct name if the system crashes in a small window, fsync() or no. Seemingly Python on Windows doesn't support an atomic rename operation at all.
It might be simplest for it to only do the remove if rename throws an OSError, or only if the platform is Windows. Ideally it should call fsync() as well, of course.