| 2019-10-14 11:23:34 |
Ievgenii Meshcheriakov |
description |
Parallel builds that use GNU ar, objcopy, or strip may fail or generate corrupted files on Windows due to unsafe temporary file handling by the version of MinGW-w64 used to build binutils. The usual symptom are messages like: "unable to rename: <file name>; reason: No such file or directory" or "unable to rename: <file name>; reason: Permission denied", or silent output file corruption. This applies to the latest currently available version of the toolchain (8-2019-q3-update) and at least some previous versions.
The root cause of the issue is usage of MinGW-w64 that is missing this fix for mkstemp(): https://github.com/mirror/mingw-w64/commit/76119a8e8938dd23cdb4fe72843723fe4d4cc121#diff-cc1bd3d845f9a0ff2d38778e12b84a17
The presence of _O_TEMPORARY flag can be verified by disassembling gnu-arm-none-eabi-arm:
3 loc_48f97b:
4 0048f97b mov dword [esp+0x3c+var_30], 0x180 ; CODE XREF=sub_48f8c0+137
5 0048f983 mov dword [esp+0x3c+var_34], 0x10
6 0048f98b mov dword [esp+0x3c+var_38], 0x8542
7 0048f993 mov dword [esp+0x3c+var_3C], edi
8 0048f996 call j__sopen ; _sopen
9 0048f99b cmp eax, 0xffffffff
0x8542 represents flags passed to _sopen, and 0x40 is _O_TEMPORARY.
Having _O_TEMPORARY here together with code in binutils that closes the returned file descriptor immediately after it is created (https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/bucomm.c;h=06fbc462e242467fe6f15f46e8515d8ddf6b1456;hb=7e27a9d5f22f9f7ead11738b1546d0b5c737266b#l523) creates a race condition allowing multiple processes to use the same temporary file. Usage of rand() for generating candidate file names in mkstemp() makes this bug relatively easy to reproduce.
To reproduce the bug try creating multiple archives in the _same_ _directory_ using arm-none-eabi-ar in parallel. This may require multiple iterations. I'm attaching a script that can be used to generate build.ninja file that will attempt to create multiple archives in parallel. It requires modification to specify input archives (any should work).
Fixing this bug should be as easy as rebuilding the toolchain using a newer version of MinGW-w64 (or MinGW). |
Parallel builds that use GNU ar, objcopy, or strip may fail or generate corrupted files on Windows due to unsafe temporary file handling by the version of MinGW-w64 used to build binutils. The usual symptom are messages like: "unable to rename: <file name>; reason: No such file or directory" or "unable to rename: <file name>; reason: Permission denied", or silent output file corruption. This applies to the latest currently available version of the toolchain (8-2019-q3-update) and at least some previous versions.
The root cause of the issue is usage of MinGW-w64 that is missing this fix for mkstemp(): https://github.com/mirror/mingw-w64/commit/76119a8e8938dd23cdb4fe72843723fe4d4cc121#diff-cc1bd3d845f9a0ff2d38778e12b84a17
The presence of _O_TEMPORARY flag can be verified by disassembling arm-none-eabi-ar:
3 loc_48f97b:
4 0048f97b mov dword [esp+0x3c+var_30], 0x180 ; CODE XREF=sub_48f8c0+137
5 0048f983 mov dword [esp+0x3c+var_34], 0x10
6 0048f98b mov dword [esp+0x3c+var_38], 0x8542
7 0048f993 mov dword [esp+0x3c+var_3C], edi
8 0048f996 call j__sopen ; _sopen
9 0048f99b cmp eax, 0xffffffff
0x8542 represents flags passed to _sopen, and 0x40 is _O_TEMPORARY.
Having _O_TEMPORARY here together with code in binutils that closes the returned file descriptor immediately after it is created (https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/bucomm.c;h=06fbc462e242467fe6f15f46e8515d8ddf6b1456;hb=7e27a9d5f22f9f7ead11738b1546d0b5c737266b#l523) creates a race condition allowing multiple processes to use the same temporary file. Usage of rand() for generating candidate file names in mkstemp() makes this bug relatively easy to reproduce.
To reproduce the bug try creating multiple archives in the _same_ _directory_ using arm-none-eabi-ar in parallel. This may require multiple iterations. I'm attaching a script that can be used to generate build.ninja file that will attempt to create multiple archives in parallel. It requires modification to specify input archives (any should work).
Fixing this bug should be as easy as rebuilding the toolchain using a newer version of MinGW-w64 (or MinGW). |
|
