AppVeyor mingw builds broken (claiming missing epicsTempFile.obj)

Bug #1827225 reported by Ralph Lange
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
EPICS Base
Status tracked in 7.0
3.15
Medium
Ralph Lange
7.0
Medium
Ralph Lange

Bug Description

It looks like the epicsTempName() commit (6201d37 * 3.15) broke the mingw builds.

E.g. on AppVeyor

mingw32-make[4]: Leaving directory 'C:/projects/epics-base/modules/libcom/src/O.win32-x86-mingw'
mingw32-make[4]: *** No rule to make target 'epicsTempFile.obj', needed by 'e_flex.exe'. Stop.

Revision history for this message
Andrew Johnson (anj) wrote :

Both my native MinGW builds (windows-x64-mingw with gcc 7.1.0 from Strawberry Perl 5.26.2) and cross-builds from RHEL-7.6 still work fine. Have you seen any error messages from when epicsTempFile.cpp is being built? This looks similar to my Solaris build failure on Jenkins, neither compiler appears to be printing an error when compiling that file, they just don't generate any output.

Hmm, there are actually no object files at all in the src/libCom/O.<host> directory, is this perhaps a compiler problem instead? IIRC the epicsTempFile.cpp source just happens to be the first one that gets compiled, so it just *looks* like that may be the cause. I suspect my Solaris compilers may have died (see the issue with the procServ build recently), and I get the exact same symptoms on Windows if I try to do a native windows-x64 build without having configured the shell window properly for MSVC.

Changed in epics-base:
status: New → Incomplete
Revision history for this message
mdavidsaver (mdavidsaver) wrote :

I don't see a connection with 6201d377561c3230ca68e6e9871db8e2912644d2,
which didn't change any makefiles. The appveyor log seems to show epicsTempFile.obj
being built, though not long before (a parallel build issue?)

https://ci.appveyor.com/project/epics-base/epics-base/builds/24205837/job/nnqja3bmvo8llb32

Maybe irrelevant to this issue, but is it still necessary for e_flex and antelope to refer to epicsTempFile.obj instead of epicsTempFile.cpp? I just tried with .cpp and it seems to work with make 4.1 and '-j8' on Linux.

Revision history for this message
Ralph Lange (ralph-lange) wrote :

My phone is just not large enough (!) for doing this kind of investigations. (1 May is a public holiday here.)
I was simply suspecting the last commit that changed things in that area. Probably wrong.

summary: - epicsTempName() commit breaks mingw builds
+ AppVeyor mingw builds broken (claiming missing epicsTempFile.obj)
Revision history for this message
Ralph Lange (ralph-lange) wrote :

Last working build (on the 7.0 branch):
base-7.0-307 (https://ci.appveyor.com/project/epics-base/epics-base/builds/19914995/job/j1hssminkqp7bu0h)

There is a major version jump of MinGW and Make in the AppVeyor builds between this and the next build.
base-7.0-307: MinGW 5.3.0, Make 4.1
base-7.0-308: MinGW 8.1.0, Make 4.2.1 (this is still the current version)

Investigating...

Revision history for this message
Ralph Lange (ralph-lange) wrote :

The root cause seems related to the fact that at that point the Chocolatey packagers of MinGW switched their upstream from the original MinGW project to MinGW-w64.
(see https://chocolatey.org/packages/mingw/8.1.0)

On my local machine, I cannot use Chocolatey packages (no Admin rights), compiling on the latest direct install of MinGW-w64/MSYS2 works, but uses different versions of things (e.g. gcc 8.3.0 for 64bit, gcc 7.4.0 for 32bit).

I will work around for the time being by explicitly using the choco MinGW 5.3.0 version as before, and check for updates once in a while.

Changed in epics-base:
status: Incomplete → In Progress
Changed in epics-base:
assignee: nobody → Ralph Lange (ralph-lange)
importance: Undecided → Medium
Revision history for this message
mdavidsaver (mdavidsaver) wrote :

> Make 4.2.1

Interesting... I don't think I've tested with this version yet. Debian 9 still has 4.1.

> neither compiler appears to be printing an error when compiling that file, they just don't generate any output.

Would make delete these as incomplete intermediate files on error?

Revision history for this message
Andrew Johnson (anj) wrote :

> Would make delete these as incomplete intermediate files on error?

The compiler isn't returning an error status though, it just doesn't seem to be creating the output file, or at least not the one we're asking for. For example from the last build that failed (-256) before Ralph fixed it, here's the build for ccl.c:

mingw32-make[3]: Entering directory 'C:/projects/epics-base-5e851/src/libCom/O.windows-x64-mingw'
gcc -D_MINGW -O3 -Wall -m64 -DEPICS_BUILD_DLL -DEPICS_CALL_DLL -I. -I../O.Common -I. -I../../../src/libCom/osi/compiler/gcc -I../../../src/libCom/osi/compiler/default -I. -I../../../src/libCom/osi/os/WIN32 -I../../../src/libCom/osi/os/default -I.. -I../../../src/libCom/as -I../../../src/libCom/bucketLib -I../../../src/libCom/calc -I../../../src/libCom/cvtFast -I../../../src/libCom/cppStd -I../../../src/libCom/cxxTemplates -I../../../src/libCom/dbmf -I../../../src/libCom/ellLib -I../../../src/libCom/env -I../../../src/libCom/error -I../../../src/libCom/fdmgr -I../../../src/libCom/flex -I../../../src/libCom/freeList -I../../../src/libCom/gpHash -I../../../src/libCom/iocsh -I../../../src/libCom/log -I../../../src/libCom/macLib -I../../../src/libCom/misc -I../../../src/libCom/osi -I../../../src/libCom/pool -I../../../src/libCom/ring -I../../../src/libCom/taskwd -I../../../src/libCom/timer -I../../../src/libCom/yacc -I../../../src/libCom/yacc -I../../../src/libCom/yajl -I../../../include/compiler/gcc -I../../../include/os/WIN32 -I../../../include -o ccl.obj -c ../../../src/libCom/flex/ccl.c
mingw32-make[3]: Leaving directory 'C:/projects/epics-base-5e851/src/libCom/O.windows-x64-mingw'

Then later when make tries to use ccl.obj it just gives:

mingw32-make[3]: *** No rule to make target 'ccl.obj', needed by 'e_flex.exe'. Stop.

Hmm, since the appveyor-prepare.bat script installs my Make-4.1 for all builds (Make-4.2.1 is also available), why don't the mingw builds use it? The above messages show that they're using mingw32-make, confirmed by the appveyor-make.bat file.

@Ralph I see this is using an older ActiveState Perl, is that configurable somehow? The latest StrawberryPerl packages (available through Chocolatey) install GNU Make-4.2.1 (named "gmake") and gcc as well, so they provide everything we need for a MinGW build. That's what I'm using here and had no problems with.

Revision history for this message
mdavidsaver (mdavidsaver) wrote :

> I don't think I've tested with this version yet

I built make 4.2.1 from source and did some quick test builds of Base (Linux native and cross mingw). No anomalies.

Revision history for this message
Ralph Lange (ralph-lange) wrote :

I know. It's not make. (At least not alone.)

On the failing 8.1.0 choco MinGW install, using make 4.2.1 (from Andrew / old EPICS site) that is known to work with all kinds of compilers (including cygwin on AppVeyor) does not change the failure. (@Andrew: I tested this today in https://ci.appveyor.com/project/ralphlange/epics-base-5e851/builds/24250315)

On my local Windows box the native MinGW-w64/MSYS install with gcc 8.3.0 and make 4.2.1 works fine.

Revision history for this message
Ralph Lange (ralph-lange) wrote :

Looking at the build outputs of my tests (https://ci.appveyor.com/project/ralphlange/epics-base-5e851/history)...

The failing builds all have weird newlines in the compile commands - the working old build doesn't. Could it be that we're hitting a line length limit of command.com?

Revision history for this message
Ralph Lange (ralph-lange) wrote :

For the record:
Please ignore my last comment. Not the case.

I found that AppVeyor nowadays does have complete recent versions of MinGW pre-installed on the builder VMs (used to not be the case).
Switching to those made the builds successful (and simpler and faster).

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers