support MingW cross compile

[Steven Boswell]

I just successfully built 1.11.0 (well, latest bzr of the 1.11 branch) under MS Windows XP, and made an installer too.
The enclosed patch fixes some issues that I encountered along the way.

1) scons couldn't find the protoc executable; it's in the winlib directory. The winlib directory had never been added to the path! The change to build/depends.py takes care of that.

2) My VC_redist.exe didn't contains a msp_kb2565063.msp, and Google doesn't seem to know much about it either. The installer, and installed program, seems to function fine without it. (My best guess is that it's a security patch that isn't needed if you're using the SP1 version of the MSVC++ 2010 redistributable package.) Unless someone knows what this is, my patch to build/nsis/Mixxx.nsi gets rid of the reference to it.

3) The MS Windows build currently depends on being done from a source-code directory from a bzr checkout. This leads to some damaged data being written to mixxx.rc.include, leading to a VERY cryptic error being generated by MS' rc compiler. http://www.mixxx.org/forums/viewtopic.php?f=3&t=4382 is an unanswered forum post on the same topic. The first half of my patch to src/SConscript works around the problem.

4) http://mixxx.org/wiki/doku.php/compiling_on_windows contains a note: "If you're using Qt 4.8 or above, you might need to copy all of the Qt4 DLL files from C:\qt\bin to C:\qt\lib since that's where our build script looks for them. (We should fix that.)" The second half of my patch to src/SConscript fixes that.

As soon as I'm ready for more punishment, I'm going to attempt a Qt Creator/MinGW build. ;-)

[Sean M. Pappalardo (pegasus-renegadetech)]

Great work! Thanks for tackling these issues.

For (2), msp_kb2565063.msp is indeed a security patch and must be downloaded manually. I should add instructions somewhere about where to find it. (Search microsoft.com/downloads for that KB article number in the meantime.)

For (4), I would rather that scons looks in both places for the files. If it can't find a file in one place, search the other. The reason for this is because building Qt from source puts them in one place while downloading the pre-compiled DLLs puts them in the other and there's no telling which situation a particular user will have. (I just don't know enough python yet to add that intelligence or I would've done it already.)

[Steven Boswell (ulatekh)]

Re #2: I'll see if I can track down that file.
Re #4: I'll see if I can figure out how to make that seamless.

Apparently I'm a glutton for punishment...it became apparent pretty quickly that build/qtcreator/mixxx.pro was woefully out of date. Now, it's a lot less out of date. The enclosed archive contains a patch, and a new file, that significantly advances the mingw build!

Some paths (e.g. mixxx-mingw, where I installed all the mingw-built dependent packages) are hardcoded, but given how broken mixxx.pro was before, I doubt this makes it any more broken. :-) At some point, I can include all the instructions and patch files for the mingw build of the dependencies. (If I get REALLY motivated, I can send those patches upstream to the dependency package maintainers.)

It built, linked, and even ran, mostly...it crashed in a very strange way:

Program received signal SIGSEGV, Segmentation fault.
#0 google::protobuf::internal::RepeatedPtrFieldBase::Reserve (this=0x0,
    new_size=2685088) at google/protobuf/repeated_field.cc:46
#1 0x00738040 in google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf
::RepeatedPtrField<mixxx::skin::SkinManifest_Attribute>::TypeHandler> (this=0x28f8a0)
    at ../mixxx-winlib/google/protobuf/repeated_field.h:675
#2 0x0073748d in google::protobuf::RepeatedPtrField<mixxx::skin::SkinManifest_Attribute
>::Add (this=0x28f8a0) at ../mixxx-winlib/google/protobuf/repeated_field.h:870
#3 0x007346b8 in mixxx::skin::SkinManifest::add_attribute (this=0x28f884)
    at ./proto/skin.pb.h:772
#4 0x005e4456 in LegacySkinParser::getSkinManifest (skinDocument=...)
    at ..\mixxx-winsrc\src\skin\legacyskinparser.cpp:225
#5 0x005e480d in LegacySkinParser::parseSkin (this=0x28f984, skinPath=..., pParent=
    0x23e9b6d8) at ..\mixxx-winsrc\src\skin\legacyskinparser.cpp:256
#6 0x005e20bf in SkinLoader::loadDefaultSkin (this=0x23d0d668, pParent=0x23e9b6d8,
    pKeyboard=0x1d1bc260, pPlayerManager=0x1da66d08, pControllerManager=0x20e88c58,
    pLibrary=0x20dffa48, pVCMan=0x347dd8)
    at ..\mixxx-winsrc\src\skin\skinloader.cpp:70
#7 0x004bd667 in MixxxApp::MixxxApp (this=0x1d1ba378, pApp=0x28fe2c, args=...)
    at ..\mixxx-winsrc\src\mixxx.cpp:456
#8 0x004b54ea in qMain (argc=1, argv=0x1d1aaa20) at ..\mixxx-winsrc\src\main.cpp:284

I'm not sure how "this" can become NULL between google::protobuf::internal::RepeatedPtrFieldBase::Add() and google::protobuf::internal::RepeatedPtrFieldBase::Reserve(), so I'm stuck.

So hopefully you'll be willing to apply these patches. I tested them with the MSVC Win32 build and the Linux (i.e. Fedora Core 18) build, and all seems well.

Until I figure out what to do about the above crash, I think I'll try to get scons to build mixxx with the mingw toolchain.

[Steven Boswell (ulatekh)]

I have great news! I finally decided to try out the MinGW suite that's available for Fedora Core, i.e. cross-compile the MS Windows version of Mixxx using Linux.

It's running! It's not totally working yet, but the GUI comes up, and there are no mysterious crashes.

I'll probably make a branch with all of my changes, and eventually ask that it be merged into the trunk.

[RJ Skerry-Ryan (rryan)]

Cool stuff -- I got a mingw build cross-compiled from Windows back in 2009
(and an OS X build cross-compiled from Linux) but it wasn't pretty. All the
references to crosscompile in the SConscript are from that experiment.

On Thu, May 16, 2013 at 2:14 PM, Steven Boswell <email address hidden> wrote:

> ** Branch linked: lp:~ulatekh/mixxx/mingw
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/1179683
>
> Title:
> Mixxx 1.11.0 MS Windows build
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/1179683/+subscriptions
>

[Steven Boswell (ulatekh)]

The current cross-compiled MS Windows build mostly works -- I've played tracks successfully. But I don't get any tracks displayed in the library! (I can play tracks from File -> Load track to deck, and by dragging a track from Explorer.) Right now I'm building a newer version of Qt (4.8.4-17, the one Fedora Core 18 is running natively) than the one that currently comes with MinGW (4.8.4-1) to see if that has any effect.

I intend to try to cross-compile an MS Windows build from Linux using scons, really soon now. (The Qt cross-compile is taking forever.)

At some point, I'll also post my MinGW dependency-build results (i.e. new spec files, and any patches referred to by them), probably as a separate repo filed under the Mixxx project. So far, I have MinGW versions of flac, gsm, libid3tag, libmad, libsndfile, portaudio, portmidi, protobuf, and taglib, built both for win32 and win64. portaudio is currently build with DirectSound support; I need to rebuild it and add JACK support.

I've made a lot more progress than I thought I would :-)

[Steven Boswell (ulatekh)]

Thank you, RJ, for doing most of the hard work!
It didn't take very long to get your crosscompile code working again.
My changes are now checked in.

Now I'm hell-bent to get the dependency packages cleaned up and ready for submission to the repos.
Before that, I'll post instructions on how to build them yourself.

[Steven Boswell (ulatekh)]

Here's the extra data needed to build MinGW packages of a minimal set of Mixxx's dependencies.
I'm still trying to build jack-audio-connection-kit under MinGW; until then, Mixxx will use DirectSound.
I still want to build phonon for MinGW, but that has a lot of nasty dependencies (like KDE :-).

Attached is an archive that should get unpacked to your ~/rpmbuild directory. build.sh is a shell script that will build the dependencies for Fedora Core 18 as of today. The SPECS directory contains the original spec files from the non-MinGW packages; you can diff them with the MinGW spec files to see how I transformed each one into the MinGW variant.

I'm trying to get these packages submitted to Fedora, but based on the lengthy list of package-review requests that have never been looked at, it seems like Fedora is where package-requests go to die from neglect. So, outside of mixxx.org having its own yum repo, building Mixxx's MinGW dependencies may not get any more convenient than this.

[Steven Boswell (ulatekh)]

Now there's a more convenient location for the latest data needed to build MinGW packages of Mixxx's dependencies:

https://github.com/ulatekh/fedora-mingw-ardour

There's a lot of crossover between Ardour's dependencies and Mixxx's.

With my latest checkin, making an MS Windows installer with scons works under Fedora Core. I can build a win64 version of Mixxx, but it doesn't run under MS Windows; all I get is an unhelpful "bad image" (i.e. 0xc000007b) error.

[Steven Boswell (ulatekh)]

I consider this project done! I've requested a merge of the branch.

Still to do:

1) Get the dependent packages approved by Fedora and RPMFusion, so that building them locally is no longer necessary.
2) Continue to build ffmpeg's dependencies, so that an MS Windows version of Tuukka's ffmpeg branch can be built using this method. (As of last night, I've built a minimal MinGW ffmpeg, and the resulting MS Windows version was able to play a .wma file. I only have several dozen dependencies to do now. :-)

[RJ Skerry-Ryan (rryan)]

Thanks Steven -- I took the various fixes from your patch in #3 and committed them.

* The mixxx.pro changes have rotted unfortunately. We should probably delete them as we don't support any build system other than SCons and they are always out of date.

* Instead of adding extern C to lut.cpp and timecoder_win32.cpp I just changed them to be C files (and removed the C++ stuff that had been added). Not sure why they were C++ in the first place since they're just our Win32-specific hacks to xwax.

[RJ Skerry-Ryan (rryan)]

Oops, now I see why those are C++ -- it's just easier given xwax uses features MSVS doesn't support for C. Reverted and added extern C.

[RJ Skerry-Ryan (rryan)]

Renaming this bug to describe mingw cross-compile support.

The Git conversion of ulatekh's branch is here for anyone looking to work on merging it:
https://github.com/mixxxcommunity/mixxx/commits/ulatekh-mingw

[Be (be.ing)]

There's no need for adding this maintenance burden.

[Steven Boswell (ulatekh)]

Well, not if you sit on it for almost seven years.

I regret getting involved with this apathetic project.

[Daniel Schürmann (daschuer)]

It sounds like your are upset that we have closed the bug. I am sorry about that.

What is your use case?

We are now at CMake and CMake is able to build with MinGW out of the box.

I am sure there are some tweaks required to get it to work, but from my experience it should not be a maintenance burden at all to keep MinGW support in our CMakeList.txt.
This is just because we have Windows support and GCC support in it, which is maintained anyway.

The maintenance burden comes in if we maintain this with our CI and together with all dependencies. But If this bug here melts down to keep our CMakeList.txt clean, ther is nothing against it IMHO.

What do you think?

[Be (be.ing)]

If all it takes is a few changes to CMakeLists.txt, sure we could merge that if someone does the work. But nobody has explained why this would be beneficial. As far as I can tell, the only reason to do this would be to say that it can be done. I would be opposed to using CI resources on this without a good reason. Without that, it would be up to whoever adds the support to ensure it keeps working and I don't think we should accept responsibility for keeping it working.

[RJ Skerry-Ryan (rryan)]

Ah, the only reason to do this is to avoid having to interact with Windows at all (in CI / build servers). The original motivation when I first got mingw cross compiling hacked into our SConscripts in.... 2010.. 2011? was that I didn't want to have to run Windows VMs on the Mixxx build server.

Which is a worthy goal :) but the cure may be worse than the illness given the mysterious crashes in the mingw-produced binaries that I experienced.

[Be (be.ing)]

Well, nowadays we have free VMs updated for us on GitHub Actions, so that's not something we need to worry about anymore.

[Be (be.ing)]

We cannot avoid using Windows VMs for CI and only crosscompile from Linux. I know MinGW can work on either Windows or Linux, but if we don't have CI running on Windows we may commit regressions that break the build on Windows. We already have a hard enough time attracting and retaining Windows and macOS developers. Let's not make it harder for them.

[Swiftb0y (swiftb0y)]

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/7030