MacPorts boost lib not detected on OS X

[vszakats]

Hi,

Tested with latest bzr repository, after installing all dependencies via MacPorts 1.8.1,
this error remains when launching build with scons:
---
...
Checking for C++ header file boost/version.hpp... no
 boost not found.
 Note: You might have the lib but not the headers
---
boost headers *are* installed in /opt/local/include/boost/*, but it's not picked.
(all other MacPorts dependencies are recognized)

If I comment out the check in SConstruct and add -I/opt/local/include to CXXFLAGS,
it builds correctly.

[Steven Sheehy (steven-sheehy)]

Sounds like your system's include path is not setup properly if it can't locate a common include directory like /opt/local/include. I don't see a reason to hardcode an include path; it could in fact break builds for other mac users (see bug #362878). I would suggest you fix your system's include path or add -I/opt/local/include to CXXFLAGS before executing scons (`CXXFLAGS=-I/opt/local/include scons`). You can also export the CXXFLAGS in your profile so it is set automatically at startup.

[vszakats (vszakats)]

Hi,

That's fine and I'll do that. The reason I reported the issue,
was that all the other MacPorts components (f.e. libnotify)
were properly recognized in this same header dir without
any special configuration; except boost, which was not.

This may be a sign of some internal differences between
C vs C++ header detection in scons.

Brgds,
Viktor

[vszakats (vszakats)]

Ops, one more: Above trick may be worth mentioning in readme, as this
is a quite common scenario. The other scenario depicted in bug #362878
is less usual AFAIK, since it uses MacPorts g++, not the more common
compiler versions supplied with Apple XCode, which I also used.

[Steven Sheehy (steven-sheehy)]

If you're willing to write up a few formal sentences describing the issue and its solution, I'll add it to the readme.

[vszakats (vszakats)]

Here it is:

Mac OS X (Darwin) instructions
----------------------------------------

You'll have to have these components installed:
- XCode
- MacPorts
- Certain MacPorts ports:
  $ sudo /opt/local/bin/port install bzr scons pkgconfig libglade2 libnotify boost

For a successful build, you'll have to explicitly specify MacPorts header
dir so that boost lib gets found (make sure to start with a fresh source
tree):

$ export CXXFLAGS=-I/opt/local/include
$ scons PREFIX=<prefix, f.e. ~/linuxdcpp>
$ scons install

To run linuxdcpp without missing icons, you'll have to install hicolor
GTK theme from http://icon-theme.freedesktop.org/releases/:

$ wget http://icon-theme.freedesktop.org/releases/hicolor-icon-theme-0.11.tar.gz
$ tar -zxvf hicolor-icon-theme-0.11.tar.gz
$ cd hicolor-icon-theme-0.11
$ ./configure
$ sudo make

Finally, run linuxdcpp either by starting it from Terminal, or
by double clicking on it in Finder.

[Steven Sheehy (steven-sheehy)]

Thanks for the write up. I was actually thinking of just adding a "Known Issues" section with the mac include path issue. We can also integrate the mac dependencies you provided into the regular dependencies section. The rest of the text is mostly covered by the existing text. Later once launchpad finally adds a wiki (bug #240067), we can add your complete write up there. But if you think it's better to have a separate Mac section we can do that. What do you think?

[vszakats (vszakats)]

Hi,
Current structure of Readme.txt holds no separate section for different platforms,
and it's good that way, so it'd be IMO prettier to include just a separate Mac OS X
prerequisites subsection (below), which you can easily integrate into current
"Build dependencies:" section.

Mac OS X (Darwin) prerequisites
------------------------------------------

You'll have to have these components installed:
- XCode
- MacPorts
- Certain MacPorts ports:
   $ sudo /opt/local/bin/port install bzr scons pkgconfig libglade2 libnotify boost
- GTK hicolor theme from http://icon-theme.freedesktop.org/releases/:
   $ wget http://icon-theme.freedesktop.org/releases/hicolor-icon-theme-0.11.tar.gz
   $ tar -zxvf hicolor-icon-theme-0.11.tar.gz
   $ cd hicolor-icon-theme-0.11
   $ ./configure
   $ sudo make
- You'll have to explicitly specify MacPorts header dir so that boost lib gets found
  (make sure to start with a fresh source tree):
   $ export CXXFLAGS=-I/opt/local/include
---

(From this point the same applies as for Linux.)

[Ximin Luo (infinity0)]

> The other scenario depicted in bug #362878 is less usual AFAIK, since it uses MacPorts g++,

No, if you use MacPorts libraries such as libboost, then you must use MacPorts compiler. It is that simple. MacPorts itself uses its own gcc and g++. You don't mix compilers and libraries from different packaging systems.

[Ximin Luo (infinity0)]

The instructions given above are *not* a good way to compile LinuxDC++ on Mac OS X, because it mixes libraries and compilers from different packaging systems (XCode vs MacPorts).

If you want to use MacPorts' libboost to compile linuxdcpp, then the proper way to do this is to make a MacPorts port of LinuxDC++, either by adding it to the official macports repository, or running your own MacPorts repository.

I went for the 2nd option since I have no time to maintain a MacPorts port myself - I don't use a Mac and only did this as a favour for a few friends that needed a Mac ADC client. However, if someone is willing, I can supply the MacPorts metafiles I created, and you can file a request to add it to the official MacPorts repository, and help maintain it.

[vszakats (vszakats)]

I don't agree that proper way is to use MacPorts own GCC builds.

Not even MacPorts requires you to install its own GCC build in order to install/install any MacPorts packages.
It uses XCode GCC happily. MacPorts clearly says on its install page that XCode is one of its requirements:
   http://www.macports.org/install.php

Also notice that XCode GCC (and now clang) is the officially supported C compiler on OS X platform, so any software claiming OS X compatibility, should be able to build with it.

Even LinuxDC++ builds/works this way perfectly, and we're using MacPorts packages with XCode GCC in other projects without a single hitch, all you have to do.

Also note you can freely mix libs/objects created by different builds of GCC (as long as the
CPU architectures match).

The only difference between XCode GCC and MacPorts GCC, is that the latter "knows" about MacPorts header locations, so you don't have to configure them manually. Although it's unclear for me why only boost lib headers are missed this way.

To put it shortly: The point of MacPorts is not to supply an alternate universe with the whole development toolchain, but rather to nicely plugin into the default OS X development infrastructure (which is XCode).

[Ximin Luo (infinity0)]

> Not even MacPorts requires you to install its own GCC build in order to install/install any MacPorts packages.

Installing MacParts GCC allows you to *build* MacPorts packages automatically using `port`, or that was the case when we tried it. Also Last time I checked, XCode used an old version of gcc (4.1?); some tools require gcc 4.3 which is only available in MacPorts.

> MacPorts clearly says on its install page that XCode is one of its requirements

This is for the MacPorts software *itself*, not its packages.

> any software claiming OS X compatibility, should be able to build with it.

LinuxDC++ does not claim this.

> The point of MacPorts is not to supply an alternate universe with the whole development toolchain

Not for normal users. However if you're a developer, making a MacPorts package, you should be building stuff using the MacPorts toolchain.

> but rather to nicely plugin into the default OS X development infrastructure (which is XCode).

No, this is not true. Why do you think they install everything in /opt/local instead of the system folders??

[vszakats (vszakats)]

Sorry, but you are wrong in several points. XCode GCC is used to build/compile _ports_. Not
MacPorts itself. When you install MacPorts, it will simply copy it's pre-built binary files (port
executable most importantly) into appropriate locations (/opt/local tree). No GCC needed for
that.

Check:
    http://distfiles.macports.org/MacPorts/MacPorts-1.8.1-10.6-SnowLeopard.dmg

Actually, you can also build MacPorts from source and naturally you'll need XCode for that.
I reckon most users won't bother building it from source though, when there is a plain installer.

Then, when you install ports, you do not need MacPorts GCC. I never tried to install it, and it's
not present by default, yet, I can install any ports without problems, and each port is built locally
from source. Since there is no other GCC on my machine than XCode one, it's the one being
used.

Could be that some ports have special requirements, but please don't form the general idea
around a few exceptions.

MacPorts installs everything under /opt/local because MacPorts is not part of the operating system. You may
even build packages which are already present in base OS install (but with older version f.e.) and
overwriting system parts could be fatal, but for sure not very clean thing to do. Not to mention
there exist other "ports" system for OS X, and these might also collide if they'd use system locations.
So they are installed to a separate, non-system place.

XCode comes with 4.0.x and 4.2.x (both) GCC version since Leopard, where 4.0.x was
the default in Leopard, and 4.2.1 is default with new XCode which came out around
Snow Leopard.

[Ximin Luo (infinity0)]

> Actually, you can also build MacPorts from source and naturally you'll need XCode for that.
This is the scenario I am talking about. Here, it's better ("more correct") to use MP gcc to compile MP packages from MP source, because then you make sure everything is self-contained in the MP system. Using XCode is mixing two toolchains and asking for disaster.

> I reckon most users won't bother building it from source though, when there is a plain installer.
As a MacPorts developer, you should build MP packages by making a Portfile for it, then using "port build" or whatever the command is. Any other way (such as using "gcc" directly) might work, but is essentially an improper hack.

> Could be that some ports have special requirements, but please don't form the general idea around a few exceptions.
This is the *rule*, not the exception. In any source-based packaging system, packages have a "build-depends" list of packages from the same packaging system, that are required to build the package. The build is done using the packaging system's own toolchain. Eg. dpkg-buildpackage for debian, port for MP.

> You may even build packages which are already present in base OS install
You should realise that this is "the smoking gun". MP needs its own version of the packages for its own build system, because it is a *self-contained packaging system*, like dpkg/apt. For example, in any debian system you can use gcc to compile a program, but this is not the "proper way" of creating a software package for debian (this is analogous to what you seem to be doing - installing MacPorts libraries then using gcc to compile another non-MacPorts software). Anyone who wants to do this should realise that they might need to add some extra flags or hacks for the compile to work.

[Ximin Luo (infinity0)]

>> Actually, you can also build MacPorts from source and naturally you'll need XCode for that.
> This is the scenario I am talking about

My mistake, I read that wrong. I thought you were talking about building MP packages.

> I can install any ports without problems, and each port is built locally from source. Since there is no other GCC on my machine than XCode one, it's the one being used.

In this case, `port` is probably adding /opt/local to the include paths automatically when it calls `gcc`. Did you use a Portfile to build LinuxDC++ or did you just use XCode gcc directly? (I'm guessing the latter?)

> overwriting system parts could be fatal, but for sure not very clean thing to do.

BTW, MacPorts will never overwrite system parts because everything is in /opt/local. This won't affect the rest of your system at all.

for more information about the MacParts toolchain see http://guide.macports.org/

[vszakats (vszakats)]

> I reckon most users won't bother building it from source though, when there is a plain installer.
> As a MacPorts developer, you should build MP packages by making a Portfile for it, then using "port build" or whatever the
> command is. Any other way (such as using "gcc" directly) might work, but is essentially an improper hack.

Well, if your aim is to include linuxdcpp as a MacPorts port, which is a reasonable
goal, you should naturally deal with some extra problems, but currently linuxdcpp
isn't available as standard port, and all my instructions were written to cover
current situation. In current situation direct usage of GCC is the _only_ way to
build linuxdcpp on OS X, and I was covering exactly that.

Definitely for MacPorts ports 'install <portname>' is the way to go for users, and I'm
not sure who (and why) would want to use hacks instead, if it's possible at all.

Anyhow someone may want to manually build packages (outside the realm of
MacPorts or other port engines), which is also perfectly valid, and may even work,
and in this case using GCC compiler directly is the way to go. (exactly like with
linuxdcpp)

> Could be that some ports have special requirements, but please don't form the general idea around a few exceptions.
> This is the *rule*, not the exception. In any source-based packaging system, packages have a "build-depends" list of
> packages from the same packaging system, that are required to build the package. The build is done using the packaging
> system's own toolchain. Eg. dpkg-buildpackage for debian, port for MP.

IMO you're confusing the packaging system with the compiler toolchain.

Just to repeat: My system, which is a fairly default one, has NO MacPorts
GCC installed, only XCode supplied GCC, and installation (build) of any
MacPorts ports is possible.

> You may even build packages which are already present in base OS install
> You should realise that this is "the smoking gun". MP needs its own version of the packages for its own build system,
> because it is a *self-contained packaging system*, like dpkg/apt. For example, in any debian system you can use gcc to
> compile a program, but this is not the "proper way" of creating a software package for debian (this is analogous to what you > seem to be doing - installing MacPorts libraries then using gcc to compile another non-MacPorts software). Anyone who
> wants to do this should realise that they might need to add some extra flags or hacks for the compile to work.

The whole point of MacPorts is to provide binary packages (tools or libs)
to be used by OS X apps. For libs you'll of course need to make sure ABI /
CPU arch are matching (You DON'T have to use the exact same GCC
build or version, though!). From this point it's safe to use MacPorts libs
from non-MacPorts apps. And this takes us back to the original point:

At this moment, for linuxdcpp to be built on OS X, you must use MacPorts
ports (maybe other porting efforts could also work, but we stick with MacPorts now).

Here there seems to be a user choice to use compiler toolchain:
1) GCC from XCode
2) GCC from MacPorts
+1) probably even a mixture would also work.

I chose 1) becaus...

[vszakats (vszakats)]

> In this case, `port` is probably adding /opt/local to the include paths automatically when it calls `gcc`. Did you use a Portfile to
> build LinuxDC++ or did you just use XCode gcc directly? (I'm guessing the latter?)

I used GCC directly. I used the exact steps you find in current instructions,
and after having installed XCode, its included GCC 4.2.1 is the default GCC
compiler on my system.

> overwriting system parts could be fatal, but for sure not very clean thing to do.
> BTW, MacPorts will never overwrite system parts because everything is in /opt/local. This won't affect the rest of your system at
> all.

I know, this was exactly my point also.

[ Besides MacPorts, there is also Fink as a package manager, which uses /sw dir
to store its files. I didn't make tests with it and linuxdcpp though. ]

[Ximin Luo (infinity0)]

vszakats wrote:
> Well, if your aim is to include linuxdcpp as a MacPorts port, which is a reasonable
> goal, you should naturally deal with some extra problems, but currently linuxdcpp
> isn't available as standard port

Sorry, this is what I meant. We created our own custom MacPorts repository and
LinuxDC++ Portfile. I realise my first post was a bit confusing now.

>> Could be that some ports have special requirements, but please don't form the general idea around a few exceptions.
>> This is the *rule*, not the exception. In any source-based packaging system, packages have a "build-depends" list of
>> packages from the same packaging system, that are required to build the package. The build is done using the packaging
>> system's own toolchain. Eg. dpkg-buildpackage for debian, port for MP.
>
> IMO you're confusing the packaging system with the compiler toolchain.

I'm talking about the packaging toolchain, which builds packages. This sits on
top of the compiler toolchain and automatically configures things like compiler
flags, dependencies, etc.

Using the compiler directly is usually not done for distribution forms of free
software - instead, we make a packaging metafile (for MacPorts this is the
Portfile) and call the packaging system's toolchain to build it automatically.

> The whole point of MacPorts is to provide binary packages (tools or libs)
> to be used by OS X apps. For libs you'll of course need to make sure ABI /
> CPU arch are matching (You DON'T have to use the exact same GCC
> build or version, though!). From this point it's safe to use MacPorts libs
> from non-MacPorts apps. And this takes us back to the original point:

The whole point of MacPorts is "to design an easy-to-use system for compiling,
installing, and upgrading either command-line, X11 or Aqua based open-source
software on the Mac OS X operating system."

Using MacPorts libraries to compile standalone (as in not a MacPorts package)
software is basically a hack, and unsupported by the MacPorts team. It's like
using libraries from a .deb package to compile a binary for a .rpm package. If
it works, then it's only by chance that all the files happen to be in the same
locations under both systems.

Maybe this isn't a problem for Mac, I don't know, but it's not a good solution
in principle, especially if you intend to distribute the resulting binaries.

> I chose 1) because it's the more natural and simple. This is what's covered
> in the instructions. (which BTW, you're not obliged to use, I've sent it in my
> free time, just to add some minor help to this project)

Sorry, I know you're trying to help. I suppose the instructions are fine if you
understand what you're doing, but I was looking at them through the eyes of a
Mac user who "just wants to run LinuxDC++" and doesn't understand the concept
of a packaging system, and the potential problems of mixing software from two.

The best solution would be for someone to maintain an official MacPorts
package. Do you want to do it? As I said, I have a Portfile already - it'll
need a few tweaks, but it's mostly in working condition. I don't want to do it
myself because I don't actually own a Mac so can't tes...

[Ximin Luo (infinity0)]

> Well, if your aim is to include linuxdcpp as a MacPorts port, which is a reasonable
> goal, you should naturally deal with some extra problems, but currently linuxdcpp
> isn't available as standard port

Sorry, this is what I meant. We created our own custom MacPorts repository and
LinuxDC++ Portfile. I realise my first post was a bit confusing now.

>> Could be that some ports have special requirements, but please don't form the general idea around a few exceptions.
>> This is the *rule*, not the exception. In any source-based packaging system, packages have a "build-depends" list of
>> packages from the same packaging system, that are required to build the package. The build is done using the packaging
>> system's own toolchain. Eg. dpkg-buildpackage for debian, port for MP.
>
> IMO you're confusing the packaging system with the compiler toolchain.

I'm talking about the packaging toolchain, which builds packages. This sits on
top of the compiler toolchain and automatically configures things like compiler
flags, dependencies, etc.

Using the compiler directly is usually not done for distribution forms of free
software - instead, we make a packaging metafile (for MacPorts this is the
Portfile) and call the packaging system's toolchain to build it automatically.

> The whole point of MacPorts is to provide binary packages (tools or libs)
> to be used by OS X apps. For libs you'll of course need to make sure ABI /
> CPU arch are matching (You DON'T have to use the exact same GCC
> build or version, though!). From this point it's safe to use MacPorts libs
> from non-MacPorts apps. And this takes us back to the original point:

The whole point of MacPorts is "to design an easy-to-use system for compiling,
installing, and upgrading either command-line, X11 or Aqua based open-source
software on the Mac OS X operating system."

Using MacPorts libraries to compile standalone (as in not a MacPorts package)
software is basically a hack, and unsupported by the MacPorts team. It's like
using libraries from a .deb package to compile a binary for a .rpm package. If
it works, then it's only by chance that all the files happen to be in the same
locations under both systems.

Maybe this isn't a problem for Mac, I don't know, but it's not a good solution
in principle, especially if you intend to distribute the resulting binaries.

> I chose 1) because it's the more natural and simple. This is what's covered
> in the instructions. (which BTW, you're not obliged to use, I've sent it in my
> free time, just to add some minor help to this project)

Sorry, I know you're trying to help. I suppose the instructions are fine if you
understand what you're doing, but I was looking at them through the eyes of a
Mac user who "just wants to run LinuxDC++" and doesn't understand the concept
of a packaging system, and the potential problems of mixing software from two.

The best solution would be for someone to maintain an official MacPorts
package. Do you want to do it? As I said, I have a Portfile already - it'll
need a few tweaks, but it's mostly in working condition. I don't want to do it
myself because I don't actually own a Mac so can't test updates easily...

[vszakats (vszakats)]

> Sorry, this is what I meant. We created our own custom MacPorts repository and
> LinuxDC++ Portfile. I realise my first post was a bit confusing now.

Okay that clears it up, BTW the first thing I checked was whether
MacPorts hosts linuxdcpp, but couldn't find it. It would be nice
if it would be there.

> I'm talking about the packaging toolchain, which builds packages. This sits on
> top of the compiler toolchain and automatically configures things like compiler
> flags, dependencies, etc.

> Using the compiler directly is usually not done for distribution forms of free
> software - instead, we make a packaging metafile (for MacPorts this is the
> Portfile) and call the packaging system's toolchain to build it automatically.

In this context it makes perfect sense.

> The whole point of MacPorts is "to design an easy-to-use system for compiling,
> installing, and upgrading either command-line, X11 or Aqua based open-source
> software on the Mac OS X operating system."
>
> Using MacPorts libraries to compile standalone (as in not a MacPorts package)
> software is basically a hack, and unsupported by the MacPorts team. It's like
> using libraries from a .deb package to compile a binary for a .rpm package. If
> it works, then it's only by chance that all the files happen to be in the same
> locations under both systems.

It's not really about locations IMO, since this is quite easy to solve
in most cases. But rather the potential incompatibilities between "ports"
and system libs.

> Maybe this isn't a problem for Mac, I don't know, but it's not a good solution
> in principle, especially if you intend to distribute the resulting binaries.

I agree there can be potential issues with distribution, but I haven't
dealt with this part yet, so can't tell anything meaningful.

Anyhow, currently this is the only reasonable way to create OS X apps
NOT hosted by MacPorts (which may happen easily with ones app) and
at the same time building upon the many available libs in a straightforward
way. Simply, unlike on Linux, there'll always be a boundary between
system hosted libs and "extras" provided by MacPorts / Fink. The other way
is to build non-system packages directly from source, with all the pros and
cons. (Plus, MacPorts can't really think seriously that developers will build
solely upon MacPorts tools to build final apps.)

Still, my LinuxDC++ build works perfectly in above mixture. Maybe it's a
hack, but it works.

> Sorry, I know you're trying to help. I suppose the instructions are fine if you
> understand what you're doing, but I was looking at them through the eyes of a
> Mac user who "just wants to run LinuxDC++" and doesn't understand the concept
> of a packaging system, and the potential problems of mixing software from two.

Well, it's an INSTALL instructions doc :) Building stuff from source obviously
requires some level of involvement in these sorts of things. Believe me, I'd have
also been happier if I could just simply download a .dmg file and drag&drop
Linux DC++.app on my Desktop.

> The best solution would be for someone to maintain an official MacPorts
> package. Do you want to do it? As I said, I have a Portfile alr...

[JackDavies (davies828)]

Hello,

I'm having trouble working out what to do here. I am installing linuxdcpp via macports, and get the same error as above. I've done export CXXFLAGS=-I/opt/local/include but I'm not sure in which directory I should be running the scons commands, or what I should be specifying for PREFIX=

Any help would be great, I'm not that experienced with macports.

Regards,
Jack Davies

[Ximin Luo (infinity0)]

If you have time to maintain a MacPorts package, I'm happy to supply the packaging files to you. You need to have a general idea of how the MacPorts packaging system works.

Trying to compile linuxdcpp directly using scons by mixing MacPorts libraries with system libraries is convoluted, and will give you various headaches depending on how your system is already set up.