libmemcached

libmemcached does not compile on OSX Mavericks pre-release version

Bug #1216521 reported by Jarek Potiuk on 2013-08-25

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	libmemcached	Fix Committed	Medium	Brian Aker	libmemcached 1.0.18

Bug Description

I tried to install libmemcached on OSX Mavericks with brew, and finally got it working with some workarounds, but I am not sure if my solution is good, probably you can provide better fix (or maybe it's Apple's pre-release software problem, it's hard for me to judge). The history of the problem is described in detail in the github issue for homebrew: https://github.com/mxcl/homebrew/issues/20635

Here is the summary/excerpt:

When you try to install libmemcached using brew it fails with missing tr1 include. Apparently other libraries have similar problem.
It worked fine on Mountain Lion.

Steps to Reproduce:

Have XCode installed
Install homebrew ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
type 'brew install libmemcached'
Expected Results:

The library should be compiled and installed.

Actual Results:

The following output is displayed:

==> Downloading https://launchpad.net/libmemcached/1.0/1.0.17/+download/libmemcached-1.0.17.tar.gz
Already downloaded: /Library/Caches/Homebrew/libmemcached-1.0.17.tar.gz
==> ./configure --prefix=/usr/local/Cellar/libmemcached/1.0.17
==> make install
# include <tr1/cinttypes>
^
1 error generated.
make[1]: *** [libmemcached/libmemcachedinternal_libmemcachedinternal_la-instance.lo] Error 1
make: *** [install] Error 2

Workaround:

I managed to workaround it by adding path to OSX's includes containing tr1 includes as well as adding libstdc++ directly to LIBS in order to avoid some missing symbols (https://gist.github.com/potiuk/6332854)

Possible reasons:

Some analysis what could go wrong. I am not an expert in C++ libraries and compilation (i have not worked with C++ for some 10 years or so), but I've learned a ton about it and brew mechanisms in the process ;) (BTW. Thanks to brew team for --verbose --debug switches. they made my day!). Besides missing include path, we also have the missing libstdc++. From what I understand libc++ (which was already linked) is a c++11 implementation of standard libraries. OSX's llvm is already 100% compliant with it but some symbols were missing (see below for some example stdlib symbols missing in libc++)

So either libmemcached is 'almost c++11 compliant' and still uses some deprecated C++ stdlib methods or the library in OSX missed some methods in this prerelease version.

I presume it's a libmemcached problem because just putting tr1 includes on the include path causes some macro redefnition warnings, which (guts feeling) - are caused by mem_config.h using some old stdlib includes.

Macro redefinitions:

In file included from /Applications/Xcode5-DP6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1/tr1/cinttypes:38:
/Applications/Xcode5-DP6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1/tr1/cstdint:42:9: warning: '__STDC_LIMIT_MACROS' macro redefined
#define __STDC_LIMIT_MACROS
^
./mem_config.h:581In file included from :9libmemcached/instance.cc:: 38:
In file included from note: ./libmemcached/common.h:previous114 :
definitionIn file included from ./libmemcached-1.0/memcached.his: 46here:
In file included from
/Applications/Xcode5-DP6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1/tr1/cinttypes:38:
/Applications/Xcode5-DP6.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1/tr1/cstdint#define __STDC_LIMIT_MACROS 1:
42: ^9
: warning: '__STDC_LIMIT_MACROS' macro redefined
#define __STDC_LIMIT_MACROS
^
./mem_config.h:581:9: note: previous definition is here

Missing symbols:

Undefined symbols for architecture x86_64:
  "std::basic_ios<char, std::char_traits<char> >::widen(char) const", referenced from:
      _main in core_count.o
      help_command(char const*, int, int, option const*) in core_count.o
  "std::ostream::put(char)", referenced from:
      _main in core_count.o
      help_command(char const*, int, int, option const*) in core_count.o
  "std::ostream::flush()", referenced from:
      _main in core_count.o
      help_command(char const*, int, int, option const*) in core_count.o
  "std::ostream& std::ostream::_M_insert<unsigned long>(unsigned long)", referenced from:
      _main in core_count.o
  "std::ostream::operator<<(int)", referenced from:
      _main in core_count.o
      help_command(char const*, int, int, option const*) in core_count.o

Tags:

Revision history for this message

Jarek Potiuk (potiuk) wrote on 2013-08-25:

I also reported the problem to Apple: 14827477 (no link, because I believe you can only see your own problems reported to Apple :( ), I am not sure who is best to address it, but If you happen to know what Apple should correct in order to have it fixed, I might update the ticket there.

Revision history for this message

Dominik (t-o-p-p-i) wrote on 2013-09-02:

Download full text (3.5 KiB)

This is expected behavior, according to... some people at some company.
There are however two paths to get going again.

A little background:
The tr1 headers were an experiment by the C++ standards committee, nothing more. They are not part of any standard. Apple is involved with the standards committee in an effort to make future committee experiments more obvious, and less likely to be mistaken for stable parts of a standard.

Additionally Apple is in the middle of transitioning from a gcc-based implementation of the std::lib (based on gcc-4.2) to a new-from-the-ground-up implementation that fully supports the new C++11 standard. This new implementation is also open source and the open source project can be found here: http://libcxx.llvm.org

Since 10.7/Lion, two implementations of the std::lib, the old gcc libstdc++, and this new llvm libc++ have been shipped.
With -lets say "newer"- OSes, the new libc++ is becoming the default used by the compiler clang.
This switch in the default std::lib is what's causing the symptom described in this ticket.

The older libstdc++ is still available and still has the same tr1 headers it always has.
The newer libc++ lacks the tr1 experiment. However most of tr1 was standardized with C++11, and now lives in namespace std, and in headers not prefixed with tr1/.

The C++ committee purposefully used the experimental std::tr1 namespace for the experiment so that they could (if need be) modify the tr1 components as they were standardized. And indeed this has happened. For the most part you will not notice any difference as you migrate from tr1 implementations to the C++11 std implementations. However there do exist a few corner cases. I would not worry about these corner cases. If you hit one, it will almost certainly come in the form of a compile-time error.

You are faced with two options:

1. Migrate your code from tr1 to std. I will show how to do this below. It is very easy. And it can be done in a manner in which your code is portable to both the old libstdc++ and the new libc++. This is the path "people at some company recommend".

2. Override the default std::lib choice and specifically choose the older gcc-4.2 based libstdc++. This is the easier path. But if this path is chosen, you are ultimately simply delaying the (relatively easy) work of doing step 1.

To override the default on the command line, add -stdlib=libstdc++ to both the compile and link command lines. You can also specifically choose the newer libc++ with -stdlib=libc++.

To do (1), the first step is to auto-detect which std::lib you are using. If you have already included any std header, you can do:

#ifdef _LIBCPP_VERSION

// using the new libc++

#else

// using the old libstdc++

#endif

If you have not yet included a std header, and need to include one gratuitously to find out whether or not it will define _LIBCPP_VERSION, do:

#include <ciso646> // detect std::lib

<ciso646> is a wonderful little C++ header that is specified to do absolutely nothing. So it is very cheap to include, and it will define _LIBCPP_VERSION if you are using the newer libc++, and otherwise won't.

Once you know which std::lib you are...

This is expected behavior, according to... some people at some company.
There are however two paths to get going again.

A little background:
The tr1 headers were an experiment by the C++ standards committee, nothing more.  They are not part of any standard.  Apple is involved with the standards committee in an effort to make future committee experiments more obvious, and less likely to be mistaken for stable parts of a standard.

Since 10.7/Lion, two implementations of the std::lib, the old gcc libstdc++, and this new llvm libc++ have been shipped. 
With -lets say "newer"- OSes, the new libc++ is becoming the default used by the compiler clang.
This switch in the default std::lib is what's causing the symptom described in this ticket.

The older libstdc++ is still available and still has the same tr1 headers it always has.
The newer libc++ lacks the tr1 experiment.  However most of tr1 was standardized with C++11, and now lives in namespace std, and in headers not prefixed with tr1/.

The C++ committee purposefully used the experimental std::tr1 namespace for the experiment so that they could (if need be) modify the tr1 components as they were standardized.  And indeed this has happened.  For the most part you will not notice any difference as you migrate from tr1 implementations to the C++11 std implementations.  However there do exist a few corner cases.  I would not worry about these corner cases.  If you hit one, it will almost certainly come in the form of a compile-time error.

You are faced with two options:

1.  Migrate your code from tr1 to std.  I will show how to do this below.  It is very easy.  And it can be done in a manner in which your code is portable to both the old libstdc++ and the new libc++.  This is the path "people at some company recommend".

2.  Override the default std::lib choice and specifically choose the older gcc-4.2 based libstdc++.  This is the easier path.  But if this path is chosen, you are ultimately simply delaying the (relatively easy) work of doing step 1.

To override the default on the command line, add -stdlib=libstdc++ to both the compile and link command lines.  You can also specifically choose the newer libc++ with -stdlib=libc++.

To do (1), the first step is to auto-detect which std::lib you are using.  If you have already included any std header, you can do:

#ifdef _LIBCPP_VERSION

//  using the new libc++

#else

//  using the old libstdc++

#endif

If you have not yet included a std header, and need to include one gratuitously to find out whether or not it will define _LIBCPP_VERSION, do:

#include <ciso646>  // detect std::lib

<ciso646> is a wonderful little C++ header that is specified to do absolutely nothing.  So it is very cheap to include, and it will define _LIBCPP_VERSION if you are using the newer libc++, and otherwise won't.

Once you know which std::lib you are using, it becomes very easy to do things such as:

#ifdef _LIBCPP_VERSION
//  using the new libc++

#include <memory>
typedef std::shared_ptr<MyType> MyTypePtr;

#else
//  using the old libstdc++

#include <tr1/memory>
typedef std::tr1::shared_ptr<MyType> MyTypePtr;

#endif

Once done, it's easy to switch between libstdc++ and libc++ for testing purposes.

Revision history for this message

Jarek Potiuk (potiuk) wrote on 2013-09-02:

Yeah I got exactly the same copy&pasted answer in my issue :)..

Revision history for this message

Xiyue Deng (manphiz) wrote on 2013-10-06:

Beware that Dominik's last suggestion can be problematic, as it is undefined behavior to include C++11 headers in C++03 code and should be forbidden by proper compilers (which is likely to happen in GCC). The real fix is still to follow Dominik's first 2 suggestions: either migrate the whole code base to C++11, or fallback to libstdc++ before that happens.

Brian Aker (brianaker) on 2013-10-11