clang armhf in trusty simply doesn't work

[Niall Douglas]

I'm running Ubuntu 14.04 on armhf. clang is from the standard repo:

root@tegra-ubuntu:~# apt-cache show clang
Package: clang
Priority: optional
Section: universe/devel
Installed-Size: 27
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: LLVM Packaging Team <email address hidden>
Architecture: armhf
Source: llvm-defaults (0.21ubuntu1)
Version: 1:3.4-0ubuntu1
Replaces: clang (<< 3.2-1~exp2)
Depends: clang-3.4 (>= 3.4~rc3-1~)
Filename: pool/universe/l/llvm-defaults/clang_3.4-0ubuntu1_armhf.deb
Size: 2478
MD5sum: eadb3f7c344e364480bdf7fdd0f698ab
SHA1: ec4d0bfcad42bf536ee8d9257f8930dd32e1a261
SHA256: f6d9a8fbfc06d93cd4c1a2437114454cc1eec805a635efc490a7e1e6eec2b3de
Description-en: C, C++ and Objective-C compiler (LLVM based)
 Clang project is a C, C++, Objective C and Objective C++ front-end
 for the LLVM compiler. Its goal is to offer a replacement to the GNU Compiler
 Collection (GCC).
 .
 Clang implements all of the ISO C++ 1998 and 2001 standards and also provides
 a partial support of C++1y.
 .
 This is a dependency package providing the default clang compiler.
Description-md5: ea1f164ac255f39c6ec78685f71ef19b
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

Anyway, it does not produce working output. Here are some examples:

https://ci.nedprod.com/view/Boost.AFIO/job/Boost.AFIO%20Build%20POSIX_ARM_clang%203.4/9/consoleFull

This log shows how clang has the wrong default target CPU, so it outputs unsupported ARM instructions (see the end after the warnings spew).

If I force the target cpu to a cortex-a15 it now at least compiles and links and starts to run unit tests:

https://ci.nedprod.com/view/Boost.AFIO/job/Boost.AFIO%20Test%20POSIX_ARM_clang%203.4/7/consoleFull

... but hangs in the first unit test, and is timed out. Before you think it the unit tests, here is the exact same thing for GCC 4.8:

https://ci.nedprod.com/view/Boost.AFIO/job/Boost.AFIO%20Test%20POSIX_ARM_GCC%204.8/7/console

The unit tests all run and pass as expected. Everything also works fine on x86 and x64 with clang 3.4 on Ubuntu 14.04, this appears to be an armhf misconfiguration.

Niall

[Dimitri John Ledkov (xnox)]

Given that you see this failure, maybe you should report it to Boost upstream, since beta version of boost is attempted to be compiled here? And it's claimed that upstream supports clang 3.4.

In Ubuntu, boost 1.54 is fully supported in trusty with the default gcc toolchain.

[Niall Douglas (s-launchpad-k)]

FYI I am one of the Boost core dev team, and this problem emerged during our prerelease testing of Boost 1.56.

I thought a reasonable test of who is at fault would be if I purged all trace of clang and llvm from the system, and then download these prebuilt binaries from llvm:

http://llvm.org/releases/3.4.2/clang+llvm-3.4.2-armv7a-linux-gnueabihf.tar.xz

I unpack these into /usr/local and ensure perms and ownership are reset to correct values.

If I now repeat the earlier test I see:

1. I no longer need to manually override -mcpu, because the correct local CPU is now chosen and the atomics compile instead of the assembler not recognising instructions. I would assume the default triple of armv7a-linux-gnueabihf helps a lot here.

2. All the unit tests still hang in the same place as with your clang.

This suggests to me that something is very wrong with Boost on clang on armhf.

I'll investigate more this end, but fixing the default cpu such that it supports atomics would be a great help, so I'll leave this issue open for now. I'll come back if I figure out the cause of the hangs and it's related to packaging.

Niall

[Dimitri John Ledkov (xnox)]

On 30 July 2014 14:59, Niall Douglas <email address hidden> wrote:
> FYI I am one of the Boost core dev team, and this problem emerged during
> our prerelease testing of Boost 1.56.
>
> I thought a reasonable test of who is at fault would be if I purged all
> trace of clang and llvm from the system, and then download these
> prebuilt binaries from llvm:
>
> http://llvm.org/releases/3.4.2/clang+llvm-3.4.2-armv7a-linux-
> gnueabihf.tar.xz
>
> I unpack these into /usr/local and ensure perms and ownership are reset
> to correct values.
>
> If I now repeat the earlier test I see:
>
> 1. I no longer need to manually override -mcpu, because the correct
> local CPU is now chosen and the atomics compile instead of the assembler
> not recognising instructions. I would assume the default triple of
> armv7a-linux-gnueabihf helps a lot here.
>

I believe our default clang triplet is the same as debian's, as we
don't modify it.
Which I think should be something is v7 without NEON.
Have you tried debian's armhf clang-3.4?
I'll investigate further to see what is the baseline cpu all three
target (ubuntu, debian and llvm upstream prebuilts)

> 2. All the unit tests still hang in the same place as with your clang.
>

=((((( *sad*

> This suggests to me that something is very wrong with Boost on clang on
> armhf.
>
> I'll investigate more this end, but fixing the default cpu such that it
> supports atomics would be a great help, so I'll leave this issue open
> for now. I'll come back if I figure out the cause of the hangs and it's
> related to packaging.
>

Ack.

--
Regards,

Dimitri.

[Niall Douglas (s-launchpad-k)]

BTW I think some history may be at work here. clang didn't support native cpu detection back when I was the chief clang person at BlackBerry, it may well still not. I would assume the default choice of arm-linux-gnueabihf, judging from google searching, was mainly driven by RaspPI enthusiasts who obviously wanted an ARMv6 clang. Now, a Cortex-A8 supports atomics just fine, but it could be that arm-linux-gnueabihf means one thing to GCC (an ARMv6) and something rather less (an ARM7tdmi for example, the lowest supported ARM CPU) to clang which is ARMv4 or something.

It might be worth experimenting with a default triple for clang of armv6-linux-gnueabihf instead. After all, Ubuntu surely does not work on systems without atomics. I'll give this a try on Saturday and see what happens.

BTW there is something real weird in Boost on clang armhf, exception throw catches aren't reliable, this was causing an infinite loop of terminate(). I'd say 1.56 is going to ship anyway, I'll see if I can figure out the problem for 1.57, again this will be Saturday.

Niall

[Niall Douglas (s-launchpad-k)]

Good news on this, though it took a day of head scratching to figure it out.

It turns out that ARM clang 3.3 and earlier implements C++ exceptions using the inefficient SJLJ exception ABI - you know, the one that calls setjmp all the time for every place an unwind might happen. Anyway, they finally got round to implementing the ARM EHABI exception ABI which is zero runtime cost and they went ahead and turned it on by default in 3.4, or at least it is being turned on by default in the Debian/Ubuntu binaries as well as the LLVM binaries.

Unfortunately, it was very broken indeed. It produces ARM binaries which simply cannot catch non-trivial C++ exceptions, though otherwise work fine. This caused Boost.Thread when interrupting a thread wait to enter an infinite loop at described above, indeed if you EVER catch a type with RTTI it infinite loops.

Fortunately, Chromium realised this shortly after the 3.4 release, and they've been hard at work making a EHABI implementation which actually works for 3.5. I just finished compiling 3.5 from trunk and I can confirm that all Boost.AFIO unit test pass swimmingly with it for armhf.

You can read more about the 3.5 ARM exception handling improvements at http://llvm.org/docs/ReleaseNotes.html#changes-to-the-arm-backend.

Can I recommend you upcall this to Debian as well and recommend that they mark clang 3.4 as broken on ARM? Either that or have them patched to disable EHABI exception handling?

Niall