statically linking libpthread is problematic

[junyer]

While investigating https://github.com/google/re2/issues/22, we discovered that statically linking libpthread is problematic since Ubuntu 14.10. I suspect that the TLS (Thread-Local Storage) is not initialised properly, which results in hangs or crashes.

junyer@whalre:~$ lsb_release -rd
Description: Ubuntu 14.10
Release: 14.10
junyer@whalre:~$ apt-cache policy libc6-dev
libc6-dev:
  Installed: 2.19-10ubuntu2.3
  Candidate: 2.19-10ubuntu2.3
  Version table:
 *** 2.19-10ubuntu2.3 0
        500 http://au.archive.ubuntu.com/ubuntu/ utopic-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ utopic-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2.19-10ubuntu2 0
        500 http://au.archive.ubuntu.com/ubuntu/ utopic/main amd64 Packages
junyer@whalre:~$

[junyer (junyer)]

junyer@whalre:~$ g++ -Wall -Werror foo.cc -o foo -lpthread
junyer@whalre:~$ ./foo
foo: pthread_rwlock_init(&lock, NULL): 0
foo: pthread_rwlock_wrlock(&lock): 0
foo: pthread_rwlock_unlock(&lock): 0
foo: pthread_rwlock_wrlock(&lock): 0
foo: pthread_rwlock_unlock(&lock): 0
foo: pthread_rwlock_destroy(&lock): 0
junyer@whalre:~$ g++ -Wall -Werror foo.cc -o foo -static -lpthread
junyer@whalre:~$ ./foo
foo: pthread_rwlock_init(&lock, NULL): 0
foo: pthread_rwlock_wrlock(&lock): 0
foo: pthread_rwlock_unlock(&lock): 0
^C
junyer@whalre:~$ g++ -Wall -Werror foo.cc -o foo -static -Wl,--whole-archive -lpthread -Wl,--no-whole-archive
junyer@whalre:~$ ./foo
foo: pthread_rwlock_init(&lock, NULL): 0
foo: pthread_rwlock_wrlock(&lock): 0
foo: pthread_rwlock_unlock(&lock): 0
foo: pthread_rwlock_wrlock(&lock): 0
foo: pthread_rwlock_unlock(&lock): 0
foo: pthread_rwlock_destroy(&lock): 0
junyer@whalre:~$

[junyer (junyer)]

Note that the second case hung and thus required me to hit Ctrl-C.

The proximate cause seems to be that the TID (Thread ID) in the TLS (Thread-Local Storage) is not initialised properly:

1. The first pthread_rwlock_wrlock() call sets __writer to the TID, which is 0.
2. The first pthread_rwlock_unlock() call thinks that we had a reader lock because __writer is 0, so it decrements __nr_readers from 0 to 4294967295.
3. The second pthread_rwlock_wrlock() call hangs because it's waiting forever to be woken by a reader that never existed.

[junyer (junyer)]

Note that the third case worked because the use of `-Wl,--whole-archive -lpthread -Wl,--no-whole-archive' is a workaround that evidently results in the TLS (Thread-Local Storage) being initialised properly.

The root cause of this bug is currently uncertain, but https://gcc.gnu.org/ml/gcc-help/2010-05/msg00029.html could be relevant.