pthread_setattr_default_np race condition with pthread_create.

[Maxim Egorushkin]

pthread_setattr_default_np race condition with pthread_create.

When creating a new thread with pthread_create and a NULL pthread_attr_t* argument, the pthread_attr_t* argument must come from the last call to pthread_setattr_default_np preceding the call to pthread_create.

It seems that there is a race condition in glibc implementation such that pthread_attr_t* argument comes from a pthread_setattr_default_np call that succeeds a pthread_create call.

Example:

[max@supernova:~/src/test] $ g++ --version
g++ (Ubuntu 8.3.0-6ubuntu1~18.04.1) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[max@supernova:~/src/test] $ cat test.cc
#include <cstdio>
#include <thread>
#include <system_error>
#include <pthread.h>

void set_default_thread_stack_size(size_t size) {
    pthread_attr_t attr;
    if(int err = ::pthread_attr_init(&attr))
        throw std::system_error(err, std::system_category(), "pthread_attr_init");
    if(int err = ::pthread_attr_setstacksize(&attr, size))
        throw std::system_error(err, std::system_category(), "pthread_attr_setstacksize");
    if(int err = ::pthread_setattr_default_np(&attr))
        throw std::system_error(err, std::system_category(), "pthread_setattr_default_np");
    if(int err = ::pthread_attr_destroy(&attr))
        throw std::system_error(err, std::system_category(), "pthread_attr_destroy");
}

void another_thread() {
    pthread_attr_t attr;
    if(int err = ::pthread_attr_init(&attr))
        throw std::system_error(err, std::system_category(), "pthread_attr_init");
    size_t stack_size;
    if(int err = ::pthread_attr_getstacksize(&attr, &stack_size))
        throw std::system_error(err, std::system_category(), "pthread_attr_getstacksize");
    if(int err = ::pthread_attr_destroy(&attr))
        throw std::system_error(err, std::system_category(), "pthread_attr_destroy");
    std::printf("thread stack size is %zu\n", stack_size);
}

int main() {
    another_thread();
    std::thread t0(another_thread);
    set_default_thread_stack_size(1024 * 1024);
    std::thread t1(another_thread);
    t0.join();
    t1.join();
}

[max@supernova:~/src/test] $ g++ -std=gnu++14 -W{all,extra,error} -O3 -march=native -o test test.cc -pthread

[max@supernova:~/src/test] $ ldd test
 linux-vdso.so.1 (0x00007ffd74d74000)
 libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f597ed00000)
 libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f597eae8000)
 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f597e8c9000)
 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f597e4d8000)
 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f597e13a000)
 /lib64/ld-linux-x86-64.so.2 (0x00007f597f28d000)

[max@supernova:~/src/test] $ /lib/x86_64-linux-gnu/libc.so.6
GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1) stable release version 2.27.
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 7.3.0.
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<https://bugs.launchpad.net/ubuntu/+source/glibc/+bugs>.

[max@supernova:~/src/test] $ ./test
thread stack size is 8720384
thread stack size is 1048576
thread stack size is 1048576

The above output is incorrect. The expected output is:

thread stack size is 8720384
thread stack size is 8720384
thread stack size is 1048576

If `t0.join();` line is moved to right after `std::thread t0(another_thread);` line then the code produces the correct output.

It appears that setting the default pthread_attr_t is not atomic/ordered with respect to pthread_create calls.

[Florian Weimer (fweimer)]

I assume the code in another_thread attempts to obtain the size of the current thread. But the code just reads back the default stack size, which may have changed due to the set_default_thread_stack_size call.

One way to obtain the size of the current thread stack is this call:

      if(int err = ::pthread_getattr_np(pthread_self(), &attr))
                throw std::system_error(err, std::system_category(), "pthread_getattr_np");

This should give consistent results all the time.

[Maxim Egorushkin (max0x7ba)]

> I assume the code in another_thread attempts to obtain the size of the current thread.

The intention is to use pthread_setattr_default_np followed by a call to std::thread contructor, because the latter doesn't allow specifying the thread attribute.

So, instead of:

pthread_attr_t attr;
<initialize attr>
pthread_create(..., &attr, ...)

Use:

pthread_attr_t attr;
<initialize attr>
pthread_setattr_default_np(&attr)
std::thread t(...);

But this race condition prevents such usage of pthread_setattr_default_np.

[Florian Weimer (fweimer)]

No, the race condition is in your stack size test (which is written incorrectly), not in the stack creation.

However, this code is inherently race-prone anyway:

pthread_attr_t attr;
<initialize attr>
pthread_setattr_default_np(&attr)
std::thread t(...);

You just need something else in the process using std::thread which does not go through the wrapper you wrote to trigger this.

[Maxim Egorushkin (max0x7ba)]

> No, the race condition is in your stack size test (which is written incorrectly), not in the stack creation.

You are right, the test should use pthread_getattr_np instead of pthread_attr_init.