Comment 1 for bug 1760471

Revision history for this message
Victor Stinner (vstinner) wrote :

The performance difference likely comes from close_fds=True of subprocess.Popen. On Python 2, Popen calls close(fd) on all file descriptors from 3 to SC_OPEN_MAX. On my Fedora 27 "host", SC_OPEN_MAX is 1,024. But in docker, SC_OPEN_MAX is... 1,048,576: 1,000x larger.

On Python 3, Popen is smarter. On Linux, it lists the content of /proc/self/fd/ to only close open file descriptors. It doesn't depend on SC_OPEN_MAX value.

The quick workaround is to use close_fds=False, but this change can have an impact on security. close_fds=True is a cheap protection to prevent leaking file descriptors to child processes.

See also my PEP 446 which made file descriptors not inheritable by default in Python 3.4.