The performance difference likely comes from close_fds=True of subprocess.Popen. On Python 2, Popen calls close(fd) on all file descriptors from 3 to SC_OPEN_MAX. On my Fedora 27 "host", SC_OPEN_MAX is 1,024. But in docker, SC_OPEN_MAX is... 1,048,576: 1,000x larger.
On Python 3, Popen is smarter. On Linux, it lists the content of /proc/self/fd/ to only close open file descriptors. It doesn't depend on SC_OPEN_MAX value.
The quick workaround is to use close_fds=False, but this change can have an impact on security. close_fds=True is a cheap protection to prevent leaking file descriptors to child processes.
See also my PEP 446 which made file descriptors not inheritable by default in Python 3.4.
The performance difference likely comes from close_fds=True of subprocess.Popen. On Python 2, Popen calls close(fd) on all file descriptors from 3 to SC_OPEN_MAX. On my Fedora 27 "host", SC_OPEN_MAX is 1,024. But in docker, SC_OPEN_MAX is... 1,048,576: 1,000x larger.
On Python 3, Popen is smarter. On Linux, it lists the content of /proc/self/fd/ to only close open file descriptors. It doesn't depend on SC_OPEN_MAX value.
The quick workaround is to use close_fds=False, but this change can have an impact on security. close_fds=True is a cheap protection to prevent leaking file descriptors to child processes.
See also my PEP 446 which made file descriptors not inheritable by default in Python 3.4.