gcc crashes python 2.3 in intrepid

[Roger Binns]

Binary package hint: gcc

If you compile Python 2.3.7 in Intrepid then the resulting binary crashes on startup. This has not been a problem before with Gutsy/Hardy. Intrepid current as of two hours ago on AMD 64. To repeat:

$ wget http://python.org/ftp/python/2.3.7/Python-2.3.7.tar.bz2
$ tar xfj Python-2.3.7.tar.bz2
$ cd Python-2.3.7
$ ./configure
$ make

The built binary is run during the build process. This is what I get:

*** buffer overflow detected ***: ./python terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x37)[0x36c64ff887]
/lib/libc.so.6[0x36c64fd750]
/lib/libc.so.6[0x36c64fde0b]
./python(PySys_SetArgv+0x14d)[0x49f06d]
./python(Py_Main+0x336)[0x411686]
/lib/libc.so.6(__libc_start_main+0xe6)[0x36c641e466]
./python[0x411259]
======= Memory map: ========
00400000-004f0000 r-xp 00000000 00:16 1553913 /tmp/problem/Python-2.3.7/python
006f0000-006f1000 r--p 000f0000 00:16 1553913 /tmp/problem/Python-2.3.7/python
006f1000-00718000 rw-p 000f1000 00:16 1553913 /tmp/problem/Python-2.3.7/python
00718000-0071c000 rw-p 00718000 00:00 0
01398000-0141f000 rw-p 01398000 00:00 0 [heap]
36c3e00000-36c3e1f000 r-xp 00000000 08:02 12861453 /lib/ld-2.8.90.so
36c401e000-36c401f000 r--p 0001e000 08:02 12861453 /lib/ld-2.8.90.so
36c401f000-36c4020000 rw-p 0001f000 08:02 12861453 /lib/ld-2.8.90.so
36c6400000-36c6569000 r-xp 00000000 08:02 12861457 /lib/libc-2.8.90.so
36c6569000-36c6768000 ---p 00169000 08:02 12861457 /lib/libc-2.8.90.so
36c6768000-36c676c000 r--p 00168000 08:02 12861457 /lib/libc-2.8.90.so
36c676c000-36c676d000 rw-p 0016c000 08:02 12861457 /lib/libc-2.8.90.so
36c676d000-36c6772000 rw-p 36c676d000 00:00 0
36c6800000-36c6802000 r-xp 00000000 08:02 12861462 /lib/libdl-2.8.90.so
36c6802000-36c6a02000 ---p 00002000 08:02 12861462 /lib/libdl-2.8.90.so
36c6a02000-36c6a03000 r--p 00002000 08:02 12861462 /lib/libdl-2.8.90.so
36c6a03000-36c6a04000 rw-p 00003000 08:02 12861462 /lib/libdl-2.8.90.so
36c6c00000-36c6c84000 r-xp 00000000 08:02 12861461 /lib/libm-2.8.90.so
36c6c84000-36c6e83000 ---p 00084000 08:02 12861461 /lib/libm-2.8.90.so
36c6e83000-36c6e84000 r--p 00083000 08:02 12861461 /lib/libm-2.8.90.so
36c6e84000-36c6e85000 rw-p 00084000 08:02 12861461 /lib/libm-2.8.90.so
36c7000000-36c7017000 r-xp 00000000 08:02 12861463 /lib/libpthread-2.8.90.so
36c7017000-36c7216000 ---p 00017000 08:02 12861463 /lib/libpthread-2.8.90.so
36c7216000-36c7217000 r--p 00016000 08:02 12861463 /lib/libpthread-2.8.90.so
36c7217000-36c7218000 rw-p 00017000 08:02 12861463 /lib/libpthread-2.8.90.so
36c7218000-36c721c000 rw-p 36c7218000 00:00 0
36cd000000-36cd016000 r-xp 00000000 08:02 12861502 /lib/libgcc_s.so.1
36cd016000-36cd216000 ---p 00016000 08:02 12861502 /lib/libgcc_s.so.1
36cd216000-36cd217000 r--p 00016000 08:02 12861502 /lib/libgcc_s.so.1
36cd217000-36cd218000 rw-p 00017000 08:02 12861502 /lib/libgcc_s.so.1
36cd400000-36cd4f1000 r-xp 00000000 08:02 79874701 /usr/lib/libstdc++.so.6.0.10
36cd4f1000-36cd6f1000 ---p 000f1000 08:02 79874701 /usr/lib/libstdc++.so.6.0.10
36cd6f1000-36cd6f8000 r--p 000f1000 08:02 79874701 /usr/lib/libstdc++.so.6.0.10
36cd6f8000-36cd6fa000 rw-p 000f8000 08:02 79874701 /usr/lib/libstdc++.so.6.0.10
36cd6fa000-36cd70d000 rw-p 36cd6fa000 00:00 0
36d2e00000-36d2e02000 r-xp 00000000 08:02 12861529 /lib/libutil-2.8.90.so
36d2e02000-36d3001000 ---p 00002000 08:02 12861529 /lib/libutil-2.8.90.so
36d3001000-36d3002000 r--p 00001000 08:02 12861529 /lib/libutil-2.8.90.so
36d3002000-36d3003000 rw-p 00002000 08:02 12861529 /lib/libutil-2.8.90.so
2af72668d000-2af726690000 rw-p 2af72668d000 00:00 0
2af7266ae000-2af7266f3000 rw-p 2af7266ae000 00:00 0
2af7266f4000-2af7267a7000 rw-p 2af7266f4000Aborted
make: *** [sharedmods] Error 134

In case you are wondering why I am doing this, I develop a Python extension module and support Python 2.3 onwards. Consequently my testing has to be against all supported Python versions.

[Matthias Klose (doko)]

probably will close as won't fix. does lowering the optimization level will help? does gcc-4.1 or gcc-4.2 dow work better for you?

[Roger Binns (ubuntu-rogerbinns)]

It works just under gcc-4.2. If I use -O0 under gcc-4.3 then it is also fine, but even -O1 is sufficient for the crash.

$ ./configure CC=gcc-4.2 # works
$ ./configure OPT="-g -O0" # works
$ ./configure OPT="-g -O1" # splat

Running gdb on the binary I get this:

(gdb) bt
#0 0x00000036c6432fd5 in raise () from /lib/libc.so.6
#1 0x00000036c6434b43 in abort () from /lib/libc.so.6
#2 0x00000036c6473fa8 in ?? () from /lib/libc.so.6
#3 0x00000036c64ff887 in __fortify_fail () from /lib/libc.so.6
#4 0x00000036c64fd750 in __chk_fail () from /lib/libc.so.6
#5 0x00000036c64fde0b in __realpath_chk () from /lib/libc.so.6
#6 0x0000000000478733 in PySys_SetArgv (argc=<value optimized out>, argv=0x7fff2a3f6868) at /usr/include/bits/stdlib.h:44
#7 0x000000000041192e in Py_Main (argc=4, argv=0x7fff2a3f6858) at Modules/main.c:386
#8 0x00000000004112a5 in main (argc=485, argv=0x1e5) at Modules/python.c:23

To get to the code that is crashing, this is effectively what is happening:

    char *argv0="./setup.py"; /* a member of argv passed to main */
    char fullpath[PATH_MAX];

    if(argc > 0 && argv0 != NULL)
       if (realpath(argv0, fullpath))
          argv0=fullpath;

The realpath call is where things are dying. PATH_MAX is 1024 hence that is the size of fullpath. The realpath output will trivially fit in fullpath. Either the realpath implementation really does need more than 1kb of space to do its calculations or this is a false positive with the fortify stuff getting it wrong.

The current Python code uses MAXPATHLEN instead of PATH_MAX and doesn't trigger whatever voodoo is whining in realpath. This is a workaround:

$ ./configure OPT="-O3 -DPATH_MAX=4096"

[Roger Binns (ubuntu-rogerbinns)]

As an addendum the workaround above only works during an early part of the build process. The full process fails with the same error above. Heck it even fails with -DPATH_MAX=16384. My conclusion is that the code in stdlib.h is pointlessly causing a failure without looking at the actual size of the buffer, or even how much of it is used. That is most definitely a bug.

[Ori Peleg (orip)]

There's a workaround without disabling optimization: http://orip.org/2008/10/building-python-235-on-ubuntu-intrepid.html

Short version: add BASECFLAGS=-U_FORTIFY_SOURCE to the 'configure' command line.

[Roger Binns (ubuntu-rogerbinns)]

Thanks for the other workaround. The real bug is that gcc 4.3 with FORTIFY causes a binary to fail if there could be a buffer overrun, not if there definitely is one.

[Kees Cook (kees)]

Marking as "Won't Fix" since this is an issue with the unpackaged python 2.3 source (and how it interacts with the Intrepid gcc). Short of fixing the realpath usage, the workaround mentioned (-D_FORTIFY_SOURCE=0) is right.

[Kees Cook (kees)]

(Or -U_FORTIFY_SOURCE) See https://wiki.ubuntu.com/CompilerFlags for more details.