python2.7 & python imaging (PIL) dumps core with buffer overflow

Bug #717843 reported by Damon Lynch
26
This bug affects 3 people
Affects Status Importance Assigned to Milestone
python-imaging (Ubuntu)
Fix Released
Medium
Barry Warsaw
Natty
Invalid
Medium
Barry Warsaw
Oneiric
Fix Released
Medium
Barry Warsaw

Bug Description

Binary package hint: python2.7

Python 2.7 dumps core when doing python imaging (PIL) operations on a TIF file. Below is a sample doing a resize; but another operation like converting the color mode will also dump core.

The unusual color mode may be contributing to the bug.

This cannot be replicated under maverick, where it works as expected.

I will either attach the TIF file to this bug report, or if too large, I will link to it elsewhere.

>>> from PIL import Image
>>> image = Image.open('/home/damon/tmp/samsung.tif')
>>> image
<PIL.TiffImagePlugin.TiffImageFile image mode=YCbCr size=3872x2592 at 0x2904128>
>>> image.thumbnail((242,162), Image.NEAREST)
*** buffer overflow detected ***: python terminated
======= Backtrace: =========
/lib/libc.so.6(__fortify_fail+0x37)[0x7f87c2581477]
/lib/libc.so.6(+0xf7390)[0x7f87c2580390]
/usr/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNewPrologueSubtype+0x97)[0x7f87bf3b11e7]
/usr/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNewArray+0x11)[0x7f87bf3b1691]
/usr/lib/python2.7/dist-packages/PIL/_imaging.so(+0xfc7b)[0x7f87bf393c7b]
python(PyEval_EvalFrameEx+0x56f3)[0x4ac1a3]
python(PyEval_EvalFrameEx+0x654d)[0x4acffd]
======= Memory map: ========
00400000-0066c000 r-xp 00000000 08:17 131874 /usr/bin/python2.7
0086b000-0086c000 r--p 0026b000 08:17 131874 /usr/bin/python2.7
0086c000-008d4000 rw-p 0026c000 08:17 131874 /usr/bin/python2.7
008d4000-008e6000 rw-p 00000000 00:00 0
01eff000-02968000 rw-p 00000000 00:00 0 [heap]
7f87bf160000-7f87bf183000 r-xp 00000000 08:17 134333 /usr/lib/libjpeg.so.62.0.0
7f87bf183000-7f87bf382000 ---p 00023000 08:17 134333 /usr/lib/libjpeg.so.62.0.0
7f87bf382000-7f87bf383000 r--p 00022000 08:17 134333 /usr/lib/libjpeg.so.62.0.0
7f87bf383000-7f87bf384000 rw-p 00023000 08:17 134333 /usr/lib/libjpeg.so.62.0.0
7f87bf384000-7f87bf3c1000 r-xp 00000000 08:17 269344 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
7f87bf3c1000-7f87bf5c1000 ---p 0003d000 08:17 269344 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
7f87bf5c1000-7f87bf5c4000 r--p 0003d000 08:17 269344 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
7f87bf5c4000-7f87bf5c7000 rw-p 00040000 08:17 269344 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
7f87bf5c7000-7f87bf5e6000 r-xp 00000000 08:17 141864 /usr/lib/python2.7/lib-dynload/_ctypes.so
7f87bf5e6000-7f87bf7e5000 ---p 0001f000 08:17 141864 /usr/lib/python2.7/lib-dynload/_ctypes.so
7f87bf7e5000-7f87bf7e6000 r--p 0001e000 08:17 141864 /usr/lib/python2.7/lib-dynload/_ctypes.so
7f87bf7e6000-7f87bf7ea000 rw-p 0001f000 08:17 141864 /usr/lib/python2.7/lib-dynload/_ctypes.so
7f87bf7ea000-7f87bf7eb000 rw-p 00000000 00:00 0
7f87bf7eb000-7f87bf8ed000 r-xp 00000000 08:17 133829 /usr/lib/libapt-pkg.so.4.10.1
7f87bf8ed000-7f87bfaed000 ---p 00102000 08:17 133829 /usr/lib/libapt-pkg.so.4.10.1
7f87bfaed000-7f87bfaf0000 r--p 00102000 08:17 133829 /usr/lib/libapt-pkg.so.4.10.1
7f87bfaf0000-7f87bfaf1000 rw-p 00105000 08:17 133829 /usr/lib/libapt-pkg.so.4.10.1
7f87bfaf1000-7f87bfaf2000 rw-p 00000000 00:00 0
7f87bfaf2000-7f87bfb34000 r-xp 00000000 08:17 141396 /usr/lib/python2.7/dist-packages/apt_pkg.so
7f87bfb34000-7f87bfd34000 ---p 00042000 08:17 141396 /usr/lib/python2.7/dist-packages/apt_pkg.so
7f87bfd34000-7f87bfd35000 r--p 00042000 08:17 141396 /usr/lib/python2.7/dist-packages/apt_pkg.so
7f87bfd35000-7f87bfd3d000 rw-p 00043000 08:17 141396 /usr/lib/python2.7/dist-packages/apt_pkg.so
7f87bfd3d000-7f87bfd5a000 r-xp 00000000 08:17 141871 /usr/lib/python2.7/lib-dynload/_io.so
7f87bfd5a000-7f87bff59000 ---p 0001d000 08:17 141871 /usr/lib/python2.7/lib-dynload/_io.so
7f87bff59000-7f87bff5a000 r--p 0001c000 08:17 141871 /usr/lib/python2.7/lib-dynload/_io.so
7f87bff5a000-7f87bff63000 rw-p 0001d000 08:17 141871 /usr/lib/python2.7/lib-dynload/_io.so
7f87bff63000-7f87bff72000 r-xp 00000000 08:17 141891 /usr/lib/python2.7/lib-dynload/pyexpat.so
7f87bff72000-7f87c0171000 ---p 0000f000 08:17 141891 /usr/lib/python2.7/lib-dynload/pyexpat.so
7f87c0171000-7f87c0172000 r--p 0000e000 08:17 141891 /usr/lib/python2.7/lib-dynload/pyexpat.so
7f87c0172000-7f87c0174000 rw-p 0000f000 08:17 141891 /usr/lib/python2.7/lib-dynload/pyexpat.so
7f87c0174000-7f87c0187000 r-xp 00000000 08:17 141881 /usr/lib/python2.7/lib-dynload/datetime.so
7f87c0187000-7f87c0386000 ---p 00013000 08:17 141881 /usr/lib/python2.7/lib-dynload/datetime.so
7f87c0386000-7f87c0387000 r--p 00012000 08:17 141881 /usr/lib/python2.7/lib-dynload/datetime.so
7f87c0387000-7f87c038b000 rw-p 00013000 08:17 141881 /usr/lib/python2.7/lib-dynload/datetime.so
7f87c038b000-7f87c038e000 r-xp 00000000 08:17 141869 /usr/lib/python2.7/lib-dynload/_heapq.so
7f87c038e000-7f87c058d000 ---p 00003000 08:17 141869 /usr/lib/python2.7/lib-dynload/_heapq.so
7f87c058d000-7f87c058e000 r--p 00002000 08:17 141869 /usr/lib/python2.7/lib-dynload/_heapq.so
7f87c058e000-7f87c0590000 rw-p 00003000 08:17 141869 /usr/lib/python2.7/lib-dynload/_heapq.so
7f87c0590000-7f87c05b6000 r-xp 00000000 08:17 261712 /lib/libexpat.so.1.5.2
7f87c05b6000-7f87c07b6000 ---p 00026000 08:17 261712 /lib/libexpat.so.1.5.2
7f87c07b6000-7f87c07b8000 r--p 00026000 08:17 261712 /lib/libexpat.so.1.5.2
7f87c07b8000-7f87c07b9000 rw-p 00028000 08:17 261712 /lib/libexpat.so.1.5.2
7f87c07b9000-7f87c07c0000 r-xp 00000000 08:17 292210 /lib/librt-2.12.2.so
7f87c07c0000-7f87c09bf000 ---p 00007000 08:17 292210 /lib/librt-2.12.2.so
7f87c09bf000-7f87c09c0000 r--p 00006000 08:17 292210 /lib/librt-2.12.2.so
7f87c09c0000-7f87c09c1000 rw-p 00007000 08:17 292210 /lib/librt-2.12.2.so
7f87c09c1000-7f87c09d6000 r-xp 00000000 08:17 261662 /lib/libgcc_s.so.1
7f87c09d6000-7f87c0bd5000 ---p 00015000 08:17 261662 /lib/libgcc_s.so.1
7f87c0bd5000-7f87c0bd6000 r--p 00014000 08:17 261662 /lib/libgcc_s.so.1
7f87c0bd6000-7f87c0bd7000 rw-p 00015000 08:17 261662 /lib/libgcc_s.so.1
7f87c0bd7000-7f87c0cbf000 r-xp 00000000 08:17 131103 /usr/lib/libstdc++.so.6.0.14
7f87c0cbf000-7f87c0ebe000 ---p 000e8000 08:17 131103 /usr/lib/libstdc++.so.6.0.14
7f87c0ebe000-7f87c0ec6000 r--p 000e7000 08:17 131103 /usr/lib/libstdc++.so.6.0.14
7f87c0ec6000-7f87c0ec8000 rw-p 000ef000 08:17 131103 /usr/lib/libstdc++.so.6.0.14
7f87c0ec8000-7f87c0edd000 rw-p 00000000 00:00 0
7f87c0edd000Aborted (core dumped)

ProblemType: Crash
DistroRelease: Ubuntu 11.04
Package: python2.7-minimal 2.7.1-3
ProcVersionSignature: Ubuntu 2.6.38-3.30-generic 2.6.38-rc4
Uname: Linux 2.6.38-3-generic x86_64
Architecture: amd64
AssertionMessage: *** buffer overflow detected ***: python terminated
Date: Sat Feb 12 12:44:25 2011
ExecutablePath: /usr/bin/python2.7
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha amd64 (20110202)
ProcCmdline: python
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, user)
 LC_MESSAGES=en_US.utf8
 LANG=en_US.UTF-8
 LANGUAGE=en_US:en
Signal: 6
SourcePackage: python2.7
StacktraceTop:
 raise () from /lib/libc.so.6
 abort () from /lib/libc.so.6
 ?? () from /lib/libc.so.6
 __fortify_fail () from /lib/libc.so.6
 __chk_fail () from /lib/libc.so.6
Title: python2.7 assert failure: *** buffer overflow detected ***: python terminated
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare

Revision history for this message
Damon Lynch (dlynch3) wrote :
Revision history for this message
Damon Lynch (dlynch3) wrote :

Attachment: TIF image that generates crash

visibility: private → public
Revision history for this message
Apport retracing service (apport) wrote :

StacktraceTop:
 *__GI_raise (sig=6)
 *__GI_abort () at abort.c:59

Revision history for this message
Apport retracing service (apport) wrote : Stacktrace.txt
Revision history for this message
Apport retracing service (apport) wrote : ThreadStacktrace.txt
Changed in python2.7 (Ubuntu):
importance: Undecided → Medium
tags: removed: need-amd64-retrace
Revision history for this message
icb410 (ian-berke) wrote :
Download full text (7.4 KiB)

I have this issue too with some custom scripts that reads the intensity data from a grayscale TIF image of the output of a CCD detector (7_WAXS) and then radially averages. Occurs with python2.6 as well. Another TIF image from a different type of detector works, but I'm not really sure of the difference in files. One may have 32bit samples (the one that works, 7_SAXS) as imagemagick gives an error (it's also smaller ~400x600 vs 1042x1042).
The output of file gives:

7_SAXS: TIFF image data, little-endian
7_WAXS: TIFF image data, big-endian

I'm attaching the two images, and here's the output of the core dump:

Python 2.7.1+ (r271:86832, Mar 24 2011, 00:37:39)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from exp_setup import *
>>> dsamp.avg("78_1p7mg-3.90s",es,ew,msaxs,mwaxs,plot_data=False,save_ave=True)
processing 7 ...
dezinger ...
*** buffer overflow detected ***: python terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7fd30ace51d7]
/lib/x86_64-linux-gnu/libc.so.6(+0xfd0f0)[0x7fd30ace40f0]
/usr/lib/python2.7/dist-packages/PIL/_imaging.so(ImagingNewPrologueSubtype+0x97)[0x7fd30732e1e7]
/usr/lib/python2.7/dist-packages/PIL/_imaging.so(PyImaging_MapBuffer+0xd1)[0x7fd307314751]
python(PyEval_EvalFrameEx+0x361)[0x496631]
python(PyEval_EvalFrameEx+0xb2f)[0x496dff]
python(PyEval_EvalCodeEx+0x145)[0x49d375]
python(PyEval_EvalFrameEx+0x802)[0x496ad2]
python(PyEval_EvalCodeEx+0x145)[0x49d375]
python[0x4c45a6]
python(PyObject_Call+0x44)[0x45d854]
python[0x45f47f]
python(PyObject_Call+0x44)[0x45d854]
python(PyEval_CallObjectWithKeywords+0x36)[0x495d86]
python(PyInstance_New+0x78)[0x460658]
python(PyObject_Call+0x44)[0x45d854]
python(PyEval_EvalFrameEx+0x9be)[0x496c8e]
python(PyEval_EvalCodeEx+0x145)[0x49d375]
python(PyEval_EvalFrameEx+0x802)[0x496ad2]
python(PyEval_EvalCodeEx+0x145)[0x49d375]
python(PyEval_EvalCode+0x32)[0x4ecbc2]
python[0x4fdd34]
python(PyRun_InteractiveOneFlags+0x1e2)[0x42cd9e]
python(PyRun_InteractiveLoopFlags+0xc0)[0x42cebd]
python(PyRun_AnyFileExFlags+0x39)[0x42d2e1]
python(Py_Main+0xac9)[0x418c9e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff)[0x7fd30ac05eff]
python[0x4c6305]
======= Memory map: ========
00400000-0062f000 r-xp 00000000 08:01 2560 /usr/bin/python2.7
0082e000-0082f000 r--p 0022e000 08:01 2560 /usr/bin/python2.7
0082f000-00897000 rw-p 0022f000 08:01 2560 /usr/bin/python2.7
00897000-008a9000 rw-p 00000000 00:00 0
025ce000-03c3c000 rw-p 00000000 00:00 0 [heap]
7fd302167000-7fd30237a000 rw-s 00000000 08:01 659009 /home/ian/Documents/Modislab/saxs/bnl111010/7_WAXS
7fd30237a000-7fd30237f000 r-xp 00000000 08:01 953977 /usr/lib/python2.7/lib-dynload/mmap.so
7fd30237f000-7fd30257e000 ---p 00005000 08:01 953977 /usr/lib/python2.7/lib-dynload/mmap.so
7fd30257e000-7fd30257f000 r--p 00004000 08:01 953977 /usr/lib/python2.7/lib-dynload/mmap.so
7fd30257f000-7fd302580000 rw-p 00005000 08:01 953977 /...

Read more...

Revision history for this message
icb410 (ian-berke) wrote :
Revision history for this message
icb410 (ian-berke) wrote :
Matthias Klose (doko)
affects: python2.7 (Ubuntu) → python-imaging (Ubuntu)
tags: added: python27
Barry Warsaw (barry)
Changed in python-imaging (Ubuntu Natty):
status: New → Confirmed
assignee: nobody → Barry Warsaw (barry)
milestone: none → ubuntu-11.04-beta-2
Revision history for this message
Barry Warsaw (barry) wrote :

Verified, but note that a from-source build of PIL 1.1.7 (downloaded from the pythonware.com site) built in a virtualenv of Python 2.7 also crashes in the same way with both Python 2.6 and 2.7. Also PIL seems pretty much unsupported upstream these days as the last release was Nov 2009. This will no doubt be a fun one.

Revision history for this message
icb410 (ian-berke) wrote :

I also couldn't build PIL from source on Natty, but installing the Maverick python-imaging deb file works for python2.6 on Natty. Don't like to mix-n-match, but have to have this for work.

Colin Watson (cjwatson)
Changed in python-imaging (Ubuntu Natty):
milestone: ubuntu-11.04-beta-2 → ubuntu-11.04
Changed in python-imaging (Ubuntu Natty):
milestone: ubuntu-11.04 → natty-updates
Changed in python-imaging (Ubuntu Oneiric):
milestone: ubuntu-11.04 → none
Revision history for this message
David Oftedal (rounin) wrote :

Even creating a new YCbCr image or converting an existing one to YCbCr makes PIL crash due to a buffer overflow, so there's a good chance this is due to that bug.

Revision history for this message
David Oftedal (rounin) wrote :
Revision history for this message
Damon Lynch (dlynch3) wrote : Re: [Bug 717843] Re: python2.7 & python imaging (PIL) dumps core with buffer overflow

When I first reported the bug, I mentioned it on the PIL mailing list.
Unfortunately there was no response.

Revision history for this message
David Oftedal (rounin) wrote :
Revision history for this message
icb410 (ian-berke) wrote :

I just tried installing the 1.1.7-3 build for Oneiric(python-imaging_1.1.7-3ubuntu1_amd64.deb) which includes the above Fedora patch (I think) on my Natty installation and I get the same buffer overflow.

Revision history for this message
David Oftedal (rounin) wrote :

Odd... But this thread seems to indicate that TWO fixes might be needed?

http://stackoverflow.com/questions/6080825/python-pil-ycbcr-support

Revision history for this message
icb410 (ian-berke) wrote :

I'm now running Oneiric beta2 and it seems PIL is working again. I can load my TIF images now just fine. Creating a YCbCr image seems to work ok too.

Can anyone confirm this? Or is it just me?

Revision history for this message
Damon Lynch (dlynch3) wrote :

It works on Oneiric beta 2 with the original TIFF file in this bug report. Everything also works as expected when taking the resulting PIL image and doing things like passing it between two processes with a pipe.

I will mark it as fixed for Oneiric - I hope that is ok with everyone.

Changed in python-imaging (Ubuntu Oneiric):
status: Confirmed → Fix Released
Revision history for this message
David Oftedal (rounin) wrote :

I can confirm that it works both for converting RGB images and for creating new YCbCr images in Oneiric Ocelot.

Wonderful news!

dino99 (9d9)
Changed in python-imaging (Ubuntu Natty):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.