Crashes when C calls Python callback with Python3 due to local ctypes module build configuration

[Iven Hsu]

The sample code is attached, both happens in new installed Ubuntu 11.10 and Ubuntu 12.04.

[Martin Pitt (pitti)]

After a multiple-days debugging orgy I finally found what's triggering this. Please see https://bugzilla.gnome.org/show_bug.cgi?id=669157 for the whole story, but copying the last comment here for convenience:

The Debian package runs this snippet between configure and make, to configure static (builtin) vs. dynamic (extensions) modules:

        egrep \
          "^#($$(awk -v ORS='|' '$$2 ~ /^extension$$/ {print $$1}'
debian/PVER-minimal.README.Debian.in)XX)" \
          Modules/Setup.dist \
            | sed -e 's/^#//' -e 's/-Wl,-Bdynamic//;s/-Wl,-Bstatic//' \
            >> $(1)/Modules/Setup.local

When I disable this part, the local build works.

That code results in this Modules/Setup.local:

----------------- 8< ---------------------
# Edit this file for local setup changes
array arraymodule.c # array objects
math mathmodule.c _math.c # -lm # math library functions, e.g. sin()
_struct _struct.c # binary structure packing/unpacking
time timemodule.c _time.c # -lm # time operations and variables
_random _randommodule.c # Random number generator
atexit atexitmodule.c # Register functions to be run at
interpreter-shutdown
_elementtree -DUSE_PYEXPAT_CAPI _elementtree.c # elementtree accelerator
_pickle _pickle.c # pickle accelerator
_datetime _datetimemodule.c # datetime accelerator
_bisect _bisectmodule.c # Bisection algorithms
_heapq _heapqmodule.c # Heap queue algorithm
unicodedata unicodedata.c # static Unicode character database
fcntl fcntlmodule.c # fcntl(2) and ioctl(2)
spwd spwdmodule.c # spwd(3)
grp grpmodule.c # grp(3)
select selectmodule.c # select(2); not on ancient System V
_socket socketmodule.c
_ssl _ssl.c -lssl -lcrypto
_posixsubprocess _posixsubprocess.c # POSIX subprocess module helper
_hashlib _hashopenssl.c -lssl -lcrypto
syslog syslogmodule.c # syslog daemon interface
binascii binascii.c
_ctypes _ctypes/_ctypes.c _ctypes/callbacks.c _ctypes/callproc.c
_ctypes/stgdict.c _ctypes/cfield.c _ctypes/malloc_closure.c -lffi
zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz
pyexpat pyexpat.c -lexpat
----------------- 8< ---------------------

Bisecting this quickly leads to the _ctypes module configuration.

So a small reproducer based on the upstream Python 3.2 build is:

 * Configure and build pygobject with PYTHON=python3.2 (or specify the full
path for a local python build)

 * Build python 3.2 with the broken _ctypes module configuration:

$ cd /tmp/
$ tar xf python3.2_3.2.3.orig.tar.gz
$ cd python3.2-3.2.3
$ ./configure --with-wide-unicode # to be compatible with the Debian/Ubuntu
python 3.2
$ echo '_ctypes _ctypes/_ctypes.c _ctypes/callbacks.c _ctypes/callproc.c
_ctypes/stgdict.c _ctypes/cfield.c _ctypes/malloc_closure.c -lffi' >>
Modules/Setup.local
$ make -j4

Then in the pygobject built tree, run

$ (cd tests; TEST_NAMES=test_everything.TestCallbacks PYTHONPATH=..:.
LD_LIBRARY_PATH=./.libs GI_TYPELIB_PATH=. /tmp/python3.2-3.2.3/python
./runtests.py)

which reproduces the failure.

Conversely, if I change the ctypes line in debian/PVER-minimal.README.Debian.in from

  _ctypes extension

to

  _ctypes ...

[Martin Pitt (pitti)]

The current theory is that things go wrong because libffi is loaded from both python itself (due to building in the ctypes extension) as well as dynamically through libgirepository.

I extended the test program to use ctypes and gi.repository at the same time, and both work with my local packages which build ctypes as a separate extension again.

[Martin Pitt (pitti)]

Hello Iven, or anyone else affected,

Accepted python3.2 into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Dmitry Shachnev (mitya57)]

FTBFS on amd64, see https://launchpad.net/ubuntu/+source/python3.2/3.2.3-0ubuntu2/+build/3460071.

[Martin Pitt (pitti)]

-0ubuntu3 built now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python3.2 - 3.2.3-0ubuntu3

---------------
python3.2 (3.2.3-0ubuntu3) precise-proposed; urgency=low

  * Build _ctypes as an extension, not a builtin. LP: #909292.
  * Mark symbols defined in the _ctypes extension as optional.
 -- Matthias Klose <email address hidden> Thu, 03 May 2012 13:20:08 +0200

[Iven Hsu (ivenvd)]

After updated, the bug seems fixed, thanks.

[Martin Pitt (pitti)]

I confirm that the pygobject test suite now works fine with this python3.2 version. As libpython3.2 changed ABI, I'd also like to verify that its reverse dependencies still work before calling this verified:

Reverse Depends:
  python3-uno
  znc-python
  yafaray
  uwsgi-plugin-python3
  python3-uno
  python3-pyside.qtxml
  python3-pyside.qtwebkit
  python3-pyside.qtuitools
  python3-pyside.qttest
  python3-pyside.qtsvg
  python3-pyside.qtsql
  python3-pyside.qtscript
  python3-pyside.qtopengl
  python3-pyside.qtnetwork
  python3-pyside.qthelp
  python3-pyside.qtgui
  python3-pyside.qtdeclarative
  python3-pyside.qtcore
  python3-pyside.phonon
  libshiboken-py3-1.1
  libpyside-py3-1.1
  libapache2-mod-wsgi-py3
  blender
  python3.2-dev
  postgresql-plpython3-9.1

[Martin Pitt (pitti)]

I verified most of the reverse dependencies. The ones that are still left are:

yafaray
libapache2-mod-wsgi-py3

For those I had a look at which symbols they need, and grepped for the removed symbols:

$ dpkg -L yafaray | grep .so | xargs nm -D | egrep 'PyC|stgdict|PyDict_SetItemProxy|PyDict_GetItemProxy'
                 U PyCapsule_GetPointer
                 U PyCapsule_Import
                 U PyCapsule_New

$ nm -D /usr/lib/apache2/modules/mod_wsgi.so-3.2 | egrep 'PyC|stgdict|PyDict_SetItemProxy|PyDict_GetItemProxy'
                 U PyCFunction_NewEx
                 U PyCapsule_New

These both look fine, these symbols were not removed.

[Martin Pitt (pitti)]

The test script which uses both ctypes and gi also works fine.

[Colin Watson (cjwatson)]

I ran into this while trying to port ubiquity and usb-creator. The python3.2 package in -proposed lets me get past the segfaults and onto the next error. Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package python3.2 - 3.2.3-0ubuntu3

---------------
python3.2 (3.2.3-0ubuntu3) precise-proposed; urgency=low

  * Build _ctypes as an extension, not a builtin. LP: #909292.
  * Mark symbols defined in the _ctypes extension as optional.
 -- Matthias Klose <email address hidden> Thu, 03 May 2012 13:20:08 +0200