PyPy crash on SSL accept

[Tobias Oberstein]

Tested: PyPy + Twisted, both head@trunk (as of 04/14/12).

SSL stuff will import in Twisted, and Twisted starts. However, on first SSL client connecting to server, PyPy exits with

Fatal error: pthread_mutex_unlock(&mutex_gil)
Abort trap: 6

I think this originates in PyPy source file

./pypy/translator/c/src/thread_pthread.h: Line 553

ASSERT_STATUS(pthread_mutex_unlock(&mutex_gil));

https://bitbucket.org/pypy/pypy/src/ee5f8c2f4686/pypy/translator/c/src/thread_pthread.h#cl-553

Running CPy with same Twisted and server stuff will work fine.

[Tobias Oberstein (tobias-oberstein)]

Platforms tested: FreeBSD9 i386 and Amazon Linux AMI (i686). Happens on both.

[Tobias Oberstein (tobias-oberstein)]

Ok, the error return code from pthread_mutex_unlock is "1", which is EPERM

The Posix docs say:

"""
The pthread_mutex_unlock() function may fail if:

       EPERM The current thread does not own the mutex
"""

[Tobias Oberstein (tobias-oberstein)]

Ok, I have isolated the problem to a minimal example:

https://github.com/oberstet/scratchbox/tree/master/python/twisted/pypybug1

The issue arises when there was a adbapi.ConnectionPool() created, and when using SSL.

[Tobias Oberstein (tobias-oberstein)]

No pool, everything fine:

[autobahn@autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$ pypy testcase.py
no adbapi.ConnectionPool created
connection made
received: G
received: ET / HTTP/1.1
Host: 192.168.1.120:8090
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Cache-Control: max-age=0

^Cconnection lost

With pool, the stuff crashes:

[autobahn@autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$ pypy testcase.py 1
creating adbapi.ConnectionPool
connection made
Fatal error: pthread_mutex_unlock(&mutex_gil)
Abort trap: 6
[autobahn@autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$

[Tobias Oberstein (tobias-oberstein)]

Ok, the following reproduces the issue with pure PyOpenSSL and Python threads (no Twisted):

https://github.com/oberstet/scratchbox/blob/master/python/twisted/pypybug1/testcase3.py

Executing this will immediately crash PyPy (abort in the pthread_unlock due to unlock from thread that does not own the mutex).

[Tobias Oberstein (tobias-oberstein)]

Update: I was able to reproduce the issue with the following 16 lines of code:

https://github.com/oberstet/scratchbox/blob/master/python/twisted/pypybug1/testcase4.py

On FreeBSD9 i386 it will crash PyPy (either with 1 or 2 cores).

On Linux (Amazon EC2 AMI) it will just hang PyPy.

The PyPy versions I used on those systems are different, both a couple of days old (the Linux one older).