Fatal RPython error: KeyError/Crash in JIT!

[Jeremy Stanley]

Trusty's pypy 2.2.1+dfsg-1 package seems to be affected by PyPy bugs https://bitbucket.org/pypy/pypy/issue/1669 and https://bitbucket.org/pypy/pypy/issue/1694/ (which are probably duplicates manifesting in a couple ways). This bug is fixed in https://bitbucket.org/pypy/pypy/commits/5ca1994 . Trusty should pull this fix into the pypy package.

Note this definitely affects some python projects. `git clone https://git.openstack.org/openstack/zaqar.git && cd zaqar && tox -e pypy` will trigger this. With the pypy 2.3.1+dfsg-1~ppa1 package I'm unable to reproduce the error.

Now for details, broken using:
$ lsb_release -rd
Description: Ubuntu 14.04.1 LTS
Release: 14.04
$ apt-cache policy pypy
pypy:
  Installed: 2.2.1+dfsg-1
  Candidate: 2.2.1+dfsg-1
  Version table:
 *** 2.2.1+dfsg-1 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
        100 /var/lib/dpkg/status

[Impact]

This bug can cause processes that tickle it to crash PyPy's JIT. This means there could be a significant number of Python programs that are broken when run under this interpreter. The fix for this bug should be backported to avoid seemingly random and hard to debug JIT crashes from happening when users use pypy.

[Test Case]

Download main.py and python.zip from https://bitbucket.org/pypy/pypy/issue/1669
unzip, and place main.py inside the python directory
Run pypy main.py.
It'll either crash immediately, or start outputting some lines of text (equaling success)

[Test Case - No longer relevant, worked around]

git clone https://git.openstack.org/openstack/zaqar.git
cd zaqar
tox -e pypy
# This should end with aborting during test discovery. Note if you don't have tox installed you will need to install version 1.7.2 or greater. `sudo pip install tox==1.7.2` will do this.

[Regression Potential]

The patch in question is small. If we go straight to pypy 2.3.x the diff will be larger but that isn't necessary to fix this particular issue. The biggest regression potential would be that heap cache management ends up broken by the this bug fix.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/120906

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/120906
Committed: https://git.openstack.org/cgit/openstack/zaqar/commit/?id=a72a0bd988e98f74e199a411814f55b60f6a7b4a
Submitter: Jenkins
Branch: master

commit a72a0bd988e98f74e199a411814f55b60f6a7b4a
Author: Clark Boylan <email address hidden>
Date: Thu Sep 11 11:51:01 2014 -0700

    Workaround pypy bug on trusty when running tests

    https://bitbucket.org/pypy/pypy/issue/1694/ affects zaqar when running
    pypy tests on trusty slave nodes. We can work around that bug by setting
    --jit off when running the pypy interpreter. Do this until the upstream
    bug fix is backported into trusty.

    Change-Id: Iebaa167e9e12bbc1017371929ca8c182bacc2aad
    Related-Bug: 1368418

[Stefano Rivera (stefanor)]

Hi.

Sorry, I didn't notice this report, immediately, and have been busy.

Should be trivial enough to SRU, debdiff attached.

[Stefano Rivera (stefanor)]

Hello Jeremy, or anyone else affected,

pypy has built in trusty-proposed. The package is available at https://launchpad.net/ubuntu/+source/pypy/2.2.1+dfsg-1ubuntu0.2 and in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[added by hand, because sru-accept wasn't run by the SRU team]

[Stefano Rivera (stefanor)]

Well, nobody else has verified this, but it looks good to me.

[Stefano Rivera (stefanor)]

> [added by hand, because sru-accept wasn't run by the SRU team]

Not actually true. ...because I screwed up the upload :)