Segfault in gc with cyclic trash

Bug #1367907 reported by Clark Boylan
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Python
Fix Released
Unknown
oslo.messaging
Invalid
Undecided
Clark Boylan
python3.4 (Ubuntu)
Fix Released
High
Barry Warsaw
Trusty
In Progress
Undecided
Unassigned

Bug Description

Trusty's python3.4 package is affected by python bug http://bugs.python.org/issue21435. This bug was fixed in http://hg.python.org/cpython/rev/64ba3f2de99c. Trusty should pull this fix into the python3.4 package.

Note this definitely affects some python projects. `git clone https://git.openstack.org/openstack/oslo.messaging && cd oslo.messaging && edit tox.ini file, copy py33 target and rename to py34 && tox -repy34` will segfault due to this bug. With previous versions of python unaffected by this bug there is no segfaulting.

Now for details, broken using:
$ lsb_release -rd
Description: Ubuntu 14.04.1 LTS
Release: 14.04
$ apt-cache policy python3.4
python3.4:
  Installed: 3.4.0-2ubuntu1
  Candidate: 3.4.0-2ubuntu1
  Version table:
 *** 3.4.0-2ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
        100 /var/lib/dpkg/status

If I install the following python3.4 packages for Unicorn on Trusty this bug is corrected. The Unicorn packages here on lp for python3.4 are python3.4.1 which includes the bug fix for this bug.

libpython3.4_3.4.1-10ubuntu1_amd64.deb libpython3.4-dev_3.4.1-10ubuntu1_amd64.deb libpython3.4-stdlib_3.4.1-10ubuntu1_amd64.deb python3.4-dbg_3.4.1-10ubuntu1_amd64.deb python3.4-minimal_3.4.1-10ubuntu1_amd64.deb libpython3.4-dbg_3.4.1-10ubuntu1_amd64.deb libpython3.4-minimal_3.4.1-10ubuntu1_amd64.deb python3.4_3.4.1-10ubuntu1_amd64.deb python3.4-dev_3.4.1-10ubuntu1_amd64.deb

Note I installed the -dbg packages too just in case I needed to use gdb but that wasn't necessary as the test case works with these newer versions of python.

$ apt-cache policy python3.4
python3.4:
  Installed: 3.4.1-10ubuntu1
  Candidate: 3.4.1-10ubuntu1
  Version table:
 *** 3.4.1-10ubuntu1 0
        100 /var/lib/dpkg/status
     3.4.0-2ubuntu1 0
        500 http://az2.clouds.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

[Impact]

This python bug can cause python processes that tickle it to segfault python. This means there could be a significant number of python packages that are broken when run under this interpreter. The fix for this bug should be backported to avoid seemingly random and hard to debug segfaults from happening when users use python.

Unicorn's 3.4.1 packages fix this bug by including the patch at http://hg.python.org/cpython/rev/64ba3f2de99c.

[Test Case]

There is a test case detailed in the upstream bug http://bugs.python.org/issue21435, though I have been doing the following instead:
git clone https://git.openstack.org/openstack/oslo.messaging
cd oslo.messaging
git fetch https://review.openstack.org/openstack/oslo.messaging refs/changes/90/118790/2 && git checkout FETCH_HEAD
tox -repy34
# This should end with segfaulting test runners. Note if you don't have tox installed you will need to install version 1.7.2 or greater. `sudo pip install tox==1.7.2` will do this.

[Regression Potential]

The patch in question is small. If we go straight to python 3.4.1 the diff will be larger but that isn't necessary to fix this particular issue. The biggest regression potential would be that the garbage collector is newly broken by the this bug fix. Considering that the garbage collector is already broken and this patch comes with test cases to check it is less broken the risk of regression here is less than the pain of dealing with this bug.

Clark Boylan (cboylan)
description: updated
Changed in oslo.messaging:
assignee: nobody → Clark Boylan (cboylan)
Revision history for this message
Robert Collins (lifeless) wrote :

The patch is pretty small, I think this is a good candidate for a SRU.

Clark Boylan (cboylan)
description: updated
Changed in python:
status: Unknown → Fix Released
Barry Warsaw (barry)
Changed in python3.4 (Ubuntu):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Barry Warsaw (barry)
Revision history for this message
Matthias Klose (doko) wrote :

fixed in 14.10

Changed in python3.4 (Ubuntu Trusty):
status: New → Triaged
Changed in python3.4 (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Matthias Klose (doko) wrote :

for trusty, this is fixed in the ubuntu-toolchain-r/ppa PPA.

Revision history for this message
Jeremy Stanley (fungi) wrote :

I've confirmed that upgrading a Trusty/amd64 VM to the python3.4 packages in that PPA fixes the issue we observed running unit tests for OpenStack's oslo.messaging project under that interpreter.

Revision history for this message
Jeremy Stanley (fungi) wrote :

Is the plan to solve this in Trusty via bug 1348954 or to separately SRU the upstream patch?

Mehdi Abaakouk (sileht)
Changed in oslo.messaging:
status: New → Invalid
Revision history for this message
Jeremy Stanley (fungi) wrote :

Based on E-mail discussion with Barry and Matthias, it sounds like the plan now is to SRU MRE Python 3.4.3 into Trusty once it's available (due out February 22, 2015 according to the official release schedule).

Revision history for this message
Victor Stinner (vstinner) wrote :

This issue is still biting us on the OpenStack CI. The py34 test job of the Oslo Messaging project is crashing randomly, even after disabling the test using trollius (the bug was initially found on an asyncio example, and trollius code is very close to asyncio).

Please fix it, Trusty is a LTS version! Python 3.4 matters for us! The bug was fixed in Python in May 2014 and was reported here in September 2014.

Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Clark, or anyone else affected,

Accepted python3.4 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python3.4/3.4.3-1ubuntu1~14.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python3.4 (Ubuntu Trusty):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Clark Boylan (cboylan) wrote :

I have confirmed that running `tox -epy34` against oslo.messaging with the old python3.4 packages continues to segfault. After upgrading to the newly built python3.4.3-1ubuntu1~14.04 package the segfaulting stops and the tests pass.

This looks good to me. Thank you.

Version of python from aptitude:
$ aptitude show python3.4
Package: python3.4
State: installed
Automatically installed: no
Multi-Arch: allowed
Version: 3.4.3-1ubuntu1~14.04
Priority: important
Section: python
Maintainer: Ubuntu Core Developers <email address hidden>
Architecture: amd64
Uncompressed Size: 353 k
Depends: python3.4-minimal (= 3.4.3-1ubuntu1~14.04), libpython3.4-stdlib (= 3.4.3-1ubuntu1~14.04), mime-support
Suggests: python3.4-venv, python3.4-doc, binutils
Conflicts: python3.4
Provides: python3.4:any
Description: Interactive high-level object-oriented language (version 3.4)

tags: added: verification-done
removed: verification-needed
Revision history for this message
Victor Stinner (vstinner) wrote :

I tested on a fresh Ubuntu Trusty (python3.4 package version 3.4.0-2ubuntu1.1): "python3 segfaulter.py" does crash, where segfaulter.py comes from http://bugs.python.org/issue21435#msg217918

I upgraded python3.4 package to 3.4.3-1ubuntu1~14.04.1 using "deb http://archive.ubuntu.com/ubuntu/ trusty-proposed restricted main multiverse universe" source: "python3 segfaulter.py" doesn't crash anymore!

Please push python 3.4.3 to the main Trusty repository.

Revision history for this message
Gasc Ludovic (gmludo) wrote :

+14 for the upgrade.

Revision history for this message
Victor Stinner (vstinner) wrote :

The version with the fix in trusty-proposed since 2 months, it was validated by 2 developers 1 month 1/2 ago, but then nothing happened. What's going on with this issue?

The fix was pushed into CPython 3.4 on May 2014. Come on Ubuntu, please do something for Python 3.4! This bug is a major bug. I don't understand why it is not fixed.

Revision history for this message
Barry Warsaw (barry) wrote : Re: [Bug 1367907] Re: Segfault in gc with cyclic trash

On Sep 14, 2015, at 07:28 AM, Victor Stinner wrote:

>The version with the fix in trusty-proposed since 2 months, it was
>validated by 2 developers 1 month 1/2 ago, but then nothing happened.
>What's going on with this issue?

It's possible that the SRU didn't see this bug because it was marked Fix
Committed in Trusty. According to
https://wiki.ubuntu.com/StableReleaseUpdates it should be marked In Progress.

I've now done that, so I think at this point it's waiting for SRU team to
unblock proposed migration.

Changed in python3.4 (Ubuntu Trusty):
status: Fix Committed → In Progress
Revision history for this message
Victor Stinner (vstinner) wrote :

ping?

Revision history for this message
Victor Stinner (vstinner) wrote :

"I've now done that, so I think at this point it's waiting for SRU team to unblock proposed migration."

This bug is really serious and it's not possible to work around it. I don't understand why Canonical doesn't care of this major Python crash!?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.