Ubuntu

SRU: 100% CPU usage when calling a child process from a python script

Reported by Damon Lynch on 2010-10-22
62
This bug affects 13 people
Affects Status Importance Assigned to Milestone
PyGTK
Unknown
Unknown
pygtk (Ubuntu)
Undecided
Unassigned
Lucid
Undecided
Unassigned
Maverick
Undecided
Unassigned
Natty
Undecided
Unassigned

Bug Description

== Impact ==

pygtk apps which handle signals busy-loop in the glib main loop dispatch, causing 100% CPU usage after a signal is received. This commonly occurs when spawned child processes exit.

== TEST CASE ==

A simple testcase (pygtk-signal-testcase.py) is attached. To trigger the bug, run the testcase and send the process SIGCHLD. When run with pygtk 2.22-0ubuntu1 the test process will now consume 100% CPU.

== Regression Potential ==

PyGTK is core infrastructure, so the potential impact of a regression is high. However, the patch is small and clearly correct, and the file descriptor being read from is nonblocking, so it should not cause application hangs even if it tries to read more bytes than are there.

Original bugreport follows:
When creating a child process from a python script that uses pygtk 2.21, such subprocess.check_call(), 100% of the CPU is used (technically, 100% of the core the python interpreter is using).

If the creation of the child processes are removed, the script runs as expected.

According to user exarkun on the #python IRC channel, the problem may be manifesting when the child process exits. It may be related to a bug that was fixed in 9.10, but broken in 9.04. If that is the case, this is the relevant bug report: https://bugzilla.gnome.org/show_bug.cgi?id=481569

The code in which I noticed this problem is in the videometadata.py module of Rapid Photo Downloader: http://bazaar.launchpad.net/~dlynch3/rapid/trunk/annotate/head%3A/rapid/videometadata.py#L189

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: python-gtk2 2.21.0-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.35-22.35-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic x86_64
Architecture: amd64
Date: Thu Oct 21 22:51:42 2010
InstallationMedia: Ubuntu 10.10 "Maverick Meerkat" - Release amd64 (20101007)
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: pygtk

Damon Lynch (dlynch3) wrote :

Here is the upstream bug:
https://bugzilla.gnome.org/show_bug.cgi?id=640738

I have tested the fix and it works. I believe Debian has already merged a patch for pygtk 2.17, Ubuntu should probably fix all versions, not just natty, as this affects maverick at least (I can confirm) and probably also lucid (I have not tested).

It also affects virt-manager (and probably many other applications):
https://bugs.launchpad.net/ubuntu/+source/pygtk/+bug/707313

I have now built and tested DEBs rebuilt with pygtk-signal-watch.patch, you can find them here for both maverick and natty:
http://winswitch.org/dists/maverick/main/
http://winswitch.org/dists/natty/main/

If a local DoS isn't worthy of a security update, what is?
(the example is 4 lines of python so it's easy to test)

IKT (ikt) on 2011-03-12
Changed in pygtk (Ubuntu):
status: New → Confirmed

FYI time to apply the fix:

Gentoo: 0 days (~unstable)
FreeBSD: 15 days
Fedora: 2.5 months and counting
Ubuntu: almost 4 months and counting

Sebastien Bacher (seb128) wrote :

there is no diff nor patch to this bugs and ubuntu-sponsors are not subscribed so no wonder it's not getting reviewed...

summary: - 100% CPU usage when calling a child process from a python script
+ SRU: 100% CPU usage when calling a child process from a python script
Changed in pygtk (Ubuntu):
assignee: nobody → Chris Halse Rogers (raof)
Chris Halse Rogers (raof) wrote :
description: updated
Chris Halse Rogers (raof) wrote :

The attached debdiff cherry-picks the upstream commit which fixes this.

Changed in pygtk (Ubuntu):
assignee: Chris Halse Rogers (raof) → nobody
Cas (calumlind) wrote :

Great to see this will be fixed in Natty, thanks Chris.

Could this patch could also be backported to Maverick/Lucid?

Barry Warsaw (barry) wrote :

@Chris: the test case in comment #7 doesn't cause 100% cpu for me, so I'm not sure it's a proper test case for SRU. But the debdiff in comment #8 is obviously correct on the face of it, so it seems like a reasonable patch to apply.

Barry Warsaw (barry) wrote :

Oops, never mind, it *is* reproducible.

Barry Warsaw (barry) wrote :

Patch verified to work for me in Natty.

Accepted pygtk into natty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in pygtk (Ubuntu Natty):
status: Confirmed → Fix Committed
tags: added: verification-needed
Martin Pitt (pitti) wrote :

Please fix in oneiric so that this can proceed to -updates.

SRU verification for Natty:
I have reproduced the problem with pygtk 2.22.0-0ubuntu1 in natty and have verified that the version of pygtk 2.22.0-0ubuntu1.1 in -proposed fixes the issue.

Marking as verification-done

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package pygtk - 2.22.0-0ubuntu1.1

---------------
pygtk (2.22.0-0ubuntu1.1) natty-proposed; urgency=low

  * debian/patches/04_fix_100%_cpu_in_signal_handling.patch:
    - Upstream patch to fix thinko in pygtk signal handler glib source.
      Without this, the source _check function would read 0 bytes from the fd,
      resulting in the fd always having input and hence a busy-loop.
      Fixes 100% CPU usage in a variety of pygtk-using apps (LP: #664920)
 -- Christopher James Halse Rogers <email address hidden> Tue, 26 Apr 2011 17:56:56 +1000

Changed in pygtk (Ubuntu Natty):
status: Fix Committed → Fix Released
Martin Pitt (pitti) wrote :

Copied to oneiric as well.

Changed in pygtk (Ubuntu):
status: Confirmed → Fix Released
Martin Spacek (mspacek) on 2011-05-10
Changed in pygtk (Ubuntu Maverick):
status: New → Confirmed
demilord (demilord) wrote :

Will this get a fix released to maverick ?

Stéphane Graber (stgraber) wrote :

Uploaded to maverick-proposed, waiting for SRU team to review.

Stéphane Graber (stgraber) wrote :

I had a quick look at lucid but gtk.override seems really different and so I couldn't get the patch to apply.
We'd need someone with more pygtk knowledge to port the patch to lucid's

Changed in pygtk (Ubuntu Maverick):
status: Confirmed → Fix Committed
Damon Lynch (dlynch3) wrote :

In testing with my own python scripts I never saw the bug under Lucid. It appeared only with Maverick.

Martin Pitt (pitti) wrote :

Accepted pygtk into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

tags: removed: verification-done
tags: added: verification-needed
tags: added: testcase
Changed in pygtk (Ubuntu Lucid):
status: New → Won't Fix
Changed in pygtk (Ubuntu Maverick):
status: Fix Committed → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.