SRU: 100% CPU usage when calling a child process from a python script

[Damon Lynch]

== Impact ==

pygtk apps which handle signals busy-loop in the glib main loop dispatch, causing 100% CPU usage after a signal is received. This commonly occurs when spawned child processes exit.

== TEST CASE ==

A simple testcase (pygtk-signal-testcase.py) is attached. To trigger the bug, run the testcase and send the process SIGCHLD. When run with pygtk 2.22-0ubuntu1 the test process will now consume 100% CPU.

== Regression Potential ==

PyGTK is core infrastructure, so the potential impact of a regression is high. However, the patch is small and clearly correct, and the file descriptor being read from is nonblocking, so it should not cause application hangs even if it tries to read more bytes than are there.

Original bugreport follows:
When creating a child process from a python script that uses pygtk 2.21, such subprocess.check_call(), 100% of the CPU is used (technically, 100% of the core the python interpreter is using).

If the creation of the child processes are removed, the script runs as expected.

According to user exarkun on the #python IRC channel, the problem may be manifesting when the child process exits. It may be related to a bug that was fixed in 9.10, but broken in 9.04. If that is the case, this is the relevant bug report: https://bugzilla.gnome.org/show_bug.cgi?id=481569

The code in which I noticed this problem is in the videometadata.py module of Rapid Photo Downloader: http://bazaar.launchpad.net/~dlynch3/rapid/trunk/annotate/head%3A/rapid/videometadata.py#L189

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: python-gtk2 2.21.0-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.35-22.35-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic x86_64
Architecture: amd64
Date: Thu Oct 21 22:51:42 2010
InstallationMedia: Ubuntu 10.10 "Maverick Meerkat" - Release amd64 (20101007)
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: pygtk

[Antoine Martin (antoine-nagafix)]

Here is the upstream bug:
https://bugzilla.gnome.org/show_bug.cgi?id=640738

I have tested the fix and it works. I believe Debian has already merged a patch for pygtk 2.17, Ubuntu should probably fix all versions, not just natty, as this affects maverick at least (I can confirm) and probably also lucid (I have not tested).

It also affects virt-manager (and probably many other applications):
https://bugs.launchpad.net/ubuntu/+source/pygtk/+bug/707313

[Antoine Martin (antoine-nagafix)]

I have now built and tested DEBs rebuilt with pygtk-signal-watch.patch, you can find them here for both maverick and natty:
http://winswitch.org/dists/maverick/main/
http://winswitch.org/dists/natty/main/

If a local DoS isn't worthy of a security update, what is?
(the example is 4 lines of python so it's easy to test)

[Antoine Martin (antoine-nagafix)]

FYI time to apply the fix:

Gentoo: 0 days (~unstable)
FreeBSD: 15 days
Fedora: 2.5 months and counting
Ubuntu: almost 4 months and counting

[Sebastien Bacher (seb128)]

there is no diff nor patch to this bugs and ubuntu-sponsors are not subscribed so no wonder it's not getting reviewed...

[Cas (calumlind)]

http://git.gnome.org/browse/pygtk/commit/?id=4cbd3c031289775f365c247f949d91facac4ba82

[Chris Halse Rogers (raof)]

The attached debdiff cherry-picks the upstream commit which fixes this.

[Cas (calumlind)]

Great to see this will be fixed in Natty, thanks Chris.

Could this patch could also be backported to Maverick/Lucid?

[Barry Warsaw (barry)]

@Chris: the test case in comment #7 doesn't cause 100% cpu for me, so I'm not sure it's a proper test case for SRU. But the debdiff in comment #8 is obviously correct on the face of it, so it seems like a reasonable patch to apply.

[Barry Warsaw (barry)]

Oops, never mind, it *is* reproducible.

[Barry Warsaw (barry)]

Patch verified to work for me in Natty.

[Martin Pitt (pitti)]

Accepted pygtk into natty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Please fix in oneiric so that this can proceed to -updates.

[Jean-Baptiste Lallement (jibel)]

SRU verification for Natty:
I have reproduced the problem with pygtk 2.22.0-0ubuntu1 in natty and have verified that the version of pygtk 2.22.0-0ubuntu1.1 in -proposed fixes the issue.

Marking as verification-done

[Launchpad Janitor (janitor)]

This bug was fixed in the package pygtk - 2.22.0-0ubuntu1.1

---------------
pygtk (2.22.0-0ubuntu1.1) natty-proposed; urgency=low

  * debian/patches/04_fix_100%_cpu_in_signal_handling.patch:
    - Upstream patch to fix thinko in pygtk signal handler glib source.
      Without this, the source _check function would read 0 bytes from the fd,
      resulting in the fd always having input and hence a busy-loop.
      Fixes 100% CPU usage in a variety of pygtk-using apps (LP: #664920)
 -- Christopher James Halse Rogers <email address hidden> Tue, 26 Apr 2011 17:56:56 +1000

[Martin Pitt (pitti)]

Copied to oneiric as well.

[demilord (demilord)]

Will this get a fix released to maverick ?

[Stéphane Graber (stgraber)]

Uploaded to maverick-proposed, waiting for SRU team to review.

[Stéphane Graber (stgraber)]

I had a quick look at lucid but gtk.override seems really different and so I couldn't get the patch to apply.
We'd need someone with more pygtk knowledge to port the patch to lucid's

[Damon Lynch (dlynch3)]

In testing with my own python scripts I never saw the bug under Lucid. It appeared only with Maverick.

[Antoine Martin (antoine-nagafix)]

For lucid, you need a different patch.

Probably one of those if it is using 2.17 rather than 2.2x:
http://patch-tracker.debian.org/patch/series/view/pygtk/2.17.0-4/60_pygtk-wakeupfd-fix.patch
https://bugzilla.redhat.com/attachment.cgi?id=477006&action=diff

[Martin Pitt (pitti)]

Accepted pygtk into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!