apport generates bug reports against the twisted project instead of a more relevant project

Bug #755025 reported by Jean-Paul Calderone
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
apport (Ubuntu)
Fix Released
Medium
Martin Pitt
Oneiric
Fix Released
Undecided
Unassigned

Bug Description

Sometimes apport generates a bug report like <https://bugs.launchpad.net/ubuntu/+source/twisted/+bug/748368>. I think it sets "Affects" to "twisted (Ubuntu)" because the command line tool which started the process is /usr/bin/twistd. However, twistd is a general purpose launcher, and in the case of the above linked bug, the most specific project involved is vodafone-mobile-connect. twistd is merely launching code from that project.

This is somewhat analogous to the fact that apport files the ticket against Twisted instead of against Python, even though the actual executable running is a Python interpreter.

Twisted does include some plugins which will be launched with twistd, so some twistd crashes are appropriate to file against Twisted. Any time the plugin being used is provided by some other project, though, I think the bugs should start out pointing at that other project.

There are basically two ways twistd might be directed to load someone's code. One is with the -y option (same as the --python option). This takes a path and loads some code from it. The other is with a positional argument, as in "twistd web" (though `web` is a plugin provided by Twisted itself).

SRU INFORMATION: This is a bit hard to reproduce, but as this causes needless crash reports against the twisted package, I'd still like to backport this fix into oneiric. It comes with test cases in the test suite which distill the apport information from the bugs referenced here, so if that succeeds, this will be fixed. You can run it with "/usr/share/apport/testsuite/run", this bug is covered in the "apport/report" part. (There might be some tests in other files which fail, due to apport not being enabled or running in an environment with not enough dependencies, etc., though).

Revision history for this message
Jean-Paul Calderone (exarkun) wrote :
Revision history for this message
Jean-Paul Calderone (exarkun) wrote :

Another one of these arrived in my inbox, this one for https://bugs.launchpad.net/bugs/763181

Revision history for this message
Jean-Paul Calderone (exarkun) wrote :
Revision history for this message
Glyph Lefkowitz (glyph) wrote :

Another one just arrived for me:

    https://bugs.launchpad.net/bugs/725383

Revision history for this message
Martin Pitt (pitti) wrote :

Hello Jean-Paul, thanks for your report! I agree we should special-case twistd here and do the command line -> package mapping differently. From the bugs you pointed out I see for example:

/usr/bin/python /usr/bin/twistd -r gtk2 --pidfile /tmp/vmc.pid -noy /usr/share/vodafone-mobile-connect/gtk-tap.py -l /dev/null
/usr/bin/python /usr/bin/twistd --uid root --gid root --pidfile /var/run/nanny.pid -r glib2 --logfile /var/log/nanny.log -y /usr/share/nanny/daemon/nanny.tap
ProcCmdline: /usr/bin/python /usr/bin/twistd --pidfile=/var/run/poker-network-server.pid --logfile=/var/log/poker-network-server.log --no_save --reactor=poll pokerserver

So from the manpage I see that if we find a -f or -y option, it's value should be considered as the actual program that it runs, and thus crash reports should go to that program. That would work for the first two examples.

The third one (which has tons of dupes apparently) looks more tricky, though: There is no -y or -f, just a standalone command line argument "pokerserver". I assume that's the command that should be run? but this syntax isn't documented in the twistd manpage.

Thanks for any insight!

Changed in apport:
status: New → Incomplete
Revision history for this message
Jean-Paul Calderone (exarkun) wrote :

The third one is an example of the plugin feature supported by twistd. One way to map this back to a file (which can then be mapped to a packet, I suppose) is like this:

    from twisted.plugin import getPlugins
    from twisted.application.service import IServiceMaker
    from twisted.python.modules import getModule

    [plg] = [p for p in getPlugins(IServiceMaker) if p.tapname == 'pokerserver']
    print getModule(plg.tapname).filePath.path

I think this is a fairly good way, but I haven't tested it extensively. One issue with it is that it is pointing to the file that contains the code implementing the plugin, not the file that contains the actual plugin definition itself. Usually these two files belong to the same package, but one could imagine someone doing something funny and breaking this assumption. It's probably a pretty good heuristic for apport, at least.

Changed in apport:
status: Incomplete → New
Martin Pitt (pitti)
Changed in apport:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Martin Pitt (pitti)
Revision history for this message
Jean-Paul Calderone (exarkun) wrote :

Some more of these in my mailbox this morning:

  https://bugs.launchpad.net/bugs/768582
  https://bugs.launchpad.net/bugs/769879
  https://bugs.launchpad.net/bugs/767684

Any progress on this?

Revision history for this message
Martin Pitt (pitti) wrote :

I collected the linked bugs into the test case, and fixed this in trunk r1989. Sorry for the delay!

affects: apport → apport (Ubuntu)
Changed in apport (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Jean-Paul Calderone (exarkun) wrote :

Thank you very much!

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (4.3 KiB)

This bug was fixed in the package apport - 1.24-0ubuntu1

---------------
apport (1.24-0ubuntu1) precise; urgency=low

  * New upstream release 1.23.1:
    - apport/crashdb.py: Ensure that duplicate table only has one entry per
      report ID.
    - apport-retrace: Pass correct executable path to gdb in --gdb with
      --sandbox mode.
    - apport-retrace: Do not leave behind temporary directories on errors.
    - apport-retrace: Drop assertion failure for existance of "Stacktrace".
      This isn't present in the case of gdb crashing, and there is not much we
      can do about it. This should not break the retracer.
    - apport/report.py: Unwind XError() from stack traces for the
      "StacktraceTop" field, as they take a significant part of the trace.
      This causes bugs to be duplicated which really have different causes.
  * New upstream release 1.24:
   - apport-retrace: Add --timestamp option to prepend a timestamp to log
     messages. This is useful for batch operations.
   - crash-digger: Call apport-retrace with --timestamps, to get consistent
     timestamps in log output.
   - hookutils.py: Add two new functions attach_gsettings_package() and
     attach_gsettings_schema() for adding user-modified gsettings keys to a
     report. (LP: #836489)
   - hookutils.py: Add new function in_session_of_problem() which returns
     whether the given report happened in the currently running XDG session.
     This can be used to determine if e. g. ~/.xsession-errors is relevant and
     should be attached.
   - backends/packaging-apt-dpkg.py, install_packages(): Also copy
     apt/sources.list.d/ into sandbox.
   - backends/packaging-apt-dpkg.py, install_packages(): Install apt keyrings
     from config dir or from system into sandbox. (LP: #856216)
   - packaging.py, backends/packaging-apt-dpkg.py: Define that
     install_packages() should return a SystemError for broken
     configs/unreachable servers etc., and fix the apt/dpkg implementation
     accordingly.
   - apport-retrace: Don't crash, just give a proper error message if servers
     are unreachable, or configuration files are broken. (LP: #859248)
   - backends/packaging-apt-dpkg.py: Fix crash when
     /etc/apport/native-origins.d contains any files. (LP: #865199)
   - hookutils, recent_logfile(): Fix invalid return value if log file is not
     readable. (LP: #819357)
   - test/crash: Fix race condition in the "second instance terminates
     immediately" check.
   - hookutils.py: Replace attach_gconf() with a no-op stub. It used static
     python modules like "gconf" which broke the PyGI GTK user interface, and
     gconf is rather obsolete these days.
   - ui.py, open_url(): Greatly simply and robustify by just using xdg-open.
     This already does the right thing wrt. reading the default browser from
     GNOME, KDE, XCE, and other desktops. (LP: #198449)
   - data/general-hooks/generic.py: Only attach ~/.xsession_errors if the bug
     is reported in the same XDG session as the crash happened. (LP: #869974)
   - Ignore crashes for programs which got updated in between the crash and
     reporting. (LP: #132904)
   - Special-case crashes of 'twistd': Try to determin...

Read more...

Changed in apport (Ubuntu):
status: Fix Committed → Fix Released
Martin Pitt (pitti)
description: updated
Changed in apport (Ubuntu Oneiric):
status: New → Fix Committed
Martin Pitt (pitti)
description: updated
Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Jean-Paul, or anyone else affected,

Accepted apport into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 1.23-0ubuntu4

---------------
apport (1.23-0ubuntu4) oneiric-proposed; urgency=low

  * apport-gtk: In bug update mode, make details dialog resizable and fix
    default size. Patch cherry-picked from trunk r1991. (LP: #865754)
  * backends/packaging-apt-dpkg.py: Fix crash when
    /etc/apport/native-origins.d contains any files. Patch cherry-picked from
    trunk r1973. (LP: #865199)
  * apport/report.py: Special-case crashes of 'twistd': Try to determine the
    client program and assign the report to that, or fail with an
    UnreportableReason. Patch cherry-picked from trunk r1989. (LP: #755025)
  * hookutils.py, attach_wifi(): Anonymize ESSID and AP MAC from "iwconfig"
    output. Patch cherry-picked from trunk 1993. (LP: #746900)
 -- Martin Pitt <email address hidden> Thu, 20 Oct 2011 12:08:11 +0200

Changed in apport (Ubuntu Oneiric):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.