apport-collect crashes for certain dates/locales

[Martin Olsson]

As part of the Ubuntu triage process I ask a lot of people to run "apport-collect bug_no". However, it seems that for a significant number of people this script crashes when they run it:

For example within the last hour, the happened for two persons already:
https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/367694
https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/367824

get fences failed: -1
param: 6, val: 0
Uploading additional information to Launchpad bug...
   short text data...
   attachment: Dependencies.txt...
   attachment: LsHal.gz...
Traceback (most recent call last):
  File "/usr/bin/apport-collect", line 173, in <module>
    upload(report, bug)
  File "/usr/bin/apport-collect", line 97, in upload
    filename=part.get_filename(), is_patch=False)
  File "/usr/lib/python2.6/dist-packages/launchpadlib/resource.py", line 337, in __call__
    args[key] = simplejson.dumps(value, cls=DatetimeJSONEncoder)
  File "/var/lib/python-support/python2.6/simplejson/__init__.py", line 243, in dumps
    **kw).encode(obj)
  File "/var/lib/python-support/python2.6/simplejson/encoder.py", line 360, in encode
    return encode_basestring_ascii(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: unexpected code byte

And also this one:
Collecting apport information for source package xorg...
hook /usr/share/apport/package-hooks//source_xorg.py crashed:
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/apport/report.py", line 570, in add_hooks_info
    symb['add_info'](self)
  File "/usr/share/apport/package-hooks//source_xorg.py", line 104, in add_info
    report['fglrx-loaded'] = matches
  File "/usr/lib/python2.6/dist-packages/problem_report.py", line 479, in __setitem__
    assert k.replace('.', '').isalnum()
AssertionError
Uploading additional information to Launchpad bug...
   short text data...
   attachment: Dependencies.txt...
   attachment: LsHal.txt...
   attachment: LsMod.txt...
   attachment: LsPci.txt...
   attachment: XorgConf.txt...
   attachment: XorgLog.txt...
Traceback (most recent call last):
  File "/usr/bin/apport-collect", line 173, in <module>
    upload(report, bug)
  File "/usr/bin/apport-collect", line 97, in upload
    filename=part.get_filename(), is_patch=False)
  File "/usr/lib/python2.6/dist-packages/launchpadlib/resource.py", line 337, in __call__
    args[key] = simplejson.dumps(value, cls=DatetimeJSONEncoder)
  File "/var/lib/python-support/python2.6/simplejson/__init__.py", line 243, in dumps
    **kw).encode(obj)
  File "/var/lib/python-support/python2.6/simplejson/encoder.py", line 360, in encode
    return encode_basestring_ascii(o)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 12396: unexpected code byte

[Brian Murray (brian-murray)]

In one of the bug reports the language was set to:

LANG=sv_SE.UTF-8

However, the other one had en_US.UTF-8 and I've never had a problem with that.

[Martin Olsson (mnemo)]

Also in bug 366503 user "dori" reports apport-collect failing:
https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-ati/+bug/366503
Specifically in this comment:
https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-ati/+bug/366503/comments/23

error: dbus_bus_get: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
hook /usr/share/apport/package-hooks//source_xserver-xorg-video-ati.py crashed:
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/apport/report.py", line 570, in add_hooks_info
    symb['add_info'](self)
  File "/usr/share/apport/package-hooks//source_xserver-xorg-video-ati.py", line 104, in add_info
    report['fglrx-loaded'] = matches
  File "/usr/lib/python2.6/dist-packages/problem_report.py", line 479, in __setitem__
    assert k.replace('.', '').isalnum()
AssertionError

[Martin Pitt (pitti)]

  File "/usr/share/apport/package-hooks//source_xserver-xorg-video-ati.py", line 104, in add_info
    report['fglrx-loaded'] = matches

This part is a bug in the -ati package hook. '-' is not allowed as a report key, you can only use alphanumeric and '.'

[Launchpad Janitor (janitor)]

This bug was fixed in the package xorg - 1:7.4~5ubuntu19

---------------
xorg (1:7.4~5ubuntu19) karmic; urgency=low

  * debian/apport/source_xorg.py: Rename "fglrx-loaded" report field to
    "FglrxLoaded"; '-' is not a valid character in report keys. (LP: #368004)

 -- Martin Pitt <email address hidden> Tue, 05 May 2009 01:17:58 +0200

[Martin Pitt (pitti)]

The root cause for those is a bug in launchpadlib, see bug 353805. I'll add a workaround to apport.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 1.1.1-0ubuntu2

---------------
apport (1.1.1-0ubuntu2) karmic; urgency=low

  [ Martin Pitt ]
  * hookutils.py: Do not attach /proc/version_signature, it's Ubuntu specific.
    (Merged from trunk). Instead, attach it in general-hooks/ubuntu.py.
  * general-hooks/ubuntu.py: Attach package conffile information.
  * debian/local/apport-collect: Add workaround for launchpadlib bug
    LP#353805, to avoid crashing with non-UTF8 attachments. (LP: #368004)
  * debian/local/apport-collect: Fix import of launchpadlib's HTTPError.
  * apport/hookutils.py, attach_conffiles(): Ignore empty lines from
    dpkg-query output.
  * general-hooks/ubuntu.py: Strip off '/target' prefix from ExecutablePath
    and InterpreterPath, to correctly process crash reports from the live
    system installer.
  * apport/hookutils.py, attach_conffiles(): Do not use command_output(),
    since that causes error messages to get parsed as conffiles. Use
    subprocess properly.
  * backends/packaging-apt-dpkg.py: Replace deprecated python-apt properties
    with current ones (merged from trunk). Update python-apt dependency to
    >= 0.7.9.
  * packaging-apt-dpkg.py, get_modified_files(): Do not show package list file
    as modified if the package is not installed (merged from trunk).
    (LP: #364533)
  * backends/packaging-apt-dpkg.py, install_retracing_packages(): Fix syntax
    error which broke the retracers.

  [ Andy Whitcroft ]
  * bin/apportcheckresume: the suspend _and_ hibernate logs are both in
    pm-suspend.log.
  * bin/apportcheckresume: remove redunant check for file before attaching
    stress log.

 -- Martin Pitt <email address hidden> Wed, 13 May 2009 15:22:53 +0200