apport-bug crashing on server install

[Brad Figg]

[impact]

This bug prevents apport from reporting apparmor bugs on systems with
non-UTF-8 characters showing up in syslog (the kernel reporting raw
information from hardware is a common culprit).

[steps to reproduce]

1) on a system with non-UTF-8 characters in syslog, run 'apport -p apparmor -f'
   if the bug has been addressed, this should generate a report as
   expected instead of causing apport to crash.

[regression potential]

The change in the patch for this bug solely modifies the apparmor
apport hook. It could potentially introduce more breakage there (but
couldn't affect apparmor mediation); also the lp:qa-regression-testign
test-apparmor.py attempts to exercise the apport hook, and this is
exercised in as part of every linux kernel update by the kernel team
(who frequently hit this issue due to the quirkiness of some of
their hardware).

[original description]

jenkins@dagmar:~$ sudo apport-bug apparmor --save testlib.apport

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
.....ERROR: hook /usr/share/apport/package-hooks/source_apparmor.py crashed:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport/report.py", line 197, in _run_hook
    symb['add_info'](report, ui)
TypeError: add_info() takes 1 positional argument but 2 were given

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport/report.py", line 202, in _run_hook
    symb['add_info'](report)
  File "/usr/share/apport/package-hooks/source_apparmor.py", line 48, in add_info
    report['KernLog'] = recent_kernlog(sec_re)
  File "/usr/share/apport/package-hooks/source_apparmor.py", line 24, in recent_kernlog
    for line in open(file):
  File "/usr/lib/python3.4/codecs.py", line 313, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd2 in position 6574: invalid continuation byte

[Steve Beattie (sbeattie)]

The first part of the traceback is a bit of a red herring; it's an expected exception that is being handled: apport now wants to pass two arguments to add_info(), but the apparmor hook hasn't been updated to take into account the added ui argument, so apport falls back to invoking add_info() with one argument. This wouldn't be an issue except that the second exception is being raised. That said, the apparmor hook add_info() method should be updated to take the additional ui argument.

The second exception is because non-utf-8 strings are in /var/log/kern.log and the apparmor apport hook isn't handling that properly (the non-utf-8 strings are specific to the hardware the tests are being run on, hence the reason it doesn't fail everywhere).

[Steve Beattie (sbeattie)]

Okay, here is a patch to the apparmor apport hook that should address the trace back on finding non utf-8 bytes in the system logs.

[Seth Arnold (seth-arnold)]

The patch looks good to me. (I was worried that treating the log as latin-1 would fail, since iso-8859-1 doesn't describe 0x8? or 0x9? characters, but some manual testing seemed to indicate that Python's codecs.latin_1_import() didn't throw any exceptions to them as input.)

It'd make sense to get this in before release if we can to help make sure our errors.ubuntu.com logs are as useful as possible; but the average user shouldn't notice it either way...

Thanks

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "apparmor-apport_hook_update.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.8.98-0ubuntu2

---------------
apparmor (2.8.98-0ubuntu2) utopic; urgency=medium

  * Updated to apparmor 2.9.beta4 (aka apparmor 2.8.98)
    - fix logparsing memory leak (LP: #1340927)
    - incorporate fixes to regression testsuite to compensate for
      af_unix mediation, as well as extend test coverage
      (LP: #1375403, LP: #1375516)
    - fix libapparmor's log parsing code to accept additional rejection
      types (LP: #1375413)
    - fix X abstraction for changed lightdm xauthority file locations
      (LP: #1339727)
    - parser: disable downgrade and not enforced rule messages
      by default
    - fix error when using regex profile names in IPC rules
      (LP: #1373085)
    - updates and fixes to the python utilities
    - translation updates

  [ Steve Beattie ]
  * Removed upstreamed patches:
    drop-peer_addr-with-local-addr-in-base.patch,
    update_socketpair_tests_for_af_unix.patch,
    fix_socketpair_tests.patch, sanitized-helpers-updates.patch,
    01-tests-unix_socket_lists.patch,
    02-tests-accept_unix_rules_in_mkprofile.patch,
    03-tests-unix_sockets_v7_pathnames.patch,
    04-tests-migrate_from_poll_to_sockio_timeout.patch,
    05-tests-add_abstract_socket_tests.patch,
    06-tests-use_socketpair_and_none.patch,
    07-parser-fix_local_perms.patch,
    08-phpsysinfo-policy-updates.patch,
    09-apache2-policy-instructions.patch,
    10-lp1371771.patch, 11-lp1371765.patch,
    lp1169881.patch
  * refreshed etc-writable.patch and libapparmor-layout-deb.patch
  * debian/control: add breaks on python3-apparmor against older
    apparmor-utils that used to be where python bits lived
    (LP: #1373259)
  * debian/apport/source_apparmor.py:
   - fixes the apparmor apport hook so it does not raise an exception if
     a non-unicode character is found in /var/log/kern.log or in
     /var/log/syslog. This should work under python3 or python2.7
     (LP: #1304447)
   - adjusts the add_info() function to take the expected additional ui
     argument, though it has no need for it.
   - converts the log parsing code to use with statements so as not to
     leak open file descriptors
   - updates the set of packages to query to see if installed and if so,
     report the version of.
   - adjust import to make pyflakes job easier
   - minor pep8 cleanups

  [ Jamie Strandboge ]
  * add-chromium-browser.patch: don't allow writing to the oom score and
    adjust files since this allows chromium to change the values for any
    process matching our UID
  * debian/apparmor.upstart: check if click-apparmor md5sums changed so we
    regenerate the policy if it changes too (LP: #1371574)
  * debian/apparmor.init: make corresponding upstart change to initscript
  * debian/lib/apparmor/functions: fall back to using -n1 if the parser failed
    to load a profile set. This should be removed when the parser properly
    handles profile sets with corrupted profiles (LP: 1377338)
  * debian/control: fix typo (LP: #1187447)
 -- Steve Beattie <email address hidden> Thu, 09 Oct 2014 22:39:32 -0700

[Steve Beattie (sbeattie)]

Attached is the changes as part of an SRU for trusty.

[Brad Figg (brad-figg)]

My running of the QRT apparmor tests is now passing on Trusty.

[Steve Beattie (sbeattie)]

Thanks, Brad. Marking verification-done.

[Adam Conrad (adconrad)]

The verification of the Stable Release Update for apparmor has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.