traceback in ubuntu.py's trim_dpkg_log function

[Scott Moser]

smoser@milhouse:~/src/maas-images/trunk$ sudo apport-bug /var/crash/libtesseract4.0.crash
[sudo] password for smoser:
sys:1: PyGIWarning: Wnck was imported without specifying a version first. Use gi.require_version('Wnck', '3.0') before import to ensure that the right version gets loaded.
sys:1: PyGIWarning: GdkX11 was imported without specifying a version first. Use gi.require_version('GdkX11', '3.0') before import to ensure that the right version gets loaded.
ERROR: hook /usr/share/apport/general-hooks/ubuntu.py crashed:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport/report.py", line 197, in _run_hook
    symb['add_info'](report, ui)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 66, in add_info
    check_attachment_for_errors(report, attachment)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 170, in check_attachment_for_errors
    trim_dpkg_log(report)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 468, in trim_dpkg_log
    report['DpkgTerminalLog'] = '\n'.join([str(line.encode('utf-8')) for line in lines])
  File "/usr/share/apport/general-hooks/ubuntu.py", line 468, in <listcomp>
    report['DpkgTerminalLog'] = '\n'.join([str(line.encode('utf-8')) for line in lines])
AttributeError: 'bytes' object has no attribute 'encode'

seems like a python2 to python3 conversion bug.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: apport 2.20-0ubuntu3
ProcVersionSignature: Ubuntu 4.4.0-2.16-generic 4.4.0
Uname: Linux 4.4.0-2-generic x86_64
NonfreeKernelModules: zfs zunicode zcommon znvpair zavl
ApportLog:

ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CrashReports:
 640:0:116:846362:2016-02-22 12:27:09.709648847 -0500:2016-02-22 12:27:10.709648847 -0500:/var/crash/libtesseract4.0.crash
 644:0:116:0:2016-02-22 12:27:11.629635693 -0500:2016-02-22 12:27:11.629635693 -0500:/var/crash/libtesseract4.0.upload
CurrentDesktop: Unity
Date: Mon Feb 22 12:35:39 2016
EcryptfsInUse: Yes
InstallationDate: Installed on 2015-07-23 (214 days ago)
InstallationMedia: Ubuntu 15.10 "Wily Werewolf" - Alpha amd64 (20150722.1)
PackageArchitecture: all
SourcePackage: apport
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Reis (kiko)]

Same here; upgraded today from Trusty to Xenial and crashed reporting bug 1570966.

[Robie Basak (racb)]

This is resulting in "HookError_ubuntu.txt" being uploaded to apport-filed reports. Example: bug 1562362. This example is with apport 2.20-0ubuntu3 though, which is not the latest in Xenial.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.20.1-0ubuntu4

---------------
apport (2.20.1-0ubuntu4) yakkety; urgency=medium

  * data/general-hooks/ubuntu.py: Fix stacktrace when parsing
    DpkgTerminalLog.txt. (LP: #1548421)
  * data/general-hooks/ubuntu.py: Restore starting package problem duplicate
    signatures with the word package, the package name, and its version.
    (LP: #1581682)

 -- Brian Murray <email address hidden> Mon, 16 May 2016 14:16:15 -0700

[Martin Pitt (pitti)]

Hello Scott, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

With the old version of the general ubuntu hook we get a traceback:

 $ python3 data/general-hooks/ubuntu.py ~/source-trees/bugpatterns/master/report-1581664.crash
Traceback (most recent call last):
  File "data/general-hooks/ubuntu.py", line 494, in <module>
    add_info(new_report, None)
  File "data/general-hooks/ubuntu.py", line 66, in add_info
    check_attachment_for_errors(report, attachment)
  File "data/general-hooks/ubuntu.py", line 170, in check_attachment_for_errors
    trim_dpkg_log(report)
  File "data/general-hooks/ubuntu.py", line 468, in trim_dpkg_log
    report['DpkgTerminalLog'] = '\n'.join([str(line.encode('utf-8')) for line in lines])
  File "data/general-hooks/ubuntu.py", line 468, in <listcomp>
    report['DpkgTerminalLog'] = '\n'.join([str(line.encode('utf-8')) for line in lines])
AttributeError: 'bytes' object has no attribute 'encode'

Using apport from -proposed we see:

 $ python3 /usr/share/apport/general-hooks/ubuntu.py ~/source-trees/bugpatterns/master/report-1581664.crash
dpkg-query: no packages found matching kaccounts-providers
0 items changed

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.20.1-0ubuntu2.1

---------------
apport (2.20.1-0ubuntu2.1) xenial-proposed; urgency=medium

  * data/general-hooks/ubuntu.py: Fix stacktrace when parsing
    DpkgTerminalLog.txt. (LP: #1548421)
  * data/general-hooks/ubuntu.py: Restore starting package problem duplicate
    signatures with the word package, the package name, and its version.
    (LP: #1581682)

 -- Brian Murray <email address hidden> Mon, 16 May 2016 15:02:01 -0700

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for apport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Brian Murray (brian-murray)]

The fix for this doesn't seem to work in every case. For example:

[ 1:00PM 10791 ] [ bdmurray@impulse:~/source-trees/bugpatterns/master ]
 $ ./download-crash 1571220
rusage before getting report: 25740
rusage after getting report: 51840
rusage after writing report: 51840
LP: #1571220: No match
[ 1:02PM 10792 ] [ bdmurray@impulse:~/source-trees/bugpatterns/master ]
 $ python3 /usr/share/apport/general-hooks/ubuntu.py ~/source-trees/bugpatterns/master/report-1571220.crash
Traceback (most recent call last):
  File "/usr/share/apport/general-hooks/ubuntu.py", line 502, in <module>
    add_info(new_report, None)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 70, in add_info
    check_attachment_for_errors(report, attachment)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 176, in check_attachment_for_errors
    trim_dpkg_log(report)
  File "/usr/share/apport/general-hooks/ubuntu.py", line 476, in trim_dpkg_log
    report['DpkgTerminalLog'] = '\n'.join([str(line.decode('utf-8')) for line in lines])
  File "/usr/share/apport/general-hooks/ubuntu.py", line 476, in <listcomp>
    report['DpkgTerminalLog'] = '\n'.join([str(line.decode('utf-8')) for line in lines])
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd1 in position 101: invalid continuation byte

This was on yakkety but the code is the zesty and xenial.

[Benjamin Drung (bdrung)]

I checked /usr/share/apport/general-hooks/ubuntu.py in Ubuntu 18.04 (bionic) and 22.04 (jammy) and they are fixed:

```
        report['DpkgTerminalLog'] = '\n'.join([str(line.decode('UTF-8', 'replace')) for line in lines])
```