apport-unpack cannot load gitkraken crash report - 'ascii' codec can't decode byte 0xfc in position 56

[Mingun]

Impact
======

apport-unpack and whoopsie-upload-all can crash when fed with a malformed problem report. They should print a proper error message instead of crashing. This bug does not happen that often, but the fix for this bug is the foundation for the fix for bug #1995100.

Test Plan
=========

1. Create malformed report:
```
printf 'AB\xfc:CD\n' > malformed.crash
```
2. Try to unpack it:
```
apport-unpack malformed.crash unpack
```

It should print an error message instead of a stack trace.

Where problems could occur
==========================

The patch touches the problem report handling and therefore can affect apport-unpack, apport-bug, whoopsie-upload-all. The change is covered with tests and apport has a test suite running during build and as autopkgtest.

Original report
===============

The latest GitKraken client (8.10.2 x64) is crashed every minute in cycle when you doing a rebase. When I try to inspect crash log to find the reason, the apport-unpack is not be able to process it:

/var/lib/apport/coredump$ sudo apport-unpack core._snap_gitkraken_199_usr_share_gitkraken_gitkraken.1000.5de91c04-23ab-45c8-933c-0466d56e9fad.135785.50225891 unpack
[sudo] пароль для mingun:
Traceback (most recent call last):
  File "/usr/bin/apport-unpack", line 59, in <module>
    pr.load(f, binary=False)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 194, in load
    key = key.decode('ASCII')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 56: ordinal not in range(128)

Unfortunately, crash file is too big (~9GB) and does not compress further

[Benjamin Drung (bdrung)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

apport-unpack is designed for unpackaging .crash files (in /var/crash). You specified a core file which contains only binary data and is expected to fail. apport-unpack should print an error message in this case.

[Benjamin Drung (bdrung)]

Upstream merge request: https://github.com/canonical/apport/pull/24

[Benjamin Drung (bdrung)]

The fix got merged to main: https://github.com/canonical/apport/commit/4dbc5f9d16f0d652a360e3ca4827c69954b61241

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.24.0-0ubuntu1

---------------
apport (2.24.0-0ubuntu1) lunar; urgency=medium

  * New upstream bug-fix release.
    - Catch malformed problem reports (LP: #1996040)
    - Catch ValueError: not enough values to unpack (LP: #1995100)
    - Catch FileNotFoundError for missing desktop files (LP: #1997753)
    - Catch binascii.Error: Incorrect padding (LP: #1997912)
    - Catch AttributeError: NoneType object has no attribute origins
      (LP: #1997973)
  * Drop cherry-picked upstream patches
  * oem-getlogs: Use colon instead of a dot for chown command
  * Remove redundant section from apport-noui
  * Clarify that apport-noui is empty
  * Run black, isort, pydocstyle, pylint during package build

 -- Benjamin Drung <email address hidden> Thu, 08 Dec 2022 01:37:20 +0100

[Andreas Hasenack (ahasenack)]

If the intent was to have apport-unpack deal with any invalid crash file, then it's not complete. This still happens now:
ubuntu@k-apport:~$ apport-unpack /bin/ls unpack
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/problem_report.py", line 174, in load
    (key, value) = line.split(b":", 1)
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/apport-unpack", line 66, in <module>
    pr = load_report(args.report)
  File "/usr/bin/apport-unpack", line 47, in load_report
    pr.load(f, binary=False)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 177, in load
    f"Malformed problem report: Line {line.decode()!r}"
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 24: invalid start byte

Is this something you would still like to address?

[Benjamin Drung (bdrung)]

Let's reopen this bug report again.

[Benjamin Drung (bdrung)]

Upstream merge request to fix that case: https://github.com/canonical/apport/pull/52

[Benjamin Drung (bdrung)]

https://github.com/canonical/apport/pull/52 and https://github.com/canonical/apport/pull/53 got merged into main.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.26.0-0ubuntu2

---------------
apport (2.26.0-0ubuntu2) lunar; urgency=medium

  * fix: Mock query to Github in integration test

 -- Benjamin Drung <email address hidden> Fri, 24 Feb 2023 16:17:43 +0100

[Steve Langasek (vorlon)]

I'd like to understand better exactly how it happened that the user got this crash. Is it a matter of a corrupted .crash file on input, or is it that the user tried to feed in a file that was not intended to be input for apport-unpack? I see that the command line that triggered this appears to be taking a 'core' file as input rather than a file ending in .crash, so I'm suspecting the latter.

If it's the latter, then I wouldn't normally consider this SRU-worthy (though it's borderline to consider it as part of a larger SRU).

If it's the former, then I would be concerned about why the invalid input happened in the first place, which I don't see addressed here.

[Benjamin Drung (bdrung)]

I have not enough data to answer that question. Feeding a core dump into apport-unpack can let it crash (user error). I don't know if *all* of those crashes were caused by users feeding a crash file into apport-unpack. The crash reports on errors.ubuntu.com do not contain the input file and I haven't seen a bug report where a malformed crash report was attached.

[Mingun (alexander-sergey)]

This happened in that way as I've originally posted: after some actions in the GitKraken client it starts continually crashed, core files are appeared in the /var/lib/apport/coredump. Because this file was created inside apport folder, i thought it was in apports .crash file format and it should be able to open it

[Steve Langasek (vorlon)]

Hello Mingun, or anyone else affected,

Accepted apport into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.23.1-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Benjamin Drung (bdrung)]

Verified:

```
$ printf 'AB\xfc:CD\n' > malformed.crash
$ apport-unpack malformed.crash unpack
ERROR: Malformed problem report: 'ascii' codec can't decode byte 0xfc in position 2: ordinal not in range(128). Is this a proper .crash text file?
```

[Andreas Hasenack (ahasenack)]

I verified the test results and am satisfied that they show the executed planned test case, and that the results are correct.

The package built correctly in all architectures and Ubuntu releases it was meant for.

There are no DEP8 regressions.

There is no SRU freeze ongoing at the moment.

There is no halted phasing on the previous update.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.23.1-0ubuntu3.1

---------------
apport (2.23.1-0ubuntu3.1) kinetic; urgency=medium

  * Declare python3-apport breaking older apport/apport-gtk/apport-kde
    (LP: #1997759)
  * Catch malformed problem report (LP: #1996040)
  * Catch ValueError: not enough values to unpack (LP: #1995100)

 -- Benjamin Drung <email address hidden> Thu, 08 Dec 2022 02:38:15 +0100

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for apport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.