whoopsie-upload-all crashes while processing crash file

[Chris Gagnon]

summary:
whoopsie-upload-all crashes while processing crash file

steps:
1. have a crash file on mako/n4
2. touch /var/lib/apport/autoreport
3. run ubuntu-bug

channel: ubuntu-touch/utopic-proposed
build_number: 79
version_detail: ubuntu=20140612,device=20140530,version=79
apport:
  Installed: 2.14.3-0ubuntu1
  Candidate: 2.14.3-0ubuntu1

Collecting info for /var/crash/_usr_bin_unity8.32011.crash...
Traceback (most recent call last):
File "/usr/share/apport/whoopsie-upload-all", line 137, in <module>
stamps = collect_info()
File "/usr/share/apport/whoopsie-upload-all", line 88, in collect_info
res = process_report(r)
File "/usr/share/apport/whoopsie-upload-all", line 55, in process_report
r.add_gdb_info()
File "/usr/lib/python3/dist-packages/apport/report.py", line 677, in add_gdb_info
gdb_cmd = self.gdb_command(rootdir)
File "/usr/lib/python3/dist-packages/apport/report.py", line 1516, in gdb_command
self['CoreDump'].write(f)
File "/usr/lib/python3/dist-packages/problem_report.py", line 75, in write
block = gz.read(1048576)
File "/usr/lib/python3.4/gzip.py", line 365, in read
if not self._read(readsize):
File "/usr/lib/python3.4/gzip.py", line 449, in _read
self._read_eof()
File "/usr/lib/python3.4/gzip.py", line 482, in _read_eof
crc32, isize = struct.unpack("<II", self._read_exact(8))
File "/usr/lib/python3.4/gzip.py", line 286, in _read_exact
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached
Collecting info for /var/crash/_usr_bin_unity8.32011.crash...
Traceback (most recent call last):
File "/usr/share/apport/whoopsie-upload-all", line 137, in <module>
stamps = collect_info()
File "/usr/share/apport/whoopsie-upload-all", line 88, in collect_info
res = process_report(r)
File "/usr/share/apport/whoopsie-upload-all", line 55, in process_report
r.add_gdb_info()
File "/usr/lib/python3/dist-packages/apport/report.py", line 677, in add_gdb_info
gdb_cmd = self.gdb_command(rootdir)
File "/usr/lib/python3/dist-packages/apport/report.py", line 1516, in gdb_command
self['CoreDump'].write(f)
File "/usr/lib/python3/dist-packages/problem_report.py", line 75, in write
block = gz.read(1048576)
File "/usr/lib/python3.4/gzip.py", line 365, in read
if not self._read(readsize):
File "/usr/lib/python3.4/gzip.py", line 449, in _read
self._read_eof()
File "/usr/lib/python3.4/gzip.py", line 482, in _read_eof
crc32, isize = struct.unpack("<II", self._read_exact(8))
File "/usr/lib/python3.4/gzip.py", line 286, in _read_exact
raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

[Brian Murray (brian-murray)]

Do you still have this crash file available? /var/crash/_usr_bin_unity8.32011.crash Its likely that the crash file is corrupt / incomplete.

[Chris Gagnon (chris.gagnon)]

https://bugs.launchpad.net/lrt-crashes/+bug/1330201 has one of the unity8 crash files

https://bugs.launchpad.net/lrt-crashes/?field.searchtext=_usr_share_apport_whoopsie-upload-all.0.crash has a lot of duplicates of this bug

More crash can all be found at https://bugs.launchpad.net/lrt-crashes

[Brian Murray (brian-murray)]

It seems whoopsie-upload-all should probably have some error handling for invalid .crash files, it'd probably be similar to the following change.

revno: 2789
fixes bug: https://launchpad.net/bugs/1282349
committer: Martin Pitt <email address hidden>
branch nick: apport
timestamp: Wed 2014-04-23 09:10:37 +0200
message:
  * Move error handling for invalid .crash files into collect_info(), so that it also applies when using the "Show Details..." button in the UI. Otherwise the UI just hangs eternally at this point when encountering broken core dumps. (LP: #1282349)

[Chris Gagnon (chris.gagnon)]

jibel pointed out that apport might not have written the whole crash file, maybe whoopsie-upload-all can wait for apport to finish writing the file before it tries to upload.

[Chris Gagnon (chris.gagnon)]

Pitti said I can check the file permission to make sure it's finished.

[Martin Pitt (pitti)]

Fixed in trunk r2821, thanks Brian.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.14.5-0ubuntu1

---------------
apport (2.14.5-0ubuntu1) utopic; urgency=medium

  * New upstream bug fix release. Changes since last snapshot:
    - apt/dpkg get_file_package(): If checking for uninstalled packages, don't
      use package information from the host system, but always look it up in
      the correct indexes. Otherwise this returns wrong results when files
      move to a different package between releases. Thanks Brian Murray!
      (LP: #1336062)
    - apt/dpkg install_packages(): Disable fetching apt translation indexes,
      to save some bandwidth.
    - whoopsie-upload-all: Ignore crash reports with incomplete core dumps
      instead of crashing on them. Thanks Brian Murray. (LP: #1329520)
    - etc/default/apport: Fix comment to use "service" instead of calling the
      init.d script directly.
    - whoopsie-upload-all: Collect hooks information to gather ApportVersion,
      NonfreeKernelModules, and SystemImageInfo. Do this before collecting
      package data to minimize hook run time. (LP: #1349579)
    - Adjust report.test_get_logind_session test to work with systemd >= 205.
    - Fix report.test_check_interpreted_twistd test to skip instead of fail if
      twisted is not installed.

  * Drop automatix.py general hook. Automatix is long gone.
  * etc/apport/blacklist.d/apport: Drop long obsolete npviewer/npplayer
    entries.
  * Drop unnecessary python-gi build dependency.
  * Drop python-twisted-core build dependency. It's just used for one test,
    and will still run in autopkgtest. (Avoids current twisted
    uninstallability in main)
 -- Martin Pitt <email address hidden> Tue, 29 Jul 2014 12:26:23 +0200