whoopsie fails to immediately notice/process .upload files

Bug #1245524 reported by Paul Larson
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
whoopsie (Ubuntu)
Fix Released
High
Brian Murray
Saucy
Fix Released
High
Brian Murray

Bug Description

[Impact]
whoopsie does not immediately notice .upload files produced by apport after a crash report has all its data collected, and subsequently will not upload the .crash to the error tracker right away. Instead whoopsie will wait 2 hours and then check /var/crash/ for new crash files and then upload the crash report.

[Test Case]
1) sudo rm /var/crash/*
2) sudo service whoopsie restart
3) Install d-feet
4) (d-feet &); sleep 3; killall -SEGV d-feet
5) Observe apport crash dialog and choose to send crash report
6) Wait a bit for apport to collect data then ls -lh /var/crash/
7) Observe a .crash file and a .upload for d-feet and no .uploaded file
8) grep whoopsie /var/log/syslog and notice not "Parsing.*crash" message

With the version of whoopsie from saucy-proposed you should see a .uploaded file and the "Parsing /var/crash/.*crash" file message. After installing the version of apport from -proposed be sure to remove the files in /var/crash as whoopsie does an initial check on startup for files in /var/crash/ and processes them.

[Original Report]
------
We're trying to use the whoopsie-upload-all script in apport to ensure that all .crash files get uploaded in touch images. It seems that something is wrong with the inotify watch when running on the touch images though.

As a workaround, we've found that running whoopsie-upload-all, waiting for the .upload files to appear, then restarting whoopsie will unblock things.

We can work around this in ci for now, but this needs to be investigated further as it could cause problems down the road for allowing devices to upload crash data in the wild.

Evan (ev)
Changed in whoopsie (Ubuntu):
importance: Undecided → Critical
status: New → Confirmed
Revision history for this message
Brian Murray (brian-murray) wrote :

Are you calling whoopsie-upload-all manually or is it not running automatically? If it is not running automatically does /etc/apport/autoreport exist on the image?

Changed in whoopsie (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
Brian Murray (brian-murray) wrote :

Is this on trusty/ I'm noticing an issue where whoopsie-upload-all is creating the .upload files as it should but whoopsie fails to notice the .upload files until either whoopsie is restarted or a network_changed event (disconnect / reconnect from network) occurs.

summary: - whoopsie fails to notice/process .crash files on touch images
+ whoopsie fails to notice/process .upload files on trusty
Revision history for this message
Brian Murray (brian-murray) wrote : Re: whoopsie fails to notice/process .upload files on trusty

Digging into this (or what I'm hijacking this for) some it seems that there is an issue with whoopsie on-line check as changing the upstart job for whoopsie to use the --assume-online switch causes whoopsie to notice and process the .upload and .crash files.

Changed in whoopsie (Ubuntu):
status: Incomplete → Confirmed
tags: added: trusty
Changed in whoopsie (Ubuntu):
status: Confirmed → In Progress
assignee: nobody → Brian Murray (brian-murray)
Changed in whoopsie (Ubuntu Saucy):
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → Brian Murray (brian-murray)
importance: Critical → High
Changed in whoopsie (Ubuntu):
importance: Critical → High
description: updated
summary: - whoopsie fails to notice/process .upload files on trusty
+ whoopsie fails to immediately notice/process .upload files
Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Paul, or anyone else affected,

Accepted whoopsie into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/whoopsie/0.2.24.1ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in whoopsie (Ubuntu Saucy):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package whoopsie - 0.2.24.4

---------------
whoopsie (0.2.24.4) trusty; urgency=low

  * Close and reopen the log file so that the file descriptor is not closed
    when daemonizing. (LP: #1245524)
 -- Brian Murray <email address hidden> Tue, 21 Jan 2014 08:23:30 -0800

Changed in whoopsie (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

I installed whoopsie version 0.2.24.1ubuntu1 in a saucy virtual machine and confirm that the .crash file is uploaded and that a .uploaded file is created in /var/crash.

tags: added: verification-done-saucy
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package whoopsie - 0.2.24.1ubuntu1

---------------
whoopsie (0.2.24.1ubuntu1) saucy; urgency=low

  * Close and reopen the log file so that the file descriptor is not closed
    when daemonizing. (LP: #1245524)
  * Move whoopsie starting up message out of open_log so that it is not
    called multiple times when starting whoopsie.
  * Do not close the locking file descriptor when daemonizing.
  * Duplicate the APPORT_REPORT_DIR env as a string so that it is overwritten
    by the next call to g_getenv().
  * Use g_getenv when finding an enviornmental variable.
 -- Brian Murray <email address hidden> Mon, 20 Jan 2014 07:27:32 -0800

Changed in whoopsie (Ubuntu Saucy):
status: Fix Committed → Fix Released
Revision history for this message
Colin Watson (cjwatson) wrote : Update Released

The verification of the Stable Release Update for whoopsie has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.