Support kernel OOPSes

[Evan]

There are two types of kernel reports that we track: KernelCrashes and KernelOopses (https://bugs.launchpad.net/ubuntu/+source/linux/+bugs?field.tag=kernel-oops). Both of these are inadvertently filtered out on daisy.ubuntu.com (lp:daisy) because they lack a StacktraceAddressSignature.

We will not collect KernelCrashes for the reasons mentioned in the IRC context below. Fixing KernelOopses requires writing a signature that instances of an OOPS problem can be grouped against.

My thought is that the concatenation of the type, instruction pointer and topmost 5 functions that are not prefixed by a question mark will form a sufficient signature. So for the following OOPS, we would generate the following signature:

https://launchpadlibrarian.net/79771442/OopsText.txt

kernel paging request:ext4_get_acl+0x80/0x210:d_splice_alias+0x40/0x50:ext4_check_acl+0x4a/0x90:acl_permission_check+0x97/0xa0:generic_permission+0x25/0xc0:inode_permission+0x99/0xd0

Further context from IRC:
[17:05:38] <apw> ev, do we normally get cores from crashes, that'd not be a common config
[17:06:08] <apw> and something we'd not want to be sending anywhere somewhere i suspect
[17:06:56] <ev> ah, interesting. What are you looking for, and what would be a reasonable algorithm for the signature of it?
[17:07:22] <apw> there is a common oops format in dmesg/messages
[17:07:42] <apw> we are sending those in already though right, in the apport enabled world
[17:07:44] <ev> oh I'm misreading my own email
[17:07:51] <ev> right
[17:07:55] <ev> indeed, we are sending oopses through
[17:08:02] <ev> but we don't have a signature for them
[17:08:20] <apw> i'd assume signatures could be done the way you do them for any other C style applkication
[17:08:30] <apw> the only special is there are ? entries which generally can be ignored
[17:08:53] <ev> thus my proposal of ext4_get_acl+0x80/0x210:d_splice_alias+0x40/0x50:ext4_check_acl+0x4a/0x90:acl_permission_check+0x97/0xa0:generic_permission+0x25/0xc0:inode_permission+0x99/0xd0 for https://launchpadlibrarian.net/79771442/OopsText.txt
[17:08:56] <ev> right
[17:10:56] <apw> you might also want to include the type of the blammo
[17:11:06] <apw> so thats a 'kernel paging request'
[17:11:41] <ev> apw: okay, will do. And you're happy with the rest of that as a signature?
[17:11:51] <ev> with the type included, that is
[17:13:00] <apw> ev, yeah i think until we try it we're not going to see what triggers overly commonisation or over differntiation
[17:13:08] <ev> fair point
[17:13:09] <apw> ev, when you do other C things do you include the +xx parts ?
[17:14:30] <ev> apw: until we have a fully retraced stacktrace, yes: http://people.canonical.com/~ubuntu-archive/apport-duplicates/address/_usr_bin_Thunar_6
[17:22:21] <ev> apw: out of curiosity, why don't we want to capture VmCore files?
[17:22:49] <apw> its not a simple thing to do, and not enabled by default
[17:23:12] <apw> as your machine has to crash and reboot to do it, we normally try and hang one and keep going, for least supprise
[17:23:52] <ev> ah, understood
[17:24:22] <apw> and the chances of the vmcore not having something sensitive in it is very small
[17:28:13] <ev> apw: thanks for all the advice
[17:28:29] <apw> ev thanks for looking at it