apport patches for CORE_REAL_RLIM and limit overriding do not work any more

[Martin Pitt]

Binary package hint: linux-source-2.6.22

I cannot get apport to work on 2.6.22-6.13. There is no data passed, the core dumps are 0 empty by default, and CORE_REAL_RLIM is always -1.

Let's create a crash hook and enable it:

$ echo -e '#!/bin/sh\nenv >/tmp/env\ncat >/tmp/crash' > /tmp/hook
$ chmod 755 /tmp/hook
$ echo '|/tmp/hook' | sudo tee /proc/sys/kernel/core_pattern

For the default ulimit 0:

$ ulimit -c
0
$ bash -c 'kill -SEGV $$'
Segmentation fault (core dumped)
$ ls -l /tmp/env /tmp/crash
-rw-r--r-- 1 root root 0 2007-06-08 08:25 /tmp/crash
-rw-r--r-- 1 root root 145 2007-06-08 08:25 /tmp/env
$ grep RLIM /tmp/env
CORE_REAL_RLIM=-1

In this case, CORE_REAL_RLIM should be 0 (since -1 means 'unlimited') and /tmp/crash should have the entire core dump (but is empty).

For unlimited it works fine:

$ ulimit -S -c unlimited
$ bash -c 'kill -SEGV $$'
Segmentation fault (core dumped)
$ ls -l /tmp/env /tmp/crash
-rw-r--r-- 1 root root 905216 2007-06-08 08:28 /tmp/crash
-rw-r--r-- 1 root root 145 2007-06-08 08:28 /tmp/env
$ grep RLIM /tmp/env
CORE_REAL_RLIM=-1

For a large limit, the core dump works, but CORE_REAL_RLIM is not correctly set:

$ ulimit -S -c 5000
$ bash -c 'kill -SEGV $$'
Segmentation fault (core dumped)
$ ls -l /tmp/env /tmp/crash
-rw-r--r-- 1 root root 909312 2007-06-08 08:30 /tmp/crash
-rw-r--r-- 1 root root 145 2007-06-08 08:30 /tmp/env
$ grep RLIM /tmp/env
CORE_REAL_RLIM=-1

For a small ulimit, the core dump s correctly capped, but CORE_REAL_RLIM is still -1:

$ ulimit -S -c 50
$ bash -c 'kill -SEGV $$'
Segmentation fault (core dumped)
$ ls -l /tmp/env /tmp/crash
-rw-r--r-- 1 root root 53248 2007-06-08 08:31 /tmp/crash
-rw-r--r-- 1 root root 145 2007-06-08 08:31 /tmp/env
$ grep RLIM /tmp/env
CORE_REAL_RLIM=-1

The empty core dump with 0 ulimit breaks apport in the default case, and the wrongly set CORE_REAL_RLIM breaks correct writing of core files by apport.

Thanks in advance!

[Martin Pitt (pitti)]

For comparison, this is how it looks like on 2.6.20:

$ ulimit -c
0
$ bash -c 'kill -SEGV $$'
Segmentation fault (core dumped)
$ ls -l /tmp/env /tmp/crash
-rw-r--r-- 1 root root 909312 2007-06-08 08:45 /tmp/crash
-rw-r--r-- 1 root root 143 2007-06-08 08:45 /tmp/env
$ grep RLIM /tmp/env
CORE_REAL_RLIM=0

So by default the entire core dump is piped and CORE_REAL_RLIM is 0.

For 'ulimit -S -c 50', CORE_REAL_RLIM is 51200. 2.6.20 does not cap the piped core dump to 50 KB as the current 2.6.22 seems to do; this is correct behaviour since we want the entire core dump for apport, we just want to clip the written core files to 50 KB.

[Martin Pitt (pitti)]

BTW, just for the records,

  /usr/share/apport/testsuite/test-apport kernel

checks all this. On Feisty it runs through without errors (/var/crash/ must be empty before). On gutsy it errors out immediately.

[Martin Pitt (pitti)]

I think I found the possible culprit in

  http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-gutsy.git;a=blob;h=f9e8f6f87008cc519073282ed66787c528b65665;hb=36c1aeac3968fa6ea0899e2c619a5c69a4350b52;f=fs/exec.c#l1545

It says "if (corename[0] == '|') {" and overrides the limit if true. However, corename is a local variable in do_coredump() and does not get initialized before line 1567. Thus Line 1545 should be

  if (core_pattern[0] == '|') {

[Ben Collins (ben-collins)]

Cheers

[Martin Pitt (pitti)]

Confirming that 2.6.22-7 fixes the problem, the entire test suite passes now. Thanks!