System became frozen

[Black Raven]

Seems that there's a bug in kernel related to working with memory, especially to decide when to run oom-killer.

One morning I found my system frozen with he following in logs after reboot:

May 12 07:51:41 big -- MARK --
May 12 08:00:37 big kernel: [233387.108621] type=1503 audit(1242100837.141:10): operation="capable" name="sys_admin" pid=11357 profile="/usr/sbin/cupsd"
May 12 08:00:37 big kernel: [233387.108630] type=1503 audit(1242100837.141:11): operation="capable" name="sys_resource" pid=11357 profile="/usr/sbin/cupsd"
May 12 08:00:37 big kernel: [233387.108636] type=1503 audit(1242100837.141:12): operation="capable" name="sys_rawio" pid=11357 profile="/usr/sbin/cupsd"
May 12 08:00:37 big kernel: [233387.108652] type=1503 audit(1242100837.141:13): operation="capable" name="sys_admin" pid=11357 profile="/sbin/dhclient3"
May 12 08:00:37 big kernel: [233387.108657] type=1503 audit(1242100837.141:14): operation="capable" name="sys_resource" pid=11357 profile="/sbin/dhclient3"
May 12 08:00:37 big kernel: [233387.108663] type=1503 audit(1242100837.141:15): operation="capable" name="sys_rawio" pid=11357 profile="/sbin/dhclient3"
May 12 08:00:37 big kernel: [233387.108843] rhythmbox invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
May 12 08:00:37 big kernel: [233387.108848] Pid: 11357, comm: rhythmbox Tainted: P 2.6.28-12-generic #43-Ubuntu
May 12 08:00:37 big kernel: [233387.108851] Call Trace:
May 12 08:00:37 big kernel: [233387.108862] [<c04fca36>] ? printk+0x18/0x1a
May 12 08:00:37 big kernel: [233387.108868] [<c0191525>] oom_kill_process+0x75/0x210
May 12 08:00:37 big kernel: [233387.108872] [<c0191c03>] ? select_bad_process+0xc3/0x100
May 12 08:00:37 big kernel: [233387.108877] [<c0191ce2>] out_of_memory+0xa2/0x140
May 12 08:00:37 big kernel: [233387.108881] [<c0194350>] __alloc_pages_internal+0x450/0x490
May 12 08:00:37 big kernel: [233387.108886] [<c0196c52>] __do_page_cache_readahead+0xd2/0x1d0
May 12 08:00:37 big kernel: [233387.108890] [<c0196d9b>] do_page_cache_readahead+0x4b/0x70
May 12 08:00:37 big kernel: [233387.108894] [<c018fd78>] filemap_fault+0x318/0x3e0
May 12 08:00:37 big kernel: [233387.108899] [<c01a0f1b>] __do_fault+0x3b/0x470
May 12 08:00:37 big kernel: [233387.108904] [<c04272fd>] ? skb_dequeue+0x4d/0x70
May 12 08:00:37 big kernel: [233387.108909] [<c01a1e39>] handle_mm_fault+0x119/0x380
May 12 08:00:37 big kernel: [233387.108915] [<c012ad96>] ? dequeue_entity+0x16/0x2a0
May 12 08:00:37 big kernel: [233387.108919] [<c050118d>] do_page_fault+0x1fd/0x790
May 12 08:00:37 big kernel: [233387.108924] [<c0102ab7>] ? __switch_to+0xb7/0x1a0
May 12 08:00:37 big kernel: [233387.108929] [<c0122d58>] ? default_spin_lock_flags+0x8/0x10
May 12 08:00:37 big kernel: [233387.108934] [<c04feb7e>] ? _spin_lock_irqsave+0x2e/0x40
May 12 08:00:37 big kernel: [233387.108939] [<c0152a0a>] ? hrtimer_try_to_cancel+0x3a/0x90
May 12 08:00:37 big kernel: [233387.108943] [<c0152a71>] ? hrtimer_cancel+0x11/0x20
May 12 08:00:37 big kernel: [233387.108947] [<c04fded9>] ? do_nanosleep+0x99/0xc0
May 12 08:00:37 big kernel: [233387.108951] [<c0153083>] ? hrtimer_nanosleep+0xf3/0x180
May 12 08:00:37 big kernel: [233387.108955] [<c0152440>] ? hrtimer_wakeup+0x0/0x20
May 12 08:00:37 big kernel: [233387.108959] [<c04fdebf>] ? do_nanosleep+0x7f/0xc0
May 12 08:00:37 big kernel: [233387.108963] [<c0153170>] ? sys_nanosleep+0x60/0x70
May 12 08:00:37 big kernel: [233387.108966] [<c0500f90>] ? do_page_fault+0x0/0x790
May 12 08:00:37 big kernel: [233387.108970] [<c04feff2>] error_code+0x72/0x80
May 12 08:00:37 big kernel: [233387.108973] Mem-Info:
May 12 08:00:37 big kernel: [233387.108975] DMA per-cpu:
May 12 08:00:37 big kernel: [233387.108978] CPU 0: hi: 0, btch: 1 usd: 0
May 12 08:00:37 big kernel: [233387.108981] CPU 1: hi: 0, btch: 1 usd: 0
May 12 08:00:37 big kernel: [233387.108983] Normal per-cpu:
May 12 08:00:37 big kernel: [233387.108985] CPU 0: hi: 186, btch: 31 usd: 64
May 12 08:00:37 big kernel: [233387.108988] CPU 1: hi: 186, btch: 31 usd: 111
May 12 08:00:37 big kernel: [233387.108990] HighMem per-cpu:
May 12 08:00:37 big kernel: [233387.108992] CPU 0: hi: 186, btch: 31 usd: 55
May 12 08:00:37 big kernel: [233387.108995] CPU 1: hi: 186, btch: 31 usd: 61
May 12 08:00:37 big kernel: [233387.109000] Active_anon:308247 active_file:643 inactive_anon:164139
May 12 08:00:37 big kernel: [233387.109001] inactive_file:3442 unevictable:2 dirty:4 writeback:0 unstable:0
May 12 08:00:37 big kernel: [233387.109002] free:12464 slab:8574 mapped:6798 pagetables:1932 bounce:0
May 12 08:00:37 big kernel: [233387.109007] DMA free:8124kB min:64kB low:80kB high:96kB active_anon:256kB inactive_anon:324kB active_file:0kB inactive_file:4kB unevictable:0kB present:15804kB pages_scanned:916 all_unreclaimable? yes
May 12 08:00:37 big kernel: [233387.109011] lowmem_reserve[]: 0 861 2015 2015
May 12 08:00:37 big kernel: [233387.109020] Normal free:41256kB min:3720kB low:4648kB high:5580kB active_anon:368492kB inactive_anon:367992kB active_file:440kB inactive_file:1248kB unevictable:0kB present:881880kB pages_scanned:2177312 a
ll_unreclaimable? yes
May 12 08:00:37 big kernel: [233387.109023] lowmem_reserve[]: 0 0 9235 9235
May 12 08:00:37 big kernel: [233387.109032] HighMem free:476kB min:512kB low:1756kB high:3004kB active_anon:864240kB inactive_anon:288240kB active_file:2132kB inactive_file:12516kB unevictable:8kB present:1182120kB pages_scanned:1969056
all_unreclaimable? yes
May 12 08:00:37 big kernel: [233387.109035] lowmem_reserve[]: 0 0 0 0
May 12 08:00:37 big kernel: [233387.109041] DMA: 5*4kB 5*8kB 6*16kB 5*32kB 6*64kB 6*128kB 6*256kB 6*512kB 2*1024kB 0*2048kB 0*4096kB = 8124kB
May 12 08:00:37 big kernel: [233387.109054] Normal: 1096*4kB 565*8kB 1692*16kB 45*32kB 8*64kB 4*128kB 3*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 41256kB
May 12 08:00:37 big kernel: [233387.109066] HighMem: 15*4kB 8*8kB 0*16kB 1*32kB 3*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 476kB
May 12 08:00:37 big kernel: [233387.109079] 4288 total pagecache pages
May 12 08:00:37 big kernel: [233387.109081] 1 pages in swap cache
May 12 08:00:37 big kernel: [233387.109084] Swap cache stats: add 1077658, delete 1077657, find 268689/329272
May 12 08:00:37 big kernel: [233387.109087] Free swap = 0kB
May 12 08:00:37 big kernel: [233387.109089] Total swap = 2000052kB
May 12 08:00:37 big kernel: [233387.114904] 524144 pages RAM
May 12 08:00:37 big kernel: [233387.114907] 297858 pages HighMem
May 12 08:00:37 big kernel: [233387.114909] 8794 pages reserved
May 12 08:00:37 big kernel: [233387.114911] 14931 pages shared
May 12 08:00:37 big kernel: [233387.114913] 494101 pages non-shared
May 12 10:25:21 big syslogd 1.5.0#5ubuntu3: restart.

Before submitting this bug, I searched the net about it and found this problem is common:
http://ubuntuforums.org/showthread.php?p=7289617#post7289617
https://bugs.launchpad.net/ubuntu/+source/kdepim/+bug/165143
https://bugzilla.redhat.com/show_bug.cgi?id=496680
http://www.centos.org/modules/newbb/viewtopic.php?topic_id=11185

I have 2 GiGs physical memory and 2 GiGs swap space, and I just don't believe that all this memry got used so that oom-killer should be invoked.

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
HibernationDevice: RESUME=UUID=4ff2407c-9fa8-4a06-89ed-9733c3da47a7
MachineType: System manufacturer System Product Name
NonfreeKernelModules: nvidia
Package: linux-image-2.6.28-12-generic 2.6.28-12.43
ProcCmdLine: root=UUID=bfe88395-966f-402a-9a37-8dc275c916e5 ro quiet splash vga=791
ProcEnviron:
 PATH=(custom, user)
 LANG=ru_RU.UTF-8
 SHELL=/bin/zsh
ProcVersionSignature: Ubuntu 2.6.28-12.43-generic
SourcePackage: linux

[Jeremy Foshee (jeremyfoshee)]

Hi Black,

This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/ .

If it remains an issue, could you run the following command from a Terminal (Applications->Accessories->Terminal). It will automatically gather and attach updated debug information to this report.

apport-collect -p linux 377267

Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Black Raven (dpisklov)]

With kernel 2.6.32 I haven't experienced anything like this. However currently I'm using Arch Linux and have no chance to test Ubuntu's package.
When I was using Kubuntu I also haven't such problem with 2.6.31 kernel, but had another one:
https://bugs.kde.org/show_bug.cgi?id=217253 (Upstream bug: http://oss.sgi.com/bugzilla/show_bug.cgi?id=852)
It can be related to this one.

[Brad Figg (brad-figg)]

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in a supported series, please file a new bug.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.