Out of memory error when running min_free_kbytes testcase on precise-updates on armadaxp

Bug #1034197 reported by Michael Reed on 2012-08-07
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Eilt project
Undecided
Unassigned
linux-armadaxp (Ubuntu)
Undecided
Unassigned

Bug Description

Impact: After running the testcase min_free_kbytes, the system becomes unusable

Build Version and Date: ubuntu@ubuntu:~$ uname -a
Linux ubuntu 3.2.0-1605-armadaxp #8-Ubuntu SMP Sat Jul 7 22:36:46 UTC 2012 armv7l armv7l armv7l GNU/Linux

Steps to reproduce:
1. Download, build and Install LTP testsuite
2. echo 0 > /proc/sys/kernel/hung_task_timeout_secs
3. cd /ltp-full-20120614/testcases/kernel/mem/tunable
4. sudo ./min_free_kbytes

This is the output when I ran this from a script, so I could get the oom* info
sudo ./runtest.sh
[sudo] password for ubuntu:
cat /proc/1409/task/1409/oom*
0
1
0
ubuntu@ubuntu:~/ltp-full-20120614/testcases/kernel/mem/tunable$ min_free_kbytes 0 TINFO : set overcommit_memory to 2
min_free_kbytes 0 TINFO : set min_free_kbytes to 3624
memfree is 2970220 kB before eatup mem
memfree is 81228 kB after eatup mem
min_free_kbytes 0 TINFO : set min_free_kbytes to 1513802
[ 1643.106611] Out of memory: Kill process 264 (upstart-udev-br) score 1 or sacrifice child
[ 1643.114754] Killed process 264 (upstart-udev-br) total-vm:1960kB, anon-rss:0kB, file-rss:0kB
[ 1643.267693] Out of memory: Kill process 424 (upstart-socket-) score 1 or sacrifice child
[ 1643.275814] Killed process 424 (upstart-socket-) total-vm:1892kB, anon-rss:0kB, file-rss:0kB
[ 1643.424805] Out of memory: Kill process 460 (dhclient3) score 1 or sacrifice child
[ 1643.432462] Killed process 460 (dhclient3) total-vm:1884kB, anon-rss:0kB, file-rss:0kB
[ 1643.635470] Out of memory: Kill process 657 (dbus-daemon) score 1 or sacrifice child
[ 1643.643286] Killed process 657 (dbus-daemon) total-vm:2324kB, anon-rss:0kB, file-rss:0kB
[ 1644.085534] Out of memory: Kill process 670 (rsyslogd) score 1 or sacrifice child
[ 1644.093079] Killed process 670 (rsyslogd) total-vm:30204kB, anon-rss:112kB, file-rss:140kB

The output that shows the kernel panic is in the following link:
https://pastebin.canonical.com/71710/

A description and the expected output of the testcase can be found here:
http://<email address hidden>/msg13978.html

The system is n2 and is currently available for debugging

Michael Reed (mreed8855) on 2012-08-07
summary: - Out of memory error when running min_free_kbytes test on precise-updates
- case
+ Out of memory error when running min_free_kbytes testcase on precise-
+ updates on armadaxp
description: updated
Michael Reed (mreed8855) wrote :
Download full text (11.3 KiB)

Output when running the testcase from the command line.

sudo ./min_free_k
min_free_kbytes 0 TINFO : set overcommit_memory to 2
min_free_kbytes 0 TINFO : set min_free_kbytes to 3624
memfree is 2976072 kB before eatup mem
memfree is 80376 kB after eatup mem
min_free_kbytes 0 TINFO : set min_free_kbytes to 1513804
[ 386.395436] Out of memory: Kill process 266 (upstart-udev-br) score 1 or sacrifice child
[ 386.403632] Killed process 266 (upstart-udev-br) total-vm:1960kB, anon-rss:0kB, file-rss:0kB
[ 386.555048] Out of memory: Kill process 424 (upstart-socket-) score 1 or sacrifice child
[ 386.563195] Killed process 424 (upstart-socket-) total-vm:1892kB, anon-rss:0kB, file-rss:0kB
[ 386.724271] Out of memory: Kill process 469 (dhclient3) score 1 or sacrifice child
[ 386.731900] Killed process 469 (dhclient3) total-vm:1884kB, anon-rss:0kB, file-rss:0kB
[ 387.914502] Out of memory: Kill process 669 (dbus-daemon) score 1 or sacrifice child
[ 387.922307] Killed process 669 (dbus-daemon) total-vm:2324kB, anon-rss:0kB, file-rss:0kB
[ 388.913183] Out of memory: Kill process 677 (rsyslogd) score 1 or sacrifice child
[ 388.920719] Killed process 677 (rsyslogd) total-vm:30204kB, anon-rss:40kB, file-rss:100kB
[ 388.984618] Out of memory: Kill process 690 (rs:main Q:Reg) score 1 or sacrifice child
[ 388.992595] Killed process 690 (rs:main Q:Reg) total-vm:30204kB, anon-rss:240kB, file-rss:160kB
[ 389.332725] Out of memory: Kill process 729 (getty) score 1 or sacrifice child
[ 389.340002] Killed process 729 (getty) total-vm:3836kB, anon-rss:0kB, file-rss:8kB
[ 389.912061] Out of memory: Kill process 738 (getty) score 1 or sacrifice child
[ 389.919332] Killed process 738 (getty) total-vm:3836kB, anon-rss:0kB, file-rss:8kB
[ 390.862432] Out of memory: Kill process 748 (getty) score 1 or sacrifice child
[ 390.869715] Killed process 748 (getty) total-vm:3836kB, anon-rss:0kB, file-rss:8kB
[ 391.033211] Out of memory: Kill process 751 (getty) score 1 or sacrifice child
[ 391.040482] Killed process 751 (getty) total-vm:3836kB, anon-rss:0kB, file-rss:8kB
[ 391.156255] Out of memory: Kill process 756 (getty) score 1 or sacrifice child
[ 391.163504] Killed process 756 (getty) total-vm:3836kB, anon-rss:0kB, file-rss:8kB
[ 391.276823] Out of memory: Kill process 768 (cron) score 1 or sacrifice child
[ 391.283986] Killed process 768 (cron) total-vm:1836kB, anon-rss:0kB, file-rss:0kB
[ 391.378061] Out of memory: Kill process 769 (atd) score 1 or sacrifice child
[ 391.385146] Killed process 769 (atd) total-vm:1708kB, anon-rss:0kB, file-rss:0kB
[ 391.496240] Out of memory: Kill process 772 (acpid) score 1 or sacrifice child
[ 391.503489] Killed process 772 (acpid) total-vm:1328kB, anon-rss:0kB, file-rss:4kB
[ 392.063055] Out of memory: Kill process 774 (irqbalance) score 1 or sacrifice child
[ 392.070760] Killed process 774 (irqbalance) total-vm:2600kB, anon-rss:52kB, file-rss:172kB
[ 392.132300] Out of memory: Kill process 781 (whoopsie) score 1 or sacrifice child
[ 392.139831] Killed process 781 (whoopsie) total-vm:20500kB, anon-rss:252kB, file-rss:680kB
[ 392.901843] Out of memory: Kill process 785 (getty) score 1 or sac...

Changed in eilt:
importance: High → Undecided
description: updated
mahmoh (mahmoh) on 2012-08-09
no longer affects: linux-meta (Ubuntu)
Ike Panhc (ikepanhc) on 2012-08-09
tags: added: ike-radar
mahmoh (mahmoh) on 2012-08-09
visibility: private → public
Jani Monoses (jani) wrote :

This occurs in the released version as well not only in updates.
Tested with 3.2.0-1602.5

Michael Reed (mreed8855) wrote :

This error also occurs on highbank on precise, this is probably a kernel bug, and not architecture specific. I will test it on x86 to verify.

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 3.5.0-9-highbank #9-Ubuntu SMP PREEMPT Thu Aug 9 17:12:46 UTC 2012 armv7l armv7l armv7l GNU/Linux

Michael Reed (mreed8855) wrote :

This error is also seen on an x86 system on precise.

       [ 6239.736670] Out of memory: Kill process 867 (xinetd) score 1 or sacrifice chi
       ld
       [ 6239.736746] Killed process 867 (xinetd) total-vm:2544kB, anon-rss:156kB, file
       -rss:768kB
       [ 6239.740701] Out of memory: Kill process 868 (atd) score 1 or sacrifice child
       [ 6239.740770] Killed process 868 (atd) total-vm:2452kB, anon-rss:128kB, file-rs
       s:224kB
       [ 6239.744692] Out of memory: Kill process 869 (cron) score 1 or sacrifice child
       [ 6239.744760] Killed process 869 (cron) total-vm:2600kB, anon-rss:148kB, file-r
       ss:600kB
       [ 6239.748720] Out of memory: Kill process 871 (acpid) score 1 or sacrifice chil
       d
       [ 6239.748794] Killed process 871 (acpid) total-vm:2156kB, anon-rss:96kB, file-r
       ss:500kB
       [ 6239.752530] Out of memory: Kill process 883 (whoopsie) score 1 or sacrifice c
       hild
       [ 6239.752606] Killed process 883 (whoopsie) total-vm:24448kB, anon-rss:840kB, f
       ile-rss:2716kB
       [ 6360.804206] INFO: task init:1 blocked for more than 120 seconds.
       [ 6360.804272] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
       message.
       [ 6360.805565] INFO: task whoopsie:883 blocked for more than 120 seconds.
       [ 6360.805611] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
       message.

Gavin Guo (mimi0213kimo) on 2013-02-18
Changed in linux-armadaxp (Ubuntu):
assignee: nobody → Gavin Guo (mimi0213kimo)
Gavin Guo (mimi0213kimo) wrote :

Having tested on n1 and the kernel oops not happened. Don't know if the problem is the hardware issue and will try to test on n2 if it's available.

Michael Reed (mreed8855) wrote :

This bug is actually not architecture specific, it happens on x86 and x86_64. It should probably move over to the kernel team

Changed in eilt:
status: New → Invalid
status: Invalid → New
status: New → Incomplete
status: Incomplete → Opinion
Gavin Guo (mimi0213kimo) on 2013-07-31
Changed in linux-armadaxp (Ubuntu):
assignee: Gavin Guo (mimi0213kimo) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers