Kernel memory disable overcommitment - java and other things dysfunctional

[a-r-k-i-b-o-t-t]

uname -a
Linux Laptop 2.6.24-19-generic #1 SMP Wed Aug 20 22:56:21 UTC 2008 i686 GNU/Linux

java -version
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Server VM (build 10.0-b22, mixed mode)

sysctl vm.overcommit_memory=2
(if you're unlucky fork will fail right from there on! so don't try this when you got importand files open)

then start java (or anything else..)

java
Error occurred during initialization of VM
Could not reserve enough space for object heap
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb7dee4da, pid=7477, tid=3084225424
#
# Java VM: Java HotSpot(TM) Server VM (10.0-b22 mixed mode linux-x86)
# Problematic frame:
# C [libc.so.6+0x6f4da] cfree+0x7a
#
# An error report file with more information is saved as:
# /home/ubuntu/hs_err_pid7477.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
Aborted

This is just a java example. Other applications segfault. The problem itself is the kernel memory overcommitment and the resulting oom. To disable it or to trying out a workaround.. just makes things worse. Maybe the vm.overcommit_memory was a bad advice i read somewhere.

There is a lot of memory free. So this error is just.. i see no reason for this. Is the kernel memory management so dependent on the bad behavior of overcommiting memory?

To return to the default:

sysctl vm.overcommit_memory=0

and java works again.

Further reading to show that this seems to be a well known problem for years - still not fixed. There they just ask for the 'kernel version' but it's there since all those versions.
http://www.redhat.com/archives/rhl-devel-list/2005-February/msg00671.html

I got enabled swap. I got enough RAM. So all those factors are allright. Still the kernel faults with crappy memory management from time to time.

Some more information rich reading about kernel memory:
http://www.win.tue.nl/~aeb/linux/lk/lk-9.html

Another thought, why this should be a concern for the maintainers. This results in random - not by the application caused - crashes. I guess some of the error reports are the result of the kernel when it gets to memory limits in one library or in any application. I have seen many errors that only seem to happen, if the memory gets 'closer' to the limits. An ubuntu with only one application running is more likely to work than a ubuntu with lots of stuff running.

[Craig Ringer (ringerc)]

Still present on 12.04 LTS with 3.0.0-14-generic-pae i686.

Memory fragmentation / contiguous allocation issues? I can't imagine why Java would demand contiguous memory, but it's a bit weird sometimes.

craig@wallace:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04 LTS
Release: 12.04
Codename: precise
craig@wallace:~$ uname -a
Linux wallace 3.0.0-14-generic-pae #23-Ubuntu SMP Mon Nov 21 22:07:10 UTC 2011 i686 i686 i386 GNU/Linux
craig@wallace:~$ free -m
             total used free shared buffers cached
Mem: 3902 3669 232 0 129 2583
-/+ buffers/cache: 957 2945
Swap: 1553 5 1547
craig@wallace:~$ cat /proc/meminfo
MemTotal: 3996268 kB
MemFree: 212484 kB
Buffers: 132820 kB
Cached: 2669820 kB
SwapCached: 2468 kB
Active: 1514732 kB
Inactive: 2094660 kB
Active(anon): 651976 kB
Inactive(anon): 167824 kB
Active(file): 862756 kB
Inactive(file): 1926836 kB
Unevictable: 80 kB
Mlocked: 80 kB
HighTotal: 3213600 kB
HighFree: 40688 kB
LowTotal: 782668 kB
LowFree: 171796 kB
SwapTotal: 1590420 kB
SwapFree: 1584292 kB
Dirty: 2584 kB
Writeback: 9920 kB
AnonPages: 805564 kB
Mapped: 148452 kB
Shmem: 12992 kB
Slab: 81328 kB
SReclaimable: 58140 kB
SUnreclaim: 23188 kB
KernelStack: 4320 kB
PageTables: 14600 kB
NFS_Unstable: 11508 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3588552 kB
Committed_AS: 3199664 kB
VmallocTotal: 122880 kB
VmallocUsed: 36856 kB
VmallocChunk: 81828 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 47096 kB
DirectMap2M: 866304 kB