irrecoverable stack overflow caused by library with executable stack

[Douglas Simon]

Description: Ubuntu 10.04.1 LTS
Release: 10.04

This is a test case to demonstrate that stack overflow detection is broken on OpenJDK6+Linux when a Java application loads a library which denotes it requires an executable stack. The cause of this outlined in more detail here:

http://blogs.sun.com/dns/entry/stacks_with_split_personalities

To reproduce:

export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
$JAVA_HOME/bin/javac StackOverflowCrash.java
gcc -c stackoverflowcrash.c -fPIC -I$JAVA_HOME/include -I$JAVA_HOME/include/linux
gcc -g -shared -lc stackoverflowcrash.o -o libstackoverflowcrash.so
$JAVA_HOME/bin/java -Djava.library.path=. -cp . StackOverflowCrash

[Douglas Simon (dnsimon)]

Native code for test case.

[Douglas Simon (dnsimon)]

Java code for test case.