Comment 50 for bug 1699772

Regarding OpenJDK 8, it crashes as soon as Xss is set to (or higher than) 1141K in a i386 JVM (32-bit).

I used the example code from bug #1700270. Please note that there is no need to even use the java class: the program will segfault while starting the JVM, so do remove lines 30-34 from either test_case1.c or test_case2.c and set Xss to 1441K (or bigger).

The OpenJDK part where the stack location and size are calculated is in os::Linux::capture_initial_stack() [1], specially _initial_thread_stack_bottom [2].

From GDB I was able to collect the following data from that function:
(gdb) p max_size
$1 = 1171456

Note: max_size is Xss rounded to vm_page_size(), thus 1144K [3].

(gdb) info locals
rlim = {rlim_cur = 8388608, rlim_max = 4294967295}
stack_size = 8380416
stack_start = 4294956864
p = 0xf7ffcf34 <__libc_stack_end>
stack_top = 4294959104
low = 0xfffdd000 ""
high = 0xffffe000 <error: Cannot access memory at address 0xffffe000>

(gdb) x p
0xf7ffcf34 <__libc_stack_end>: 0xffffd740
(gdb) x stack_top
0xffffe000: Cannot access memory at address 0xffffe000
(gdb) x low
0xfffdd000: 0x00000000
(gdb) x high
0xffffe000: Cannot access memory at address 0xffffe000
(gdb) p _initial_thread_stack_size
$43 = 1171456
(gdb) x _initial_thread_stack_bottom
0xffee0000: 0x00000000

Backtrace:
(gdb) bt
#0 os::Linux::capture_initial_stack (max_size=1171456) at ./src/hotspot/src/os/linux/vm/os_linux.cpp:1272
#1 0xf7394287 in os::init_2 () at ./src/hotspot/src/os/linux/vm/os_linux.cpp:4939
#2 0xf74ee886 in Threads::create_vm (args=0xffffd62c, canTryAgain=0xffffd5bf) at ./src/hotspot/src/share/vm/runtime/thread.cpp:3361
#3 0xf7151423 in JNI_CreateJavaVM (vm=0xffffd684, penv=0xffffd624, args=0xffffd62c) at ./src/hotspot/src/share/vm/prims/jni.cpp:5220
#4 0x5655561f in create_vm (jvm=0xffffd684) at test_case.c:16
#5 0x56555685 in main (argc=1, argv=0xffffd744) at test_case.c:25

That information is used by os::Linux::default_guard_size() [4] to fetch both 'bottom' and 'size' used to indicate the start of the guard page - and it has a nice doc explaining the stack layout. The values from default_guard_size are in turn used by os::current_stack_base() [5] to calculate what should be the stack base.

Let me know if there's any additional information I can help with.

[1] http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l1081
[2] http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l1271
[3] http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/os/linux/vm/os_linux.cpp#l5010
[4] http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/os_cpu/linux_x86/vm/os_linux_x86.cpp#l714
[5] http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/tip/src/os_cpu/linux_x86/vm/os_linux_x86.cpp#l745