openbsd: x86_64 the data limit is 1G actually

[Sebastien Marie]

The data limit of 512M was changed in 2016. It is currently 1G.

Update the default dynamic-space-size value from 444Mb to 767Mb.

I am seeing regular (but not systematic) heap exhaustion error with current value while compiling sbcl.

[Sebastien Marie (semarie)]

I would like also note that it is possible to pre-allocate memory not accounted in the datasize limit, if it is allocated with PROT_NONE protection.

It would be accounted only when used (and the protection changed with mprotect(2)).

But I don't know enough sbcl internals to know if it is a possible way to manage the memory or not.

[Douglas Katzman (dougk)]

on Linux we pass MAP_NORESERVE to avoid committing swap space until just-in-time. Does openBSD have that and would it help? Passing PROT_NONE is a departure from how #+unix works, though it's slightly similar to how #+win32 works. On #+win32 there is a similar idea by not passing MEM_COMMIT, and then we can commit individual pages on demand.

[Sebastien Marie (semarie)]

we don't have MAP_NORESERVE (well, it exists but only for userland compat: it is a nop when used).

we change PROT_NONE to not count against resource limits in 2020 to cope with at least Chromium code base which allocate large chunck of memory for javascript and only populates it sparsely.

I will take a look at win32 code.

For now, the attached patch to raise dynamic-space-size is a good compromise.

[Douglas Katzman (dougk)]

applied in https://sourceforge.net/p/sbcl/sbcl/ci/6e715406551d630c13965449805f8656375b5a20/

Shall we consider it fixed?

[Sebastien Marie (semarie)]

yes it should be considered fixed. if I advance for using PROT_NONE, I will open a different report.

Thanks.