linux-user: doesn't handle guest setting its memory ulimit very small

[aaron]

using the latest build from git (hash 041ccc922ee474693a2869d4e3b59e920c739bc0 ) and all older versions i have tested.
i am using an amd64 host with an arm chroot using "qemu-user arm cortex-a8" cpu emulation to run it

building coreutils hangs on "checking whether printf survives out-of-memory conditions"

i have not had time to dig into the build system to isolate the test yet, there were old reports of this bug but i can no longer find them on google.

[Peter Maydell (pmaydell)]

On 28 February 2015 at 09:01, aaron <email address hidden> wrote:
> Public bug reported:
>
> using the latest build from git (hash 041ccc922ee474693a2869d4e3b59e920c739bc0 ) and all older versions i have tested.
> i am using an amd64 host with an arm chroot using "qemu-user arm cortex-a8" cpu emulation to run it
>
> building coreutils hangs on "checking whether printf survives out-of-
> memory conditions"
>
> i have not had time to dig into the build system to isolate the test
> yet, there were old reports of this bug but i can no longer find them on
> google.

Yes, I seem to recall looking at this one before. QEMU's linux-user
code doesn't try to isolate the guest's memory allocations from
its own allocations. So if the guest sets the memory limit to
something very small then the chances are good that this will
result in one of QEMU's internal allocations failing, and then
QEMU will probably exit with an error or possibly crash or hang
(some of our error handling on these allocations is not good).

For this kind of test to work correctly we would need to fake
the memory limit syscalls rather than just passing them through
to the host, and then also do all the accounting to track how
much memory the guest has allocated. That's a fair amount of
work so it's unlikely this bug will be fixed unless somebody
who cares about it submits patches, I'm afraid.

-- PMM

[Peter Maydell (pmaydell)]

I've just noticed that this is a duplicate of bug 1163034 (gnutls uses a very similar configure test), so I'm going to mark this one as a dup of that one, since it's older.