make-target-2.sh: Failure under aarch64 qemu-user-static

[Antonis Kanouras]

Hello,

I'm trying to package SBCL 2.3.7 for OpenWrt 22.03 (with Glibc 2.34), on a Ubuntu 22.04 amd64 host. More specifically, I'm compiling in order:

1. ECL for x86-64
2. SBCL for x86-64 using ECL as XC host
3. SBCL for aarch64 using the x86-64 SBCL as XC host

For the 3rd step, aarch64-openwrt-linux-gcc 11.2.0 and aarch64 qemu-user-static (through binfmt) are used.

Versions of SBCL from 2.0.5 to 2.1.9 end up in an infinite loop.
Versions 2.1.10+ of SBCL end up in LDB, producing varying backtraces per version.

I've tried versions of qemu-user-static from 3.1 up to 8.1.0 (available Debian packages) - the earlier ones trigger an infinite loop, while later ones end up in LDB.

Various permutations of CFLAGS and/or SBCL feature flags, including immobile space and/or ASLR, usually result in the build failing earlier. Trying to do a cold compile also fails in the same way.

On a side note, running make-target-2.sh on the target device (Mikrotik RB5009) finishes succesfully, producing a working sbcl.

I would really appreciate your input on this, as I am not able to discern whether this is a QEMU bug, or whether SBCL tries to do something that hardware aarch64 implementations don't mind.

If you think it is a bug in SBCL, I can upload a OpenWrt buildroot and build instructions for it; although it'll take a few hours of compiling all OpenWrt packages before reaching the point of building SBCL. Alternatively, I can also arrange for SSH access to a machine with the build environment.

I'm attaching a build log and LDB backtrace for SBCL 2.3.7, as well as a small patch to the build system.

Thank you very much for your work!

[Douglas Katzman (dougk)]

How did it decide to allocate 10 GiB ?
"Heap exhausted during allocation: 1023672320 bytes available, 100940122304 requested."
My experience with QEMU user mode emulation is that it's very bad at running SBCL, maybe this is one such manifestation

[Antonis Kanouras (antonis+ubuntuone)]

Thank you for the fast response!

Re QEMU, I was afraid so. :-(

The number 100940122304 (actually a bit more than 94 GiB!) is constant among many of these crashes, even over different SBCL versions.

The message "no size function for object at..." varies between SBCL and/or QEMU versions, however.

I also just noticed that the memory address of the object in question is in unmapped space.

I will keep poking at this, hopefully I'll eventually come up with a test case that's useful for QEMU devs.