qemu-user-arm (4.0.0-rc0) crashes

[asavah]

I'm using qemu-user-arm for crosscompilation needs, usually via a wrapper.
qemu-user-arm (4.0.0-rc0) crashes with SIGILL on at least 2 instructions:

first case (sadly I don't have more data handy, can reproduce at a later time if needed):
(gdb) x/i $pc
=> 0xfffce314: vseleq.f64 d0, d17, d0

second case (llvm-config):
qemu cmdline:
qemu-arm -strace -cpu max -r 5.0.0 -L /home/asavah/kross/build/rpi3/rootfs -E LD_LIBRARY_PATH=/home/asavah/kross/build/rpi3/rootfs/usr/bin:/home/asavah/kross/build/rpi3/rootfs/usr/lib /home/asavah/kross/build/rpi3/rootfs/usr/bin/llvm-config --shared-mode

--- SIGILL {si_signo=SIGILL, si_code=2, si_addr=0xf9f89f80} ---
qemu: uncaught target signal 4 (Illegal instruction) - core dumped

output from gdb(arm) attached to qemu-user-arm
Program received signal SIGILL, Illegal instruction.
0xf9f77f80 in ?? ()
(gdb) bt
#0 0xf9f77f80 in ?? ()
#1 0xfffd796c in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/i $pc
=> 0xf9f77f80: vrintm.f64 d18, d18

The very same binaries when run with qemu-user-arm 3.1.0 (both from ubuntu 19.04 package and self built)
work flawlessly.

This is clearly a regression.
Please fix before releasing 4.0.0.

[Peter Maydell (pmaydell)]

Can you provide binaries that reproduce this, please ? Attaching them to the bug is fine.

[asavah (irherder)]

As requested I tarred the failing binaries.

The first one (first case in the original report) is g-ir-compiler from gobjact-introspection, this one has another interesting detail:
if compiled with -mcpu=cortex-a53 -mfpu=neon-fp-armv8 (the correct flags for raspberry pi 3) it crashes on 4.0.0-rc, but works fine even on 4.0.0 if compiled with -mcpu=cortex-a53 -mfpu=neon-vfpv4 , on 3.1.0 it runs fine with -mfpu=neon-fp-armv8 .

The second one (second case in original report), I'm not sure if its the llvm-config binary itself,
or the code from libLLVM so I attached the whole thing.
It only crashes if called with llvm-config --shared-mode , llvm-config --version and llvm-config --ldflags works fine.
-mfpu=neon-vfpv4 does not help here, crashes anyway, but has worked fine with 3.1.0

[Alex Bennée (ajbennee)]

asavah <email address hidden> writes:

> Public bug reported:
>
> I'm using qemu-user-arm for crosscompilation needs, usually via a wrapper.
> qemu-user-arm (4.0.0-rc0) crashes with SIGILL on at least 2 instructions:
>
> first case (sadly I don't have more data handy, can reproduce at a later time if needed):
> (gdb) x/i $pc
> => 0xfffce314: vseleq.f64 d0, d17, d0
>
> second case (llvm-config):
> qemu cmdline:
> qemu-arm -strace -cpu max -r 5.0.0 -L
> /home/asavah/kross/build/rpi3/rootfs -E
> LD_LIBRARY_PATH=/home/asavah/kross/build/rpi3/rootfs/usr/bin:/home/asavah/kross/build/rpi3/rootfs/usr/lib
> /home/asavah/kross/build/rpi3/rootfs/usr/bin/llvm-config --shared-mode

I should point out that a rpi3 is a cortex-a53 so -cpu cortex-a53 should
be all you need to run the binaries. -cpu max will enabled a bunch of
features you cannot use on an actual pi.

>
> --- SIGILL {si_signo=SIGILL, si_code=2, si_addr=0xf9f89f80} ---
> qemu: uncaught target signal 4 (Illegal instruction) - core dumped
>
> output from gdb(arm) attached to qemu-user-arm
> Program received signal SIGILL, Illegal instruction.
> 0xf9f77f80 in ?? ()
> (gdb) bt
> #0 0xf9f77f80 in ?? ()
> #1 0xfffd796c in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> (gdb) x/i $pc
> => 0xf9f77f80: vrintm.f64 d18, d18
>
>
> The very same binaries when run with qemu-user-arm 3.1.0 (both from ubuntu 19.04 package and self built)
> work flawlessly.
>
> This is clearly a regression.
> Please fix before releasing 4.0.0.
>
> ** Affects: qemu
> Importance: Undecided
> Status: New

--
Alex Bennée

[asavah (irherder)]

I should point that -cpu cortex-a53 is not available in qemu-arm,
I'm building arm 32 bit stuff.

qemu-arm -cpu help
Available CPUs:
  arm1026
  arm1136
  arm1136-r2
  arm1176
  arm11mpcore
  arm926
  arm946
  cortex-a15
  cortex-a7
  cortex-a8
  cortex-a9
  cortex-m0
  cortex-m3
  cortex-m33
  cortex-m4
  cortex-r5
  cortex-r5f
  max
  pxa250
  pxa255
  pxa260
  pxa261
  pxa262
  pxa270-a0
  pxa270-a1
  pxa270
  pxa270-b0
  pxa270-b1
  pxa270-c0
  pxa270-c5
  sa1100
  sa1110
  ti925t
  any

[Peter Maydell (pmaydell)]

Yeah, unfortunately we don't support cortex-a53 (or other 64-bit CPUs) in qemu-arm. (We also don't support them as highest-EL-is-AArch32 config in system mode.) Ideally we should fill in that gap, but in practice most people aren't building aarch32 code for ARMv8 -- either they want the back-compat with v7 CPUs or they are using 64-bit -- so it hasn't been very high priority for us.

[Alex Bennée (ajbennee)]

Peter Maydell <email address hidden> writes:

> Yeah, unfortunately we don't support cortex-a53 (or other 64-bit CPUs)
> in qemu-arm. (We also don't support them as highest-EL-is-AArch32 config
> in system mode.) Ideally we should fill in that gap, but in practice
> most people aren't building aarch32 code for ARMv8 -- either they want
> the back-compat with v7 CPUs or they are using 64-bit -- so it hasn't
> been very high priority for us.

I was going to suggest -cpu cortex-a53,aarch64=off but that seems to be
only for KVM guests.

Is there actually a v8 A profile 32 bit only CPU part?

--
Alex Bennée

[Peter Maydell (pmaydell)]

On Mon, 25 Mar 2019 at 13:29, Alex Bennée <email address hidden> wrote:
> I was going to suggest -cpu cortex-a53,aarch64=off but that seems to be
> only for KVM guests.
>
> Is there actually a v8 A profile 32 bit only CPU part?

You can for instance connect up a Cortex-A53 with the
AA64nAA32 config signals hardwired to 0 ("go into AArch32
on power-on-reset"), which is functionally the same thing.

thanks
-- PMM

[Alex Bennée (ajbennee)]

Peter Maydell <email address hidden> writes:

> On Mon, 25 Mar 2019 at 13:29, Alex Bennée <email address hidden> wrote:
>> I was going to suggest -cpu cortex-a53,aarch64=off but that seems to be
>> only for KVM guests.
>>
>> Is there actually a v8 A profile 32 bit only CPU part?
>
> You can for instance connect up a Cortex-A53 with the
> AA64nAA32 config signals hardwired to 0 ("go into AArch32
> on power-on-reset"), which is functionally the same thing.

So would it be reasonable to have a

#ifndef TARGET_AARCH64
    { .name = "cortex-a53 (32bit)", .initfn = aarch32_a53_initfn },
#endif

and the appropriate init function in cpu.c? That should be all we need right?

>
> thanks
> -- PMM

--
Alex Bennée

[Peter Maydell (pmaydell)]

On Mon, 25 Mar 2019 at 15:25, Alex Bennée <email address hidden> wrote:
> So would it be reasonable to have a
>
> #ifndef TARGET_AARCH64
> { .name = "cortex-a53 (32bit)", .initfn = aarch32_a53_initfn },
> #endif
>
> and the appropriate init function in cpu.c? That should be all we need right?

As RTH says, for 4.0 the fix for this regression is simpler.
For the longer term I think we should look a little more broadly
at what would be needed for supporting system emulation mode
CPUs which are nominally aarch64 but configured to boot into
aarch32 on reset, to see whether the linux-user mode falls
out as a subset of that. (It may be that it does not, but
it would definitely be better if we can avoid having to
duplicate the ID register and feature settings for the
64-bit CPUs in both cpu64.c and cpu.c.)

thanks
-- PMM

[Peter Maydell (pmaydell)]

I think this bug should be fixed by commit c8877d0f2f662bf01346a (which has just landed in git master and should be in rc1 when we tag it, probably later today). Please let us know if it hasn't fixed all the regressions for you.

[asavah (irherder)]

qemu-user-arm 4.0.0-rc1 no longer produces any crashes for me.
Huge thanks.