Qemu ARMv7 TCG MultiThreading for i386 guest doesn't work

[TimeMaster]

Using any Linux image (in this case Alpine Linux iso) I want to utilise all cores of my Raspberry with --accel,thread=multi. I know there is a probably still problem with memory ordering of the host, but I have also seen some very old commits which could potentially help with it.

But anyway, with version qemu-i386 version 3.1.0 (Debian 1:3.1+dfsg-7)
I can see OpenRC starting up services and then the kernel crash.

With version QEMU emulator version 3.1.93 (v4.0.0-rc3-dirty)
The whole machine crash with this error:
Illegal instruction

Using command:
./qemu-system-i386 -cdrom alpine.iso --accel tcg,thread=multi

Full Console Output:
qemu-system-i386: warning: Guest expects a stronger memory ordering than the host provides
This may cause strange/hard to debug errors
Illegal instruction

Kernel:
Linux raspberrypi 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linux

CPU:
ARMv7 Processor rev 5 (v7l)
Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm
4 cores

[Peter Maydell (pmaydell)]

This is expected. The warning from QEMU is telling you that MTTCG is not supported for the combination of an Arm host and an x86 guest. It warns that you might see "strange errors", and indeed you saw an error. Don't try to use MTTCG here.

I think this warning should probably just be an error -- QEMU shouldn't allow the user to select a feature which we know is not going to work reliably.

[TimeMaster (timemaster5)]

Thank you for the explanation. I found some patches a few years old, so I thought the implementation is now complete especially when ARM is getting more and more popular.

Does it have some priority assigned? Or does it even make sense to do it? From my perspective, yes even with the broken support the whole boot process was blazingly fast on Raspberry and seemed that kernel was able to start and then crashed. Even this was so fast compared to one core. It's interesting there are no more people wanting this. Maybe I am missing another trend :)

[Peter Maydell (pmaydell)]

Implementing a stronger guest memory model than the host has is tricky -- we would need to emit barrier instructions after each guest load or store, which would slow down straight-line execution speed by quite a lot, perhaps by so much that it would outweigh the gain from being able to run more than one thread at once. (Perhaps some of the armv8 load-acquire/store-release insns would help, but they're no good to you on a v7 CPU.)

There aren't many people overall who want to try to run emulation on anything other than x86 host.

[TimeMaster (timemaster5)]

Yes, I can imagine that.

Unfortunately, my programming knowledge isn't so strong to help you with it. But can help with testing if it is useful.

Alex Bennée pointed me to this patch:
[RFC,v3,4/5] mttcg: Implement implicit ordering semantics

Which maybe could help. I can try to compile it and do the requested tests.

[Peter Maydell (pmaydell)]

That patch is already in QEMU (commit b32dc3370a666e23). It sounds like we're a bit further forward with this than I thought we might be, but clearly not everything necessary is implemented.

[TimeMaster (timemaster5)]

Interesting, maybe that is the reason why it is not restricted to use mttcg from the command line.

So I will be waiting here if you need :)

[Richard Henderson (rth)]

When you say

> ./qemu-system-i386 -cdrom alpine.iso --accel tcg,thread=multi

you have not created multiple cpus for the guest, so thread=multi
should have no effect.

For what it's worth, a boot of debian-9.8.0-i386-netinst.iso
(which is what I happened to have handy) works fine with

  ./qemu-system-i386 -cdrom ... -m 1G --accel tcg,thread=multi

on an x-gene system, compiled for aarch32.

You should note that the default memory allocation is only 128MB,
and I'd be surprised if you can boot the alpine installation in
that little memory. Certainly it does not work with debian.
On the other hand, the failure is a simple guest kernel panic
and not any kind of Illegal Instruction.

I'll try again in a moment with a cortex-a15 system, but I'm
not expecting any different result.

Just so we're clear, which raspberry pi revision are you using?
Based on comments I'm assuming pi 2, with the 4 core cortex-a7.

[TimeMaster (timemaster5)]

>> ./qemu-system-i386 -cdrom alpine.iso --accel tcg,thread=multi

I tried both, with -smp 4 and without..

In stable Qemu release, there is just a guest kernel panic, in v4.0.0-rc3-dirty, there is also Illegal instruction error reported

> on an x-gene system, compiled for aarch32.
Yes, this should be Raspberry Pi 3, but I am not sure. Anyway, I can test this in one week when I receive one of those boards.

>Just so we're clear, which raspberry pi revision are you using?
>Based on comments I'm assuming pi 2, with the 4 core cortex-a7.

I am using Raspberry Pi 2, 4 cores, not sure where is the difference between cortex-a7 and ARMv7, in my case it is just ARMv7.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]