Instructions not supported by targeted CPU do not throw SIGILL

[Jonathan Morton]

We encountered a bug in another package that caused it to include CMOV instructions when targetting i486, resulting in an inability to run the package on real i486 and i586 hardware. We then attempted to use QEMU to reproduce the bug for easier debugging, since most developers have long since got rid of such old hardware.

QEMU appears to continue to support *all* instructions when -cpu=486 is selected, regardless of what is advertised in CPUID to the guest. CPUID describes the host environment as a reasonably close approximation to a late-model i486, with very few instruction extensions - specifically excluding CMOV, which on real hardware is an optional extension to the i686 architecture.

The result was that we could not reproduce the bug using QEMU, and must therefore attempt to debug it using a very limited stock of real hardware, which also has very limited performance for rebuilding the package. This completely defeats one of the main uses of QEMU, in our opinion.

If this bug extends to other CPU architectures, it would affect all developers wishing to check whether their code conforms to restrictions imposed by any older or more restrictive ISA specification than the latest that QEMU supports, including the distinctions between ARMv7-A-NEON, ARMv7-A-VFPv3, ARMv7-A-VFPv3-d16, ARMv7-R, ARMv7-M, ARMv6-VFPv2, ARMv5-TE, ARMv4-T... all of which are currently shipping in new devices.

Attached is a small C program which can easily be compiled to include CMOV instructions. It can be used to reproduce the bug:

$ gcc -march=i486 -O2 -c minmax.c -o minmax
$ ./minmax
No arguments!
$ ./minmax 5 6 7
max: 7 min: 5
$ gcc -march=pentium2 -O2 -c minmax.c -o minmax-p2
$ ./minmax-p2
No arguments!
$ ./minmax-p2 5 6 7
[Expected, occurs on real i4/586 hardware:] Illegal instruction
[Actual, within QEMU v1.2.0 with -cpu=486:] max: 7 min: 5
$

The bug is likely not limited to CMOV, but would also apply to more recent ISA extensions - so 3DNow! instructions would appear to run on Intel guest CPUs, AVX on a Pentium-2, and other such weirdness.

[Peter Maydell (pmaydell)]

On 15 July 2013 15:17, Jonathan Morton <email address hidden> wrote:
> We encountered a bug in another package that caused it to include CMOV
> instructions when targetting i486, resulting in an inability to run the
> package on real i486 and i586 hardware. We then attempted to use QEMU
> to reproduce the bug for easier debugging, since most developers have
> long since got rid of such old hardware.

> The result was that we could not reproduce the bug using QEMU, and must
> therefore attempt to debug it using a very limited stock of real
> hardware, which also has very limited performance for rebuilding the
> package. This completely defeats one of the main uses of QEMU, in our
> opinion.

QEMU's main aim (and where most of its testing and use happens)
is to run code which is known to already run on hardware; since
its instruction emulation is not exhaustively tested it is likely
to have bugs in odd corner cases, and not faulting nonexistent
instructions is a bug you're not going to find unless you go looking
for it.

So:
 * I agree that this is a bug in QEMU, which we should fix
 * if you want certainty about "will this run on a 486" you
   need to run on 486 hardware, or on an emulator that's been
   comprehensively validated
 * QEMU is still useful as a "first pass" test and for easier
   debugging

Incidentally, if you're using KVM acceleration then these
instructions won't fault in that setup because the host CPU
hardware doesn't provide a means for trapping them.

> If this bug extends to other CPU architectures, it would affect all
> developers wishing to check whether their code conforms to restrictions
> imposed by any older or more restrictive ISA specification than the
> latest that QEMU supports, including the distinctions between
> ARMv7-A-NEON, ARMv7-A-VFPv3, ARMv7-A-VFPv3-d16, ARMv7-R, ARMv7-M,
> ARMv6-VFPv2, ARMv5-TE, ARMv4-T... all of which are currently shipping
> in new devices.

ARM is in the same position as x86 -- we do check feature bits
and generate UNDEF on instructions that shouldn't exist, but
since we don't have a complete comprehensive set of validation
tests that we run on QEMU there may be errors.

> The bug is likely not limited to CMOV, but would also apply to more
> recent ISA extensions - so 3DNow! instructions would appear to run on
> Intel guest CPUs, AVX on a Pentium-2, and other such weirdness.

Actually, we do do testing of feature bits for things like
SSE2, 3DNow!, and so on -- it looks like we just missed cmov.

thanks
-- PMM

[Jonathan Morton (chromatix99)]

> Actually, we do do testing of feature bits for things like SSE2, 3DNow!, and so on -- it looks like we just missed cmov.

That's encouraging to hear - I hadn't specifically tested for the behaviour in the other cases I described, just extrapolated.

Is there a good chance that the CMOV thing will be fixed?

[Jonathan Morton (chromatix99)]

> Incidentally, if you're using KVM acceleration then these
> instructions won't fault in that setup because the host CPU
> hardware doesn't provide a means for trapping them.

This I already understood, and we made sure that we were running with the TCG backend.

[Peter Maydell (pmaydell)]

On 15 July 2013 16:35, Jonathan Morton <email address hidden> wrote:
>> Actually, we do do testing of feature bits for things like SSE2,
> 3DNow!, and so on -- it looks like we just missed cmov.
>
> That's encouraging to hear - I hadn't specifically tested for the
> behaviour in the other cases I described, just extrapolated.
>
> Is there a good chance that the CMOV thing will be fixed?

Well, target-i386 is listed in MAINTAINERS as status "Odd Fixes",
which means it doesn't get a great deal of love; however since
this is probably about a five-line fix you may well be lucky and
find that somebody writes a patch for it. The most tedious
bit is probably wading through the intel manuals to confirm
the affected instruction encodings and which feature bit
to test.

-- PMM

[Jonathan Morton (chromatix99)]

The CMOV feature bit (bit 15 of EDX) should affect availability of the following instructions:

CMOVcc (0F 40 -> 0F 4F)
FCMOVcc (DA C0 -> DA DF and DB C0 -> DB DF)
FCOMI family (DB E8 -> DB F7 and DF E8 -> DF F7)

HTH

[Peter Maydell (pmaydell)]

Patch which I think should fix this bug: http://patchwork.ozlabs.org/patch/259148/

[Jonathan Morton (chromatix99)]

Yes, that seems to work nicely, at least as far as CMOV itself is concerned.

[Peter Maydell (pmaydell)]

This was fixed in commit bff93281a75def2e3 (which was the patch from comment #7).

[Jonathan Morton (chromatix99)]

Wow, a full 3 years later!

Well, better late than never...

[Peter Maydell (pmaydell)]

The fix went into master and got released three years ago -- we just forgot to mark the bug as closed :-)