I hit this today on QEMU head. The problem appears to crop up when:
1. Decoding a VEX instruction (see [1]) that uses the 0x66 mandatory
prefix; and
2. The OSFXSR bit in CR4 is clear (that is, SSE is disabled)
This means that x86_64 instructions such as:
c4 e2 f9 f7 c0 shlxq %rax, %rax, %rax
fail. Similar instructions the use a different mandatory prefix
(such as `shrxq`, which uses prefix 0xf2) work fine.
Most operating systems presumably set the OSFXSR bit fairly early on, which I
guess is why this problem isn't likely to be seen except in low-level or early
boot code.
The culprit appears to be the block of code in `gen_sse` [2]:
Removing the check `... || (s->prefix & DATA_DATA)` causes QEMU to correctly
translate the instruction, and allows doug16k's test above to pass.
I must confess, I'm not clear what this clause was testing for. My best guess
is that early code (e.g. 4242b1bd8ac) required it to avoid accessing invalid
opcode tables, but we seem to be handling that more gracefully today (e.g.
[3]), so I suspect it is no longer needed.
I hit this today on QEMU head. The problem appears to crop up when:
1. Decoding a VEX instruction (see [1]) that uses the 0x66 mandatory
prefix; and
2. The OSFXSR bit in CR4 is clear (that is, SSE is disabled)
This means that x86_64 instructions such as:
c4 e2 f9 f7 c0 shlxq %rax, %rax, %rax
fail. Similar instructions the use a different mandatory prefix
(such as `shrxq`, which uses prefix 0xf2) work fine.
Most operating systems presumably set the OSFXSR bit fairly early on, which I
guess is why this problem isn't likely to be seen except in low-level or early
boot code.
The culprit appears to be the block of code in `gen_sse` [2]:
if (is_xmm
&& !(s->flags & HF_OSFXSR_MASK)
&& ((b != 0x38 && b != 0x3a) || (s->prefix & PREFIX_DATA))) {
goto unknown_op;
}
Removing the check `... || (s->prefix & DATA_DATA)` causes QEMU to correctly
translate the instruction, and allows doug16k's test above to pass.
I must confess, I'm not clear what this clause was testing for. My best guess
is that early code (e.g. 4242b1bd8ac) required it to avoid accessing invalid
opcode tables, but we seem to be handling that more gracefully today (e.g.
[3]), so I suspect it is no longer needed.
[1]: https:/ /wiki.osdev. org/X86- 64_Instruction_ Encoding# VEX.2FXOP_ opcodes /github. com/qemu/ qemu/blob/ 6b63d126121a953 5784003924fcb67 f574a6afc0/ target/ i386/tcg/ translate. c#L3078 /github. com/qemu/ qemu/blob/ 6b63d126121a953 5784003924fcb67 f574a6afc0/ target/ i386/tcg/ translate. c#L3696- L3700
[2]: https:/
[3]: https:/