On 09/19/15 12:54, Marina Kovalevna wrote: > Hello boyos, > > I got myself an Rpi2 recently and have been reading up on qemu. > > Does this mean there's a problem booting x86 images on Arm or just the > ones from that particular source? > The outlandishness of this use case (-> buy an underpowered toy, run x86 programs on it via *emulation*) is so exceptional that it tickled my fancy and I looked into it. I'm CCing Peter and Dave; I can see in the LP comments from 2011 that they looked at this in 2011-2012. We're going to have a good chuckle here I promise. So, Dave was correct in comment #1 where he wrote, "it falls with a triple fault from a divide instruction not long after a load of rdtsc stuff". I booted the same Debian i386 Squeeze (and Wheezy) "standard" images from Aurelien's website as everyone else, in TCG mode, both on an x86_64 host, and -- "brace for impact" -- on an aarch64 host (APM Mustang). The command line was simple, $ qemu-system-i386 -hda debian_squeeze_i386_standard.qcow2 The symptoms reproduced on the aarch64 host, and didn't on the x86_64 host. Then I added -d in_asm,op,int,exec,cpu,mmu,cpu_reset,ioport,unimp,guest_errors to capture the TCG logs, up to the point where grub rebooted (vs. didn't reboot, on the x86_64 host), and then diffed the logs between each other. (This wasn't so fast, on the aarch64 host, approx. 530 MB of log was written before the reboot.) Looking at the logs, I can confirm Dave's analysis from 2011 -- there's a CPUID, then an RDTSC, then a division by zero. So if you look at GRUB's code, you find the calibrate_tsc() function in "grub-core/kern/i386/tsc.c". (We're old friends with that function.) It calls grub_get_tsc() -- same file --, which explains both CPUID and RDTSC. (CPUID is only used for serialization, ie. for preventing the CPU from executing RDTSC out-of-order. RDTSCP would be an alternative, which combines both, but that's not as widely available.) Where does the division by zero come from then? Well grub fetches and stashes the TSC, then programs the PIT to sleep for some time, then re-fetches the TSC, and uses the TSC difference as denominator when calculating the "TSC rate". (It has a solid idea of the real time passed, due to the PIT frequency being a given.) Let's see where the TSC values come from in QEMU / TCG: helper_rdtsc() [target-i386/misc_helper.c] cpu_get_tsc() [hw/i386/pc.c] cpu_get_ticks() [cpus.c] cpu_get_real_ticks() [include/qemu/timer.h] Now, the cpu_get_real_ticks() implementation is *host* specific. You can find it implemented for a bunch of host architectures in "include/qemu/timer.h". Neither ARM nor AARCH64 qualify though; for those, the following pearlescent fallback gets built: > /* The host CPU doesn't have an easily accessible cycle counter. > Just return a monotonically increasing value. This will be > totally wrong, but hopefully better than nothing. */ > static inline int64_t cpu_get_real_ticks (void) > { > static int64_t ticks = 0; > return ticks++; > } Note that this code dates back to the following commit (year 2006): commit 46152182100e68f7f8aa4954af1bf91160bb3d15 Author: pbrook Date: Sun Jul 30 19:16:29 2006 +0000 Rewrite Arm host support. So... the frequency of the PIT is 1193182 per second (see PIT_FREQ in QEMU, and GRUB_SPEAKER_PIT_FREQUENCY in GRUB). Grub sleeps for 65535 such cycles in calibrate_tsc(), between the two TSC reads. That's approximately 55 milliseconds. And for that long period, grub finds that the TSC has incremented by ... one. (Side remark: you'll find that recent grub versions don't choke on this. See , from January 2015.) I applied the following extremely sophisticated patch (with the motto "it cannot get more wronger"): > diff --git a/include/qemu/timer.h b/include/qemu/timer.h > index 9939246..def22de 100644 > --- a/include/qemu/timer.h > +++ b/include/qemu/timer.h > @@ -1003,8 +1003,7 @@ static inline int64_t cpu_get_real_ticks(void) > totally wrong, but hopefully better than nothing. */ > static inline int64_t cpu_get_real_ticks (void) > { > - static int64_t ticks = 0; > - return ticks++; > + return get_clock(); > } > #endif > get_clock() is CLOCK_MONOTONIC based, has (theoretical) nanosecond resolution, and a nice flat int64_t encoding that should suffice for approx. 329 years. This should provide grub with a larger denominator. This "fix" allowed me to boot the i386 Debian image on the AARCH64 host. For a real fix... I think on AARCH64 hosts at least, a "real" cycle counter should be available, and someone who knows AARCH64 could write a function that fetches it. For 32-bit ARM, I presume the Raspberry Pi 2 and the Odroid C1 are advanced enough for a similar cycle counter reading function. I wonder though if ARM platforms remain in existence for which the 2006 patch captures the right hardware capability. For those (and perhaps as a general fallback) I think my "patch" above would be an improvement. Peter? Thanks Laszlo