qemu-system-arm hangs at start on OS X

[Eric Allen]

Both from release 2.1.2 and built from a recent source, qemu-system-arm seems to hang immediately after starting up, never getting to the point of actually booting.

I've tried qemu-system-mipsel with another image and it worked fine, so this seems to be specific to the ARM runtime. I've tried two different ARM kernels, and I also ran into this with QEMU 2.1.2 release, installed from a bottle using homebrew.

Host: Mac OS X 10.9.5 (Darwin Kernel Version 13.4.0)
QEMU version: built from HEAD@ab0302ee76
Build command: ./configure --enable-cocoa --target-list=arm-softmmu,mipsel-softmmu && make
Run command:

qemu-system-arm -M vexpress-a9 -cpu cortex-a9 -m 256 -sd disk.img -net nic,macaddr=52:54:00:fa:ce:13 -kernel vmlinuz-3.2.0-4-vexpress -initrd initrd.gz -append "root=/dev/ram" -display vnc=localhost:17 -net user,hostfwd=tcp::5022-:22 -append "console=ttyS0"

I also tried this, with a different kernel & root:

qemu-system-arm -kernel zImage -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -hda rootfs-chromium.ext2 -append "root=/dev/sda"

Thread dump:

(lldb) thread list
Process 34364 stopped
* thread #1: tid = 0x135966, 0x00007fff89f4a746 libsystem_kernel.dylib`__psynch_mutexwait + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  thread #2: tid = 0x13598b, 0x00007fff89f4ae6a libsystem_kernel.dylib`__workq_kernreturn + 10
  thread #3: tid = 0x13598c, 0x00007fff89f4b662 libsystem_kernel.dylib`kevent64 + 10, queue = 'com.apple.libdispatch-manager'
  thread #7: tid = 0x1359b2, 0x00007fff89f4acc2 libsystem_kernel.dylib`__sigwait + 10
  thread #9: tid = 0x1359c1, 0x00000001091bc5d9
  thread #11: tid = 0x1359cc, 0x00007fff89f4a716 libsystem_kernel.dylib`__psynch_cvwait + 10
  thread #12: tid = 0x1359da, 0x00007fff89f46a1a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.audio.IOThread.client'

-------
* thread #1: tid = 0x135966, 0x00007fff89f4a746 libsystem_kernel.dylib`__psynch_mutexwait + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x00007fff89f4a746 libsystem_kernel.dylib`__psynch_mutexwait + 10
    frame #1: 0x00007fff8e05f779 libsystem_pthread.dylib`_pthread_mutex_lock + 372
    frame #2: 0x000000010033e8e9 qemu-system-arm`qemu_mutex_lock(mutex=<unavailable>) + 25 at qemu-thread-posix.c:76
    frame #3: 0x000000010002d742 qemu-system-arm`qemu_mutex_lock_iothread + 98 at cpus.c:1137
    frame #4: 0x00000001002c84b5 qemu-system-arm`main_loop_wait [inlined] os_host_main_loop_wait(timeout=<unavailable>) + 191 at main-loop.c:242
    frame #5: 0x00000001002c83f6 qemu-system-arm`main_loop_wait(nonblocking=<unavailable>) + 278 at main-loop.c:494
    frame #6: 0x000000010014961a qemu-system-arm`qemu_main [inlined] main_loop + 73 at vl.c:1789
    frame #7: 0x00000001001495d1 qemu-system-arm`qemu_main(argc=<unavailable>, argv=<unavailable>, envp=<unavailable>) + 17057 at vl.c:4353
    frame #8: 0x000000010029b45e qemu-system-arm`-[QemuCocoaAppController startEmulationWithArgc:argv:](self=<unavailable>, _cmd=<unavailable>, argc=<unavailable>, argv=<unavailable>) + 30 at cocoa.m:897

[Peter Maydell (pmaydell)]

Do these guest images and command lines work on Linux QEMU? (The most common cause of "nothing happens" is "wrong kernel for this board" or similar misconfiguration, where QEMU is correctly emulating a crashed guest that never made any output...)

[Eric Allen (ericpallen)]

Ah, good question! I found an image and instructions at http://fedoraproject.org/wiki/Architectures/ARM/HowToQemu#Using_QEMU_without_libvirt that was a bit easier to work through, and sure enough, it works on Linux but not on OS X.

Linux precise64 3.2.0-37-generic:

vagrant@precise64:/opt/qemu-images/arm/fedora$ /home/vagrant/qemu-2.2.0/arm-softmmu/qemu-system-arm -M versatilepb -kernel zImage-qemu-versatile-3.0.8-4.fc17.armv5tel -hdc rootfs-f12 -append "root=0800 console=ttyAMA0" -nographic
audio: Could not init `oss' audio driver
Uncompressing Linux... done, booting the kernel.
Linux version 3.0.8 (jcapik@vega) (gcc version 4.1.2 20070925 (Red Hat 4.1.2-33.fa1)) #16 Wed Mar 28 15:07:56 CEST 2012
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00093177
CPU: VIVT data cache, VIVT instruction cache
Machine: ARM-Versatile PB
Memory policy: ECC disabled, Data cache writeback
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
...

-------------------------

OS X (just built QEMU 2.2.0 from source):

$ /Users/epall/bigcode/qemu/arm-softmmu/qemu-system-arm -M versatilepb -kernel zImage-qemu-versatile-3.0.8-4.fc17.armv5tel -hdc rootfs-f12 -append "root=0800 console=ttyAMA0" -nographic
Uncompressing Linux... done, booting the kernel.

[Peter Maydell (pmaydell)]

That zImage works for me with QEMU commit ab0302ee76 on OSX 10.9.5 (at least it boots fine to the point of the kernel complaining it couldn't find the rootfs, because I didn't bother to build that.) I tested with the v2.2.0 tag and that works fine too.

Sanity check: use md5sum to check that the images you ended up with on OSX and Linux are actually the same and didn't get corrupted in download somehow. Otherwise I'm not sure what's going on.

[Joel Gallun (joel-gallun)]

Peter, the bug occurs when mounting the rootfs.

I can reproduce this bug too, with a kernel that works perfectly on QEMU on linux, windows and linux running on the same mac under vmware. In the case where I ran it under vmware on the mac the raspi kernel is the same file shared between the host os (os x) and the guest (linux) os.

Here's how I built QEMU on my brandy new mac

   34 xcode-select --install
   35 ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
   38 brew doctor
   47 brew install pkg-config
   51 git checkout 2b7b4b3 Library/Formula/qemu.rb
   53 $: brew install https://raw.github.com/Homebrew/homebrew-dupes/master/apple-gcc42.rb
   54 brew install https://raw.github.com/Homebrew/homebrew-dupes/master/apple-gcc42.rb
   55 brew install qemu --env=std --cc=gcc-4.2
   56 cd
   57 cd Desktop/qemu

here's what happens when I run it:

GSSLA40052111:Qemu jgallun$ cat pi.sh
#!/bin/sh

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -hda 2014-09-09-wheezy-raspbian.img

I've attached a screenshot of what happens when I boot this kernel on the mac.

The kernel I used in this example came from this website: http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/

[Peter Maydell (pmaydell)]

 > brew install qemu --env=std --cc=gcc-4.2

Aha. Don't do that, that's an ancient gcc. Use the system 'clang' (which configure should pick by default).

[Joel Gallun (joel-gallun)]

Thanks for the quick response. Being a total mac n00b I just followed the directions in the top google link for installing qemu on os x and I ended up where I did. I replaced the old version of the qemu formula in my brew library with the current one and re-installed and all is well, running qemu 2.2.0. Not that it matters, but the directions I followed have you checkout an old version of qemu (1.1.0) that won't compile with clang or a modern gcc, hence the ancient version of gcc