images used as scsi disks not readable (qemu-system-arm, macos 10.8)

[Daria Brashear]

Using a arm1176 kernel and the raspbian image (10-28 or 12-16) as my disk, I get as far as mounting root and then get SCSI errors with 1.3.0 and the current origin/master. git bisect says the issue is

commit f563a5d7a820424756f358e747238f03e866838a
Merge: a273652 aee0bf7
Author: Paolo Bonzini <email address hidden>
Date: Wed Oct 31 10:42:51 2012 +0100

    Merge remote-tracking branch 'origin/master' into threadpool

    Signed-off-by: Paolo Bonzini <email address hidden>

I am using:
qemu-system-arm -no-reboot -M versatilepb -cpu arm1176 -m 256 -hda 2012-12-16-wheezy-raspbian.img -kernel kernel-qemu -append "root=/dev/sda2 rootfstype=ext4 elevator=deadline rootwait panic=1" -serial stdio -usbdevice tablet -net nic -net user,hostfwd=tcp::40022-:22

Configured on MacOS 10.8.2 with current Xcode and MacPorts installed, thus:
CPATH=/opt/local/include CFLAGS="-pipe -O2 -arch x86_64" CPPFLAGS="-I/opt/local/include" CXXFLAGS="-pipe -O2 -arch x86_64" LIBRARY_PATH="/opt/local/lib" MACOSX_DEPLOYMENT_TARGET="10.8" CXX="/usr/bin/clang++" LDFLAGS="-L/opt/local/lib -arch x86_64" OBJC=/usr/bin/clang FCFLAGS="-pipe -O2 -m64" INSTALL="/usr/bin/install -c" OBJCFLAGS="-pipe -O2 -arch x86_64" CC="/usr/bin/clang" ./configure --prefix=/opt/local --cpu=x86_64 --cc=/usr/bin/clang --objcc=/usr/bin/clang --host-cc=/usr/bin/clang --python=/opt/local/bin/python2.7 --target-list=arm-softmmu

[Joss Reeves (joseph-reeves)]

I duplicated this as well. I tried both the qemu-system-arm available in macports and also from homebrew with the same results. Host system is also 10.8 "mountain lion".

My boot command: qemu-system-arm -kernel kernel/zImage -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1" -hda pifi-4g.img -redir tcp:5022::22

I'm running QEMU emulator version 1.4.1 on OS X 10.8.3
Interestingly, this exact same boot command works perfectly on my Ubuntu virtual (in virtualbox) running QEMU emulator version 1.4.0 (Debian 1.4.0+dfsg-1expubuntu4)

So the bug would seem to either be specific to OS X or to the 1.4.1 release.

See the attached screenshot to see what happens in the boot process.

[Joss Reeves (joseph-reeves)]

I managed to capture a little more info about this bug by passing -drive file='myharddrive.img'. The kernel panic is happening in the sym53c8xx driver. See the attached screenshot for detail.

I can also attach the kernel that I'm using if needed. Just let me know.

[Peter Maydell (pmaydell)]

I suspect this may be because we were defaulting to a broken coroutine backend (a bug fixed with commit 7c2acc7). Can you retry with the current 1.5 release candidate? (source download available at http://wiki.qemu.org/Download)

[Joss Reeves (joseph-reeves)]

Hi Peter,

Thanks, that made an improvement. Now I'm just stuck in a loop of the kernel resetting the scsi bus :)

(see attachment)

[Peter Maydell (pmaydell)]

And the same QEMU/kernel/image works fine on a Linux host?

If you can provide the files I need to reproduce I might be able to take a look at it. (If it did the same thing on linux host that would be higher priority for me, so if you can cross-check that would be helpful.)

[Joss Reeves (joseph-reeves)]

I just compiled 1.5.0-rc1 on my Linux host with the same configure/compiler flags and duplicated the error (see screenshot). The configure flags are:
./configure --disable-guest-agent --disable-bsd-user --enable-sdl --target-list="arm-linux-user armeb-linux-user arm-softmmu"

As before, it goes into an infinite loop of reseting the scsi controller for each (emulated) channel. Note that the Ubuntu provided qemu-system-arm works perfectly. They are using 1.4.0 with a rather large number of patches ( I did a `dpkg source qemu` to examine the Ubuntu build setup).

Looking through the Ubuntu patches, this one looks like a likely fix:
dpkg-source: info: applying patches-arm-1.4.0/0002-hw-sd-Expose-sd_reset-as-public-function.patch

Please find a zip of my raspberrypi image (hda) and the kernel that I built from https://github.com/raspberrypi/linux.git
at https://www.dropbox.com/sh/mbz8jh4fcjvdj4m/Gh3bKFyJyC

I included the boot command in a txt file in the tarball.

cheers,
Joss

[Peter Maydell (pmaydell)]

It's very unlikely to be the patch you mention, since that's for SD card emulation and you're not using SD card emulation. It's probably just a regression between 1.4 and 1.5, and I'm fairly sure it's in some changes I made to the versatilepb PCI controller model -- I will investigate.

[Joss Reeves (joseph-reeves)]

Ah, I interpreted it to mean "scsi disk" instead of SD card :)

I'll leave this to the experts. Thanks so much for looking into this and please let me know if I can be of further assistance.

-Joss

[Joss Reeves (joseph-reeves)]

Hi Peter,

Thanks so much for the patch and including me on the thread. I can confirm that it did fix the problem running on a Linux host, but the OS X bug cited by myself and the OP still remains elusive. It's rather puzzling as I pulled from HEAD and built using the same options on both. I've gotten a bit better with the qemu options now, so I will paste the console output here instead of doing yet another screenshot :) As you can see, it's still getting a fatal exception in the interrupt code. Do you know of a kernel version that would be better behaved than the 3.6.11+ from the "raspberrypi/linux" repo on github? Could I provide a core file that would help?

Thanks again for your efforts.
Joss

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
phoenix:RaspberryPi mysfitt$ qemu-system-arm -kernel kernel/zImage -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 console=ttyAMA0" -redir tcp:5022::22 -bt hci,null -global versatile_pci.broken-irq-mapping=1 pifi-4g.img
Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0
Linux version 3.6.11+ (root@jossibox) (gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-1ubuntu1) ) #1 Fri May 10 16:46:40 EDT 2013
CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
CPU: VIPT aliasing data cache, unknown instruction cache
Machine: ARM-Versatile PB
Memory policy: ECC disabled, Data cache writeback
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 65024
Kernel command line: root=/dev/sda2 panic=1 console=ttyAMA0
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 256MB = 256MB total
Memory: 255388k/255388k available, 6756k reserved, 0K highmem
Virtual kernel memory layout:
    vector : 0xffff0000 - 0xffff1000 ( 4 kB)
    fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
    vmalloc : 0xd0800000 - 0xff000000 ( 744 MB)
    lowmem : 0xc0000000 - 0xd0000000 ( 256 MB)
    modules : 0xbf000000 - 0xc0000000 ( 16 MB)
      .text : 0xc0008000 - 0xc03f6458 (4026 kB)
      .init : 0xc03f7000 - 0xc04162bc ( 125 kB)
      .data : 0xc0418000 - 0xc043fb60 ( 159 kB)
       .bss : 0xc043fb84 - 0xc045abb0 ( 109 kB)
NR_IRQS:192
VIC @f1140000: id 0x00041190, vendor 0x41
FPGA IRQ chip 0 "SIC" @ f1003000, 21 irqs
Console: colour dummy device 80x30
Calibrating delay loop... 626.68 BogoMIPS (lpj=3133440)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
Setting up static identity map for 0x305ce0 - 0x305d3c
devtmpfs: initialized
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
Serial: AMBA PL011 UART driver
dev:f1: ttyAMA0 at MMIO 0x101f1000 (irq = 12) is a PL011 rev1
console [ttyAMA0] enabled
dev:f2: ttyAMA1 at MMIO 0x101f2000 (irq = 13) is a PL011 rev1
dev:f3: ttyAMA2 at MMIO 0x101f3000 (irq = 14) is a PL011 rev1
fpga:09: ttyAMA3 at MMIO 0x10009000 (irq = 38) is a PL011 rev1
PCI core found (sl...

[Peter Maydell (pmaydell)]

On 15 May 2013 19:02, Joss Reeves <email address hidden> wrote:
> Thanks so much for the patch and including me on the thread. I can
> confirm that it did fix the problem running on a Linux host, but the OS
> X bug cited by myself and the OP still remains elusive. It's rather
> puzzling as I pulled from HEAD and built using the same options on both.

QEMU itself actually hangs in my tests (the main loop is waiting
to lock the iothread but it never does; the cpu thread seems to
be stuck trying to do a bdrv_aio_cancel for the scsi device model).
This should never happen, regardless of what the guest does...

I suspect that if you configure on linux with --with-coroutine=sigaltstack
you might then find they both behave the same (MacOSX can't do the
ucontext coroutines we default to on linux). OTOH it might also
involve some of MacOSX's slightly different signal behaviour.

I'll continue to prod, though past experience is that MacOSX
gdb is weirdly broken and things behave differently when run
under it, which doesn't help :-(

-- PMM

[Peter Maydell (pmaydell)]

On 15 May 2013 21:18, Peter Maydell <email address hidden> wrote:
> I suspect that if you configure on linux with --with-coroutine=sigaltstack
> you might then find they both behave the same (MacOSX can't do the
> ucontext coroutines we default to on linux).

They don't, so it's a MacOSX specific issue of some kind.
PS: you don't need "-global versatile_pci.broken-irq-mapping=1"
in the command line because we do correctly autodetect and
handle your kernel now.

thanks
-- PMM

[Thomas Huth (th-huth)]

Triaging old bug tickets... can you still reproduce this issue with the latest version of QEMU? Or could we close this ticket nowadays?