QNX 4 doesn't boot on qemu >= 1.3

[JQu]

I am using virtual machine with QNX4 operating system installed on it. I updated my qemu from version
to newer and QNX4 doesn't start any more. All is ok on version 1.2 but when I try to use any newer version
(1.3, 1.4, 1.5) QNX4 doesn't boot. I tried on windows and linux ubuntu hosts - effects are the same.

When virtual machine boots qnx bootloader loads and starts operating system. In the next step
qnx starts its ide driver, which detects qemu harddisk and cdrom. Problem starts when operating system
tries mount partition - an error occur and qnx stop booting procedure:

mount -p "No bios signature in partition sector on /dev/hd0"

I have tried install qnx from cdrom but it seems that there is the same problem. QNX installer boot from
cdrom, detects hard disk and cdrom, but cdrom can't be mounted in the next step of installation procedure.

[JQu (damian-p-jakubowski)]

with qemu 1.6 is even worse - qemu crash every time when QNX detects hard disk

[Andreas Färber (afaerber)]

Please use git-bisect to find out which change between 1.2.0 and 1.3.0 broke things for you.

[JQu (damian-p-jakubowski)]

problem appeared in this commit:

commit b90600eed3c0efe5f3260853c873caf51c0677b1
Author: Avi Kivity <email address hidden>
Date: Wed Oct 3 16:42:37 2012 +0200

    dma: make dma access its own address space

    Instead of accessing the cpu address space, use an address space
    configured by the caller.

    Eventually all dma functionality will be folded into AddressSpace,
    but we have to start from something.

    Reviewed-by: Anthony Liguori <email address hidden>
    Signed-off-by: Avi Kivity <email address hidden>

[JQu (damian-p-jakubowski)]

Output from valgrind running latest qemu downloaded from git. Qemu crashed of course.
If I can check something more, please let me know.

==29109== Memcheck, a memory error detector
==29109== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==29109== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==29109== Command: qemu-system-i386 -no-kvm -hda /home/jq/QNX4.vmdk
==29109== Parent PID: 15280
==29109==
==29109== Invalid write of size 8
==29109== at 0x4C2CD8D: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==29109== by 0x4DF292: iov_from_buf (iov.c:37)
==29109== by 0x4E01B8: qemu_iovec_from_buf (iov.c:374)
==29109== by 0x1A0CA6: bdrv_aio_bh_cb (block.c:3820)
==29109== by 0x186CEB: aio_bh_poll (async.c:81)
==29109== by 0x18693D: aio_poll (aio-posix.c:188)
==29109== by 0x1870FA: aio_ctx_dispatch (async.c:205)
==29109== by 0x5081AB4: g_main_context_dispatch (gmain.c:2715)
==29109== by 0x3235CE: glib_pollfds_poll (main-loop.c:189)
==29109== by 0x3236C2: os_host_main_loop_wait (main-loop.c:234)
==29109== by 0x32379A: main_loop_wait (main-loop.c:484)
==29109== by 0x3B0776: main_loop (vl.c:2090)
==29109== Address 0x157c8ff8 is not stack'd, malloc'd or (recently) free'd
==29109==
==29109== Invalid read of size 4
==29109== at 0x3C4B85: ldl_p (bswap.h:262)
==29109== by 0x3C4CC6: ldl_le_p (bswap.h:295)
==29109== by 0x3CAAC2: address_space_rw (exec.c:1953)
==29109== by 0x3CAE0C: address_space_write (exec.c:2021)
==29109== by 0x3CB570: address_space_unmap (exec.c:2230)
==29109== by 0x1EF736: dma_memory_unmap (dma.h:146)
==29109== by 0x1EFCBD: dma_bdrv_unmap (dma-helpers.c:108)
==29109== by 0x1EFE35: dma_bdrv_cb (dma-helpers.c:146)
==29109== by 0x1A0FE0: bdrv_co_em_bh (block.c:3901)
==29109== by 0x186CEB: aio_bh_poll (async.c:81)
==29109== by 0x18693D: aio_poll (aio-posix.c:188)
==29109== by 0x1870FA: aio_ctx_dispatch (async.c:205)
==29109== Address 0x157ba000 is 0 bytes after a block of size 4,096 alloc'd
==29109== at 0x4C29CD5: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==29109== by 0x4C29D2E: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==29109== by 0x4DA0AB: qemu_memalign (oslib-posix.c:90)
==29109== by 0x3CB322: address_space_map (exec.c:2162)
==29109== by 0x1EF6BE: dma_memory_map (dma.h:137)
==29109== by 0x1EFEEF: dma_bdrv_cb (dma-helpers.c:156)
==29109== by 0x1F0205: dma_bdrv_io (dma-helpers.c:219)
==29109== by 0x1F027A: dma_bdrv_read (dma-helpers.c:228)
==29109== by 0x2724C4: ide_dma_cb (core.c:676)
==29109== by 0x278AC2: bmdma_cmd_writeb (pci.c:324)
==29109== by 0x2792AA: bmdma_write (piix.c:76)
==29109== by 0x43535C: memory_region_write_accessor (memory.c:440)
==29109==

valgrind: m_mallocfree.c:266 (mk_plain_bszB): Assertion 'bszB != 0' failed.
valgrind: This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.

==29109==...

[Paolo Bonzini (bonzini)]

On Linux hosts are you using KVM? Does it make any difference?

Is there a freely downloadable image that we can use for debugging?

Thanks,

Paolo

[JQu (damian-p-jakubowski)]

KVM doesnt make any difference.

[Ray Senez (raysenez)]

I am also experiencing this problem QNX 4.25 images that were created under 1.0 no longer work when I've upgraded to 2.0 or 2.1 qemu.

The error message that I receive is the same. The problem is with the virtual disk driver, it performs the initial boat loader, then when the OS goes to load the file system it fails.

[Thomas Huth (th-huth)]

Triaging old bug tickets ... can you still reproduce this problem with the latest version of QEMU (currently v2.9.0)?

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]

[Dmitry Ilyin (widgetii)]

I've just tried to launch QNX 4.25 ISO bootdisk on qemu-system-x86_64 built from current git.
QEMU screen is skewed (see on screenshot). I tried all different options in -vga key - all the same.

[Lukas Jirkovsky (l-jirkovsky)]

This is still a problem with QEMU 2.11.

Note that the problem in #10 is unrelated to the issue (though GUI works for with the the demo floppy). The problem mentioned in this bug is related to a QNX that is already installed to a disk. Unfortunately the QNX demo floppy that was once free and which can be still found on the internet does not allow installation to disk, so it can't be used to reproduce the bug.

At work we still maintain an application for QNX 4.25 so I can help debugging it, but I don't know where to start. Currently we use VirtualBox and VMware, but I personally would prefer QEMU much more.

Please let me know if there is any way how I could help to sort this issue out.

[Lukas Jirkovsky (l-jirkovsky)]

I wonder - would be a record using rr of any help? I can create record for QEMU 1.2.0 where it works and on current QEMU.

Also, I did a bit of debugging myself around the DMA code as per comment #3 it was introduced in a commit that changed some of the DMA. What I did was that I added some debug printfs [1] to dma_memory_rw() to QEMU 1.2.0 and to QEMU 2.11.1.

I noticed on thing - there is a big difference between writes between the two versions.
Because this stuff is completely outside my knowledge, I don't know whether this is important or not, but better more information that not enough. For recent versions of QEMU I see a few 16 B writes from address 0x6d10 and addresses close to it which contain some data. Immediately after that there is a ton of 8B writes from addresses starting at 0x102004 which contain zeros only. On the other hand, the QEMU 1.2.0 is missing the initial 16B writes, but then there's even more 8B writes from addresses around 0x102004 which contain some data instead of zeros like in the current version.

[1] the printf looks like this:

printf("DEBUG: DMA %s at address %lx %lu bytes: ", ((dir == DMA_DIRECTION_FROM_DEVICE) ? "read" : "write"), addr, len);

[Christian Bohr (christian-bohr)]

Hi,

I had the same problem, and maybe it can help. I wrote my own toy OS with a PATA / IDE driver back in 2012 using older version of QEMU and everything worked fine. These days, I tried that on a recent version (2.5) and it failed with exactly the same behaviour - lots of zeros being written during a DMA transfer.

After some research, I found that the behaviour that was changed with 1.3 is that the bus master configuration bit (bit 2 of the PCI command register) is now emulated, and my driver did not set this bit. Apparently, the BIOS does not set it either, so it was off and the DMA transfer silently failed and only wrote zeros.

So I added some code to my init routine that sets this bit, and voila - it worked. I have tried 1.2, 1.3 and 2.5.0 and this works with all of them.

I do not know the internals of QNX, but I learned that apparently also Linux did not set this bit in older version, so it might very well be that QNX does not set it either and this is the issue.

[Lukas Jirkovsky (l-jirkovsky)]

@Christiat: thank you so much, you are right! I put together a quick hack[1] to seabios to forcefully enable bus master bit on ata device and QNX booted!

[1] I just added an unconditional call to the pci_enable_busmaster(pci); to the init_pciata() function in ata.c

[Lukas Jirkovsky (l-jirkovsky)]

It turns out modifying code is not needed at all. The only thing that is needed is to configure SeaBIOS with CONFIG_ATA_DMA=y.

So the steps needed to make QNX 4 work on current QEMU are
1. Download SeaBIOS source and make sure the configuration has CONFIG_ATA_DMA=y set
2. Build SeaBIOS
3. Run qemu such as "qemu-system-i386 -bios /path/to/seabios/bios.bin -hda qnxdisk ..."

[Gerd Hoffmann (kraxel-redhat)]

qemu git master has a prebuilt seabios with CONFIG_ATA_DMA=y now.

[Thomas Huth (th-huth)]

The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid
and which could be closed already. Thus we are setting older bugs to
"Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]