Can't convert to vmdk with the streamOptimized subformat

[Sebastien Douche]

Hi all,
I'm trying to convert a qcow2 image file to the vmdk (on Ubuntu Server 12.04):

# qemu-img convert -f qcow2 -O vmdk -o Stream spv-2912.qcow2 spv-2912-3.vmdk -o subformat=streamOptimized
VMDK: can't write to allocated cluster for streamOptimized
qemu-img: error while writing sector 65: Input/output error

Same result with any input format. Others subformats work but the StreamOptimized is by far the most important one. The only workaround I found is to migrate to raw and then to use VBoxManage (VirtualBox).

[Serge Hallyn (serge-hallyn)]

Thanks for reporting this bug. While the error looks different, I assume it is related to the same general lack of support as bug 905095 (and https://bugzilla.redhat.com/show_bug.cgi?id=548723)

[Serge Hallyn (serge-hallyn)]

Actually, using the same options as you do I do not get a failure.

Can you show the result of

qemu-img info spv-2912.qcow2

[Sebastien Douche (sdouche)]

Hi Serge,
the bug was confirmed by Stefan Hajnoczi[1] and still in QEMU 1.1:

"Unfortunately there has not been a fix. The cause of the bug is known
and described in the email thread you linked. If someone has time to
write a fix and test it, it could be merged. It won't be possible for
QEMU 1.1 since the release is imminent."

[1] http://<email address hidden>/msg114031.html

[Serge Hallyn (serge-hallyn)]

Thanks, Sebastien.

[Luca Giovenzana (lucagiove)]

root@ulisse:/mnt# qemu-img convert -pO vmdk -o subformat=streamOptimized /dev/sdd optimized.vmdk
qemu-img: Could not write to allocated cluster for streamOptimized
qemu-img: error while writing sector 10: Input/output error

root@ulisse:/mnt# dpkg-query -s qemu-kvm
Package: qemu-kvm
Status: install ok installed
Priority: optional
Section: otherosfs
Installed-Size: 97
Maintainer: Ubuntu Developers <email address hidden>
Architecture: amd64
Multi-Arch: foreign
Source: qemu
Version: 2.0.0+dfsg-2ubuntu1.10
Replaces: qemu-kvm-spice, qemu-system-x86 (<< 1.7.0+dfsg-2~)
Provides: kvm, qemu-kvm-spice
Depends: qemu-system-x86 (>= 1.7.0+dfsg-2~)
Breaks: qemu-system-x86 (<< 1.7.0+dfsg-2~)
Conflicts: kvm, qemu-kvm-spice
Description: QEMU Full virtualization on x86 hardware (transitional package)
 QEMU is a fast processor emulator. This package provides just a wrapper
 script /usr/bin/kvm which run qemu-system-x86 in kvm mode.
 .
 Please note that old qemu-kvm configuration files (in /etc/kvm/) are
 no longer used.
Homepage: http://www.qemu.org/
Original-Maintainer: Debian QEMU Team <email address hidden>

root@ulisse:/mnt# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"

[Radoslav Gerganov (rgerganov)]

With the latest codebase there is no error message but the converted streamOptimized image is incorrect. I have a patch for this and will submit it for review soon.

[Radoslav Gerganov (rgerganov)]

My patch has landed: https://github.com/qemu/qemu/commit/3efffc3292d94271a15b1606b4a56adf6c6f04ed

I think we can resolve this bug as fixed. Just one note for those how are trying to use streamOptimized images with VMware: you need to patch the VMDK version because VMware products accept only version 3 for streamOptimized:

# Set VMDK version 3 for foo.vmdk
$ printf '\x03' | dd conv=notrunc of=foo.vmdk bs=1 seek=$((0x4))

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.3+dfsg-5ubuntu6

---------------
qemu (1:2.3+dfsg-5ubuntu6) wily; urgency=medium

  * Make qemu-system-common and qemu-utils depend on qemu-block-extra
    to fix errors with missing block backends. (LP: #1495895)
  * Cherry pick fixes for vmdk stream-optimized subformat (LP: #1006655)
  * Apply fix for memory corruption during live-migration in tcg mode
    (LP: #1493049)
  * Apply tracing patch to remove use of custom vtable in newer glibc
    (LP: #1491972)

 -- Ryan Harper <email address hidden> Tue, 15 Sep 2015 09:37:23 -0500

[Serge Hallyn (serge-hallyn)]

I'm confused- this is marked as affecting qemu-kvm in precise, but the
preceding patch (c6ac36e) which introduced the bug is not there either.
So I'm going to mark this as not affecting precise unless someone speaks
up to say that we in fact need the whole dependent series.

[Thiago Martins (martinx)]

I have tried, many times, to use qemu-img to create StreamOptimized for VMWare ESXi 6.0 OVAs.

It does NOT work.

After days of research, I replaced qemu-img, by vboxmanage, then, it worked!

Now, I'm using something this:

--
vboxmanage convertfromraw packer/output-vm.raw packer/output-vm-disk1.vmdk --format vmdk --variant Stream
--

I also had to edit the headers, with dd:

---
dd if=output-vm-disk1.vmdk of=output-vm-disk1.descriptor bs=1 skip=512 count=1024
sed -i -e 's/ide/lsilogic/g' output-vm-disk1.descriptor
dd conv=notrunc,nocreat if=output-vm-disk1.descriptor of=output-disk1.vmdk bs=1 seek=512 count=1024
---

This is the only way to build StreamOptimized VMDKs, for inclusion inside of OVA, that works on VMWare 6.0.

The qemu-img can not be used.

Here is my full solution to this problem (if the same problem):

https://github.com/tmartinx/svauto/blob/dev/image-factory.sh#L510

I'm just not entirely sure if we're talking about the same problem here...

Cheers!
Thiago

[Thiago Martins (martinx)]

It looks like my problem is different, since I'm not seeing this:

qemu-img: error while writing sector 65: Input/output error

But maybe, it is a light in the end of the tunnel...

[Thomas Huth (th-huth)]

Patch 3efffc3292d9427 had been released with QEMU 2.5