Bug in qemu-img when converting to streamOptimized vmdk images

[Ed Wittmann]

I reviewed #1006655, and I think I'm already on a newer version, so this is either a regression or a new bug.

I have been recently attempting to convert images from raw and qcow2 formats to vmdk. It appears that the option "subformat=streamOptimized" produces a broken or corrupted disk image.

Current versions, running on Devuan testing:

ii ipxe-qemu 1.0.0+git-20190125.36a4c85-1 all PXE boot firmware - ROM images for qemu
ii qemu 1:3.1+dfsg-8+deb10u4 amd64 fast processor emulator, dummy package
ii qemu-efi-aarch64 0~20181115.85588389-3 all UEFI firmware for 64-bit ARM virtual machines
ii qemu-efi-arm 0~20181115.85588389-3 all UEFI firmware for 32-bit ARM virtual machines
ii qemu-kvm 1:3.1+dfsg-8+deb10u4 amd64 QEMU Full virtualization on x86 hardware
ii qemu-slof 20180702+dfsg-1 all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries
ii qemu-system-arm 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (arm)
ii qemu-system-common 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (common files)
ii qemu-system-data 1:3.1+dfsg-8+deb10u4 all QEMU full system emulation (data files)
ii qemu-system-gui 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (user interface and audio support)
ii qemu-system-mips 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (mips)
ii qemu-system-misc 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (miscellaneous)
ii qemu-system-ppc 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (ppc)
ii qemu-system-sparc 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (sparc)
ii qemu-system-x86 1:3.1+dfsg-8+deb10u4 amd64 QEMU full system emulation binaries (x86)
ii qemu-utils 1:3.1+dfsg-8+deb10u4 amd64 QEMU utilities

Current uname -a:
Linux laptop-dev 5.4.0-0.bpo.3-amd64 #1 SMP Debian 5.4.13-1~bpo10+1 (2020-02-07) x86_64 GNU/Linux

Current CPU info:
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 158
model name : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
stepping : 10
microcode : 0xca
cpu MHz : 800.109
cache size : 9216 KB
physical id : 0
siblings : 12
core id : 0
cpu cores : 6
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:

$ cat /proc/meminfo
MemTotal: 16098356 kB
MemFree: 2292720 kB
MemAvailable: 12323616 kB

Steps to reproduce:
1) Create a base o/s image in .qcow2 or raw format. I am using a Debian 10 (buster) image, and my images are created using packer 1.5.5.
2) Verify that the base image in .qcow2 format boots correctly in virt-manager when attached to a new VM.
3) Convert the image to vmdk using the following command:
qemu-img convert -f qcow2 -O vmdk -o adapter_type=lsilogic,subformat=streamOptimized,hwversion=6 <image name>.qcow2 <image name>.vmdk
4) Create a new VM using the newly-created .vmdk, being sure to make the storage adapter SCSI
Result: The image, on boot, will display many disk read errors. It will boot, but will start in read-only mode.

This same image will also not boot correctly in ESXi or VirtualBox. In any of the three hypervisor environments, the bootloader menu (grub2) shows up correctly, but the machine will fail to boot correctly.

I reviewed the vmdk header, and the output is here:

dd if=base.vmdk of=output-vm-disk1.descriptor bs=1 skip=512 count=1024

$ cat output-vm-disk1.descriptor
# Disk DescriptorFile
version=1
CID=2120740c
parentCID=ffffffff
createType="streamOptimized"

# Extent description
RW 61440000 SPARSE "base.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "6"
ddb.geometry.cylinders = "3824"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

Removing the "subformat=streamOptimized" argument from the qemu-img conversion command results in a working, albeit much larger image, with no disk read errors.

[Thomas Huth (th-huth)]

The QEMU project is currently moving its bug tracking to another system.
For this we need to know which bugs are still valid and which could be
closed already. Thus we are setting older bugs to "Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]