curl driver (http) always says "No such file or directory"

[Richard Jones]

I have a remote server, on which an http disk image definitely exists. However the qemu curl block driver cannot open it. It always gives the bogus error:

CURL: Error opening file: Connection time-out
qemu-system-x86_64: -drive file=http://onuma/scratch/cirros-0.3.1-x86_64-disk.img,snapshot=on,cache=writeback,id=hd0,if=none: could not open disk image http://onuma/scratch/cirros-0.3.1-x86_64-disk.img: Could not open backing file: Could not open 'http://onuma/scratch/cirros-0.3.1-x86_64-disk.img': No such file or directory

On the server, I can see from the logs that qemu/curl is opening it:

192.168.0.175 - - [15/Jan/2014:21:25:37 +0000] "HEAD /scratch/cirros-0.3.1-x86_64-disk.img HTTP/1.1" 200 - "-" "-"
192.168.0.175 - - [15/Jan/2014:21:25:37 +0000] "GET /scratch/cirros-0.3.1-x86_64-disk.img HTTP/1.1" 206 264192 "-" "-"

I am using qemu & curl from git today.

curl: curl-7_34_0-177-gc7a76bb
qemu: for-anthony-839-g1cf892c

Here is the full command I am using:

http_proxy= \
LD_LIBRARY_PATH=/home/rjones/d/curl/lib/.libs \
LIBGUESTFS_BACKEND=direct \
LIBGUESTFS_HV=/home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64 \
guestfish -v --ro -a http://onuma/scratch/cirros-0.3.1-x86_64-disk.img run

The full output (includes qemu command itself) is:

libguestfs: launch: program=guestfish
libguestfs: launch: version=1.25.20fedora=21,release=1.fc21,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsoQctgE
libguestfs: launch: umask=0002
libguestfs: launch: euid=1000
libguestfs: command: run: /usr/bin/supermin-helper
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ -f checksum
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
supermin helper [00000ms] whitelist = (not specified)
supermin helper [00000ms] host_cpu = x86_64
supermin helper [00000ms] dtb_wildcard = (not specified)
supermin helper [00000ms] inputs:
supermin helper [00000ms] inputs[0] = /usr/lib64/guestfs/supermin.d
supermin helper [00000ms] outputs:
supermin helper [00000ms] kernel = (none)
supermin helper [00000ms] dtb = (none)
supermin helper [00000ms] initrd = (none)
supermin helper [00000ms] appliance = (none)
checking modpath /lib/modules/3.12.5-302.fc20.x86_64 is a directory
checking modpath /lib/modules/3.11.9-200.fc19.x86_64 is a directory
checking modpath /lib/modules/3.11.10-200.fc19.x86_64 is a directory
checking modpath /lib/modules/3.11.4-201.fc19.x86_64 is a directory
picked kernel vmlinuz-3.12.5-302.fc20.x86_64
supermin helper [00000ms] finished creating kernel
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/base.img.gz
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/daemon.img.gz
supermin helper [00000ms] visiting /usr/lib64/guestfs/supermin.d/hostfiles
supermin helper [00041ms] visiting /usr/lib64/guestfs/supermin.d/init.img
supermin helper [00041ms] visiting /usr/lib64/guestfs/supermin.d/udev-rules.img
supermin helper [00041ms] adding kernel modules
supermin helper [00064ms] finished creating appliance
libguestfs: checksum of existing appliance: 2017df18eaeee7c45b87139c9bd80be2216d655a1513322c47f58a7a3668cd1f
libguestfs: [00066ms] begin testing qemu features
libguestfs: command: run: /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -help
libguestfs: command: run: /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -version
libguestfs: qemu version 1.7
libguestfs: command: run: /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -machine accel=kvm:tcg
libguestfs: command: run: \ -device ?
libguestfs: [00127ms] finished testing qemu features
[00128ms] /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64 \
    -global virtio-blk-pci.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -machine accel=kvm:tcg \
    -cpu host,+kvmclock \
    -m 500 \
    -no-reboot \
    -no-hpet \
    -kernel /var/tmp/.guestfs-1000/kernel.19645 \
    -initrd /var/tmp/.guestfs-1000/initrd.19645 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=http://onuma/scratch/cirros-0.3.1-x86_64-disk.img,snapshot=on,cache=writeback,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-1000/root.19645,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -device sga \
    -chardev socket,path=/tmp/libguestfsoQctgE/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color'
CURL: Error opening file: Connection time-out
qemu-system-x86_64: -drive file=http://onuma/scratch/cirros-0.3.1-x86_64-disk.img,snapshot=on,cache=writeback,id=hd0,if=none: could not open disk image http://onuma/scratch/cirros-0.3.1-x86_64-disk.img: Could not open image: Invalid argument
libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
libguestfs: child_cleanup: 0x7f07bf6599b0: child process died
libguestfs: sending SIGTERM to process 19654
libguestfs: error: /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64 exited with error status 1, see debug messages above
libguestfs: error: guestfs_launch failed, see earlier error messages
libguestfs: closing guestfs handle 0x7f07bf6599b0 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsoQctgE

[Richard Jones (rjones-redhat)]

Turns out this is because of using "snapshot=on".

Simple reproducer:

./x86_64-softmmu/qemu-system-x86_64 -drive 'file=http://127.0.0.1/~rjones/cirros-0.3.1-x86_64-disk.img,if=virtio,snapshot=on'

[Richard Jones (rjones-redhat)]

I restored the original description. Please report a new bug.

[Thomas Huth (th-huth)]

This seems to be working now:

$ qemu-system-x86_64 -drive 'file=http://download.fedoraproject.org/pub/fedora-secondary/releases/29/Server/i386/iso/Fedora-Server-netinst-i386-29-1.2.iso,if=virtio,snapshot=on'

... boots from the ISO image, without error message. So I assume this has been fixed, and I am closing this bug now. If there is anything left to do, feel free to open it again.