Comment 0 for bug 2071734

Revision history for this message
Arnaud Morin (arnaud-morin) wrote : Regression VMDK/qcow arbitrary file access

When fixing bug #2059809, a regression of the previous bug #1996188 has been introduced.

TLDR: nova is allowing back VMDK with wrong types and QCOW with backing files.

Long:

The following steps were used to reproduce on a Bobcat (2023.2) OpenStack + nova patches for bug #2059809 (not yet merge when writing this report)

Create a vmdk file:
$ qemu-img create -f vmdk disk-vmdk.vmdk 1M -o subformat=monolithicFlat
$ sed -i -r 's|disk-vmdk-flat.vmdk|/etc/hosts|' disk-vmdk.vmdk

Create a faulty qcow image:
$ qemu-img create -f qcow2 -F raw -b /etc/hosts disk-bf.qcow2 1M

Upload both images as raw (the default)
$ for i in disk-bf.qcow2 disk-vmdk.vmdk ; do openstack image create --file $i $i ; done

Boot an instance from those images:
$ openstack server create --flavor small --image disk-vmdk.vmdk --net public disk-vmdk.vmdk
$ openstack server create --flavor small --image disk-bf.qcow2 --net public disk-bf.qcow2

Results
=======
VMDK monolithicFlat
-------------------

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2615, in _build_and_run_instance
    self.driver.spawn(context, instance, image_meta,
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4388, in spawn
    created_instance_dir, created_disks = self._create_image(
                                          ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4790, in _create_image
    created_disks = self._create_and_inject_local_root(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4921, in _create_and_inject_local_root
    self._try_fetch_image_cache(backend, fetch_func, context,
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 10940, in _try_fetch_image_cache
    image.cache(fetch_func=fetch_func,
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/imagebackend.py", line 288, in cache
    self.create_image(fetch_func_sync, base, size,
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/imagebackend.py", line 615, in create_image
    copy_raw_image(base, self.path, size)
  File "/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 414, in inner
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/imagebackend.py", line 590, in copy_raw_image
    self.resize_image(size)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/imagebackend.py", line 621, in resize_image
    disk.extend(image, size)
  File "/usr/lib/python3/dist-packages/nova/virt/disk/api.py", line 128, in extend
    processutils.execute('qemu-img', 'resize', image.path, size)
  File "/usr/lib/python3/dist-packages/oslo_concurrency/processutils.py", line 438, in execute
    raise ProcessExecutionError(exit_code=_returncode,
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Command: qemu-img resize /var/lib/nova/instances/07b5907b-5efc-4fdf-8b15-4b47a820c2f8/disk 2147483648
Exit code: 1
Stdout: ''
Stderr: "qemu-img: Could not open '/var/lib/nova/instances/07b5907b-5efc-4fdf-8b15-4b47a820c2f8/disk': Could not open '/etc/hosts': Permission denied\n"

Qemu tried to read /etc/hosts. My system permissions prevented it, but nova did nothing about it: wrong.

QEMU Backing File
-----------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2615, in _build_and_run_instance
    self.driver.spawn(context, instance, image_meta,
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4415, in spawn
    self._create_guest_with_network(
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7785, in _create_guest_with_network
    with excutils.save_and_reraise_exception():
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7763, in _create_guest_with_network
    guest = self._create_guest(
            ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7702, in _create_guest
    guest.launch(pause=pause)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 167, in launch
    with excutils.save_and_reraise_exception():
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
    self.force_reraise()
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
    raise self.value
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 165, in launch
    return self._domain.createWithFlags(flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 193, in doit
    result = proxy_call(self._autowrap, f, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 151, in proxy_call
    rv = execute(f, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 132, in execute
    six.reraise(c, e, tb)
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 86, in tworker
    rv = meth(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1409, in createWithFlags
    raise libvirtError('virDomainCreateWithFlags() failed')
libvirt.libvirtError: internal error: cannot update AppArmor profile 'libvirt-6bd32822-2454-402a-9617-6ec66e0090f4'

In libvirtd journalctl:

Jul 02 20:22:44 compute-1 libvirtd[959438]: internal error: Child process (LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/virt-aa-helper -r -u libvirt-6bd32822-2454-402a-9617-6ec66e0090f4 -F /dev/net/tun) unexpected exit status 1: virt-aa-helper: error: /etc/hosts

Here it's apparmor that prevented the boot, but nova should have catched it: wrong

Expected results
----------------

Nova should raise an exception like it does previously.
E.G. for VMDK: nova.exception.ImageUnacceptable: Image xyz is unacceptable: Invalid VMDK create-type specified