Unit tests can fail if python guestfs is installed in the test environment

[melanie witt]

Ran into this downstream [1] where a unit test failed because it actually tried to call libvirt, which of course failed:

libguestfs: error: could not create appliance through libvirt.

Try running qemu directly without libvirt using this environment variable:
export LIBGUESTFS_BACKEND=direct

Original error from libvirt: internal error: qemu unexpectedly closed the monitor: 2022-10-22T13:20:42.059709Z qemu-kvm: -blockdev {"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-2-storage","backing":null}: Image is not in qcow2 format [code=1 int1=-1]
{5} nova.tests.unit.virt.disk.test_api.APITestCase.test_can_resize_need_fs_type_specified [191.116248s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/home/zuul/src/code.engineering.redhat.com/nova/nova/tests/unit/virt/disk/test_api.py", line 47, in test_can_resize_need_fs_type_specified
        self.assertFalse(api.is_image_extendable(image))
      File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/api.py", line 190, in is_image_extendable
        fs.setup(mount=False)
      File "/home/zuul/src/code.engineering.redhat.com/nova/nova/virt/disk/vfs/guestfs.py", line 235, in setup
        self.handle.launch()
      File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 190, in doit
        result = proxy_call(self._autowrap, f, *args, **kwargs)
      File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 148, in proxy_call
        rv = execute(f, *args, **kwargs)
      File "/usr/lib/python3.9/site-packages/eventlet/tpool.py", line 121, in execute
        rv = e.wait()
      File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait
        result = hub.switch()
      File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch
        return self.greenlet.switch()
      File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 365, in run
        self.wait(sleep_time)
      File "/usr/lib/python3.9/site-packages/eventlet/hubs/poll.py", line 80, in wait
        presult = self.do_poll(seconds)
      File "/usr/lib/python3.9/site-packages/eventlet/hubs/epolls.py", line 31, in do_poll
        return self.poll.poll(seconds)
      File "/usr/lib/python3.9/site-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler
        raise TimeoutException()
    fixtures._fixtures.timeout.TimeoutException

That ^ was with libguestfs 1.46.1.

Repro steps (not identical traceback but close):

1. Make sure you have libguestfs header files

$ sudo apt install libguestfs-dev

2. Enter the unit test venv

$ source .tox/py38/bin/activate

3. Install the libguestfs python bindings into the venv. This has to be done by tarball [2]. Note that the bindings were not built for python 3 until version 1.40 [3]

$ pip install http://libguestfs.org/download/python/guestfs-1.40.2.tar.gz

4. Run the unit test

$ tox -epy38 test_can_resize_need_fs_type_specified

5. The test should fail

The code in nova will call libvirt if guestfs is installed [4] and if it is not installed, it will emit a warning and move on [5]. In my repro environment it didn't get as far as a call to libvirt though.

Upstream the test doesn't fail because neither libvirt nor guestfs are installed in the CI environment for unit test jobs.

A unit test shouldn't be able to call out to real guestfs or libvirt and it wasn't mocked out properly.

We can address it by adding mocking to the unit test and create a fixture that will poison future imports by unit or func tests.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2135856
[2] https://libguestfs.org/guestfs-python.3.html#using-python-bindings-in-a-virtualenv
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1627964#c15
[4] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/vfs/guestfs.py#L66-L70
[5] https://github.com/openstack/nova/blob/b1958b7cfa6b8aca5b76b3f133627bb733d29f00/nova/virt/disk/api.py#L189-L201

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/862769

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/862769
Committed: https://opendev.org/openstack/nova/commit/ecb11043e9c2a798950d63b55cd08684b1d77c4a
Submitter: "Zuul (22348)"
Branch: master

commit ecb11043e9c2a798950d63b55cd08684b1d77c4a
Author: melanie witt <email address hidden>
Date: Thu Oct 27 01:43:55 2022 +0000

    Add mock to avoid loading guestfs in unit test

    We recently discovered that when the perfect conditions are present
    where:

      * libguestfs-dev/el and guestfs python bindings are installed

    and

      * unit tests are not being run in a venv or guestfs python bindings
        are installed in the tox venv

    the test will end up loading the guestfs module and try to call the
    real guestfs and possibly libvirt and fail because of it.

    Our unit tests shouldn't be loading modules like guestfs, so this adds
    proper mocking to the test along with a poison fixture that will
    prevent future accidental imports of such modules.

    Closes-Bug: #1994913

    Change-Id: I676ee1fd33cf053681a07448759c28f0f2ad79d1

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 27.0.0.0rc1 release candidate.