config_drive unit tests (libvirt driver) aren't mocking genisoimage

[Augustina Ragwitz]

I was running unit tests on a bare bones vm that didn't have genisoimage installed and the test_rescue_config_drive test failed.

==============================
Failed 1 tests - output below:
==============================

nova.tests.unit.virt.libvirt.test_driver.LibvirtDriverTestCase.test_rescue_config_drive
---------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "nova/tests/unit/virt/libvirt/test_driver.py", line 16420, in test_rescue_config_drive
        instance, exists=lambda name: name != 'disk.config.rescue')
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "nova/tests/unit/virt/libvirt/test_driver.py", line 16374, in _test_rescue
        network_info, image_meta, rescue_password)
      File "nova/virt/libvirt/driver.py", line 2531, in rescue
        self._create_domain(xml, post_xml_callback=gen_confdrive)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__
        return _mock_self._mock_call(*args, **kwargs)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1128, in _mock_call
        ret_val = effect(*args, **kwargs)
      File "nova/tests/unit/virt/libvirt/test_driver.py", line 16368, in fake_create_domain
        post_xml_callback()
      File "nova/virt/libvirt/driver.py", line 3130, in _create_configdrive
        cdb.make_drive(config_drive_local_path)
      File "nova/virt/configdrive.py", line 143, in make_drive
        self._make_iso9660(path, tmpdir)
      File "nova/virt/configdrive.py", line 97, in _make_iso9660
        run_as_root=False)
      File "nova/utils.py", line 296, in execute
        return processutils.execute(*cmd, **kwargs)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 363, in execute
        env=env_variables)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 54, in __init__
        subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
      File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
        errread, errwrite)
      File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
        raise child_exception
    OSError: [Errno 2] No such file or directory

When I installed genisoimage, the test passed.

genisoimage is the default value for mkisofs_cmd (configurable). It's called in the _make_iso9660 method for creating an image. Besides the issue of shelling out to a process going beyond the scope of what a unit test should cover, this also creates a hard dependency on genisoimage.

Other areas in the code mock the call to genisoimage. This test should do something similar.
https://github.com/openstack/nova/blob/master/nova/tests/unit/test_configdrive2.py#L49

[Augustina Ragwitz (auggy)]

Also see: https://bugs.launchpad.net/nova/+bug/1629555
"Allow the usage of mkisofs instead of genisoimage"

[Augustina Ragwitz (auggy)]

Followed up with mdbooth and verified that this should be mocked, so marking as confirmed.

It was also suggested that we need to add functional test coverage that does call genisoimage. I'm working separately to figure out the best course of action there, but am leaning towards wanting the functional test coverage to be limited to just making sure it handles an ISO image rather than calling out to a specific program to generate it. I think we should depend on integration test coverage for actually calling out to specific utilities.

[Augustina Ragwitz (auggy)]

Functional test coverage is another issue and there's a bigger question of whether this should even be a configurable value. I need to gather more information but will propose a spec or bug regarding this issue once I have more info. In the meantime, this bug is just focused on mocking the call to genisoimage.

[Augustina Ragwitz (auggy)]

Found another one:
test_spawn_with_config_drive

nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_spawn_with_config_drive
-----------------------------------------------------------------------------------------

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    2016-10-05 23:36:46,652 WARNING [nova.virt.libvirt.firewall] Libvirt module could not be loaded. NWFilterFirewall will not work correctly.
    2016-10-05 23:36:46,653 INFO [os_brick.initiator.connectors.disco] Init DISCO connector
    2016-10-05 23:36:46,654 WARNING [os_brick.initiator.connectors.remotefs] Connection details not present. RemoteFsClient may not initialize properly.
    2016-10-05 23:36:46,655 INFO [nova.virt.libvirt.driver] Creating image
    2016-10-05 23:36:46,750 INFO [nova.virt.libvirt.driver] Using config drive
    2016-10-05 23:36:46,798 INFO [nova.virt.libvirt.driver] Creating config drive at /tmp/tmppwqIMN/tmpxHedSY/32dfcb37-5af1-552b-357c-be8c3aa38310/disk.config

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "nova/tests/unit/virt/libvirt/test_driver.py", line 10149, in test_spawn_with_config_drive
        image_meta, [], None)
      File "nova/virt/libvirt/driver.py", line 2593, in spawn
        post_xml_callback=gen_confdrive)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__
        return _mock_self._mock_call(*args, **kwargs)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1128, in _mock_call
        ret_val = effect(*args, **kwargs)
      File "nova/tests/unit/virt/libvirt/test_driver.py", line 10139, in fake_create_domain_and_network
        post_xml_callback()
      File "nova/virt/libvirt/driver.py", line 3130, in _create_configdrive
        cdb.make_drive(config_drive_local_path)
      File "nova/virt/configdrive.py", line 143, in make_drive
        self._make_iso9660(path, tmpdir)
      File "nova/virt/configdrive.py", line 97, in _make_iso9660
        run_as_root=False)
      File "nova/utils.py", line 296, in execute
        return processutils.execute(*cmd, **kwargs)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 363, in execute
        env=env_variables)
      File "/home/auggy/nova/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 54, in __init__
        subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
      File "/usr/lib/python2.7/subprocess.py", line 711, in __init...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/383524
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b90df7c9c7042d2a8f104f66220f81ecb2951597
Submitter: Jenkins
Branch: master

commit b90df7c9c7042d2a8f104f66220f81ecb2951597
Author: Diana Clarke <email address hidden>
Date: Thu Oct 6 21:52:03 2016 -0400

    Patch mkisofs calls

    The nova unit tests recently started to fail on systems lacking mkisofs
    (like mac osx). Skip these mkisofs calls by patching _make_iso9660.

    Change-Id: I350aafa878616f74df506c1bc9ee5f26ea06fe97
    Closes-Bug: #1630420

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/529284

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529284
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d67cfbc8ca60c7820883daa2d0917dedc0958d65
Submitter: Zuul
Branch: stable/newton

commit d67cfbc8ca60c7820883daa2d0917dedc0958d65
Author: Diana Clarke <email address hidden>
Date: Thu Oct 6 21:52:03 2016 -0400

    Patch mkisofs calls

    The nova unit tests recently started to fail on systems lacking mkisofs
    (like mac osx). Skip these mkisofs calls by patching _make_iso9660.

    Change-Id: I350aafa878616f74df506c1bc9ee5f26ea06fe97
    Closes-Bug: #1630420
    (cherry picked from commit b90df7c9c7042d2a8f104f66220f81ecb2951597)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 14.1.0 release.