Nova Unit Testing with older libvirt versions fails

[Christopher Lefelhocz]

When packaging and running unit tests with older versions of libvirt < 0.9.13, the unit tests will fail even though there is nothing wrong with the actual code. An example failure is:

14:01:03 pythonlogging:'': {{{
14:01:03 INFO [nova.network.driver] Loading network driver 'nova.network.linux_net'
14:01:03 INFO [nova.virt.libvirt.driver] Unable to use bulk domain list APIs, falling back to slow code path: 'module' object has no attribute 'VIR_CONNECT_LIST_DOMAINS_ACTIVE'
14:01:03 }}}
14:01:03
14:01:03 Traceback (most recent call last):
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/repositories//nova/tests/virt/libvirt/test_driver.py", line 2971, in test_list_instance_domains_filtering
14:01:03 doms = drvr._list_instance_domains()
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/repositories//nova/virt/libvirt/driver.py", line 872, in _list_instance_domains
14:01:03 alldoms = self._list_instance_domains_slow(only_running)
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/repositories//nova/virt/libvirt/driver.py", line 822, in _list_instance_domains_slow
14:01:03 if self._conn.numOfDomains() > 0:
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/virtualenv/lib/python2.6/site-packages/mox.py", line 1002, in __call__
14:01:03 expected_method = self._VerifyMethodCall()
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/virtualenv/lib/python2.6/site-packages/mox.py", line 1049, in _VerifyMethodCall
14:01:03 expected = self._PopNextMethod()
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/virtualenv/lib/python2.6/site-packages/mox.py", line 1035, in _PopNextMethod
14:01:03 raise UnexpectedMethodCallError(self, None)
14:01:03 UnexpectedMethodCallError: Unexpected method call Stub for <property object at 0x6a87208>.numOfDomains() -> None
14:01:03
14:01:03
14:01:03 ======================================================================
14:01:03 FAIL: nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase.test_list_instance_domains_fast
14:01:03 ----------------------------------------------------------------------
14:01:03 _StringException: Empty attachments:
14:01:03 stderr
14:01:03 stdout
14:01:03
14:01:03 pythonlogging:'': {{{INFO [nova.network.driver] Loading network driver 'nova.network.linux_net'}}}
14:01:03
14:01:03 Traceback (most recent call last):
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/repositories//nova/tests/virt/libvirt/test_driver.py", line 2839, in test_list_instance_domains_fast
14:01:03 doms = drvr._list_instance_domains_fast()
14:01:03 File "/home/jenkins/workspace/test-nova-rax-patches-change/repositories//nova/virt/libvirt/driver.py", line 812, in _list_instance_domains_fast
14:01:03 flags = libvirt.VIR_CONNECT_LIST_DOMAINS_ACTIVE
14:01:03 AttributeError: 'module' object has no attribute 'VIR_CONNECT_LIST_DOMAINS_ACTIVE'

[Davanum Srinivas (DIMS) (dims-v)]

Christopher, please see if this works for you

http://paste.openstack.org/show/86563/

(just replace libvirt with fakelibvirt in 2 spots)

[Christopher Lefelhocz (christopher-lefelhoc)]

The paste won't solve the problem completely as the code also has some issues where the *_fast function would fail, but the test code doesn't handle it properly.

I'm not sure why, but the following patch may be a better option. It should have come up here when I opened the bug.

https://review.openstack.org/#/c/106851/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/107101

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christopher Lefelhocz (<email address hidden>) on branch: master
Review: https://review.openstack.org/106851
Reason: Going to abandon in favor of https://review.openstack.org/107101.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/107101
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ee57c65fa26d0be473be9d5e55748abd6693ad0d
Submitter: Jenkins
Branch: master

commit ee57c65fa26d0be473be9d5e55748abd6693ad0d
Author: Daniel P. Berrange <email address hidden>
Date: Tue Jul 15 17:15:18 2014 +0100

    libvirt: fix recent test changes to work on libvirt < 0.9.13

    The newly used 'listAllDomains' method was only introduced in
    libvirt 0.9.13. Although the driver code was written to cope
    with older libvirt, the test suite code itself would fail
    when faced with such an old libvirt because it was trying to
    mock out methods which did not exist.

    The fix here recognises that since '_list_instance_domains_fast'
    and '_list_instance_domains_slow' both have explicit test cases,
    there is no point in mocking out the low level 'listAllDomains'
    method in all the other test cases. Instead we can mock out
    the much higher level '_list_instance_domains' which means we
    avoid any interaction with libvirt in these test cases. The
    opportunity is taken to convert the tests in question to use
    'mock' instead of 'mox' since it also simplifies the code.

    Finally in the test for '_list_instance_domains_fast' simply
    skip the test if libvirt is too old to have the required
    constants defined.

    Closes-bug: 1341729
    Change-Id: I554352ea14e6b3306176e7524502e005cebd8c16

[Corey Wright (coreywright)]

committed fix works for me

[Johannes Erdfelt (johannes.erdfelt)]

This appears to have regressed again:

======================================================================
FAIL: nova.tests.unit.virt.libvirt.test_host.HostTestCase.test_list_instance_domains_fallback
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{2015-01-28 09:38:04,551 INFO [nova.virt.libvirt.host] Unable to use bulk domain list APIs, falling back to slow code path: 'module' object has no attribute 'VIR_CONNECT_LIST_DOMAINS_ACTIVE'}}}

Traceback (most recent call last):
  File "/home/johannes/virtualenvs/migrations/local/lib/python2.7/site-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "nova/tests/unit/virt/libvirt/test_host.py", line 516, in test_list_instance_domains_fallback
    libvirt.VIR_CONNECT_LIST_DOMAINS_ACTIVE)
AttributeError: 'module' object has no attribute 'VIR_CONNECT_LIST_DOMAINS_ACTIVE'

This is running on Debian 7 which has libvirt 0.9.12.3