test_quota.py QuotaReserveSqlAlchemyTestCase fails to clean up changes to QUOTA_SYNC_FUNCTIONS

Bug #1329482 reported by Mike Bayer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Mike Bayer

Bug Description

In nova/tests/test_quota.py -> QuotaReserveSqlAlchemyTestCase.setUp(), alternate function definitions are swapped into nova.db.sqlalchemy.api.QUOTA_SYNC_FUNCTIONS, however they are not reverted in any corresponding tearDown() method. I'm guessing this isn't typically noticed as when using either nose or the "testr"-style tools, test_quota.py is run well after other tests which rely on these functions, such as those in nova/tests/api/ec2/test_cinder_cloud.py. However, I've been using py.test which has a different natural test ordering. The issue can be seen using Nose by running test_cinder_cloud after test_quota:

$ ../.venv/bin/nosetests -v nova/tests/test_quota.py nova/tests/api/ec2/test_cinder_cloud.py -x
nova.tests.test_quota.BaseResourceTestCase.test_no_flag ... ok
nova.tests.test_quota.BaseResourceTestCase.test_quota_no_project ... ok
nova.tests.test_quota.BaseResourceTestCase.test_quota_with_project ... ok
nova.tests.test_quota.BaseResourceTestCase.test_with_flag ... ok

    [ ... tests continue to run ... ]

nova.tests.test_quota.QuotaReserveSqlAlchemyTestCase.test_quota_reserve_until_refresh ... ok
nova.tests.api.ec2.test_cinder_cloud.CinderCloudTestCase.test_create_image ... /Users/classic/dev/redhat/.venv/lib/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:33: SAWarning: The IN-predicate on "instances.uuid" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance.
  return o[0](self, self.expr, op, *(other + o[1:]), **kwargs)
ERROR

======================================================================
ERROR: nova.tests.api.ec2.test_cinder_cloud.CinderCloudTestCase.test_create_image
----------------------------------------------------------------------
_StringException: pythonlogging:'': {{{
AUDIT [nova.service] Starting conductor node (version 2014.2)
INFO [nova.virt.driver] Loading compute driver 'nova.virt.fake.FakeDriver'
AUDIT [nova.service] Starting compute node (version 2014.2)
AUDIT [nova.compute.resource_tracker] Auditing locally available compute resources
AUDIT [nova.compute.resource_tracker] Free ram (MB): 7680
AUDIT [nova.compute.resource_tracker] Free disk (GB): 1028
AUDIT [nova.compute.resource_tracker] Free VCPUS: 1
AUDIT [nova.compute.resource_tracker] PCI stats: []
INFO [nova.compute.resource_tracker] Compute_service record created for 93851743013149aabf5b0a5492cef513:fake-mini
AUDIT [nova.service] Starting scheduler node (version 2014.2)
INFO [nova.network.driver] Loading network driver 'nova.network.linux_net'
AUDIT [nova.service] Starting network node (version 2014.2)
AUDIT [nova.service] Starting consoleauth node (version 2014.2)
INFO [nova.virt.driver] Loading compute driver 'nova.virt.fake.FakeDriver'
AUDIT [nova.service] Starting compute node (version 2014.2)
AUDIT [nova.compute.resource_tracker] Auditing locally available compute resources
AUDIT [nova.compute.resource_tracker] Free ram (MB): 7680
AUDIT [nova.compute.resource_tracker] Free disk (GB): 1028
AUDIT [nova.compute.resource_tracker] Free VCPUS: 1
AUDIT [nova.compute.resource_tracker] PCI stats: []
INFO [nova.compute.resource_tracker] Compute_service record created for d207a83c4c3f4b0ab622668b19210a10:fake-mini
WARNING [nova.service] Service killed that has no database entry
}}}

Traceback (most recent call last):
  File "/Users/classic/dev/redhat/nova/nova/tests/api/ec2/test_cinder_cloud.py", line 1019, in test_create_image
    ec2_instance_id = self._run_instance(**kwargs)
  File "/Users/classic/dev/redhat/nova/nova/tests/api/ec2/test_cinder_cloud.py", line 750, in _run_instance
    rv = self.cloud.run_instances(self.context, **kwargs)
  File "/Users/classic/dev/redhat/nova/nova/api/ec2/cloud.py", line 1352, in run_instances
    block_device_mapping=kwargs.get('block_device_mapping', {}))
  File "/Users/classic/dev/redhat/nova/nova/hooks.py", line 103, in inner
    rv = f(*args, **kwargs)
  File "/Users/classic/dev/redhat/nova/nova/compute/api.py", line 1338, in create
    legacy_bdm=legacy_bdm)
  File "/Users/classic/dev/redhat/nova/nova/compute/api.py", line 997, in _create_instance
    block_device_mapping)
  File "/Users/classic/dev/redhat/nova/nova/compute/api.py", line 818, in _provision_instances
    context, instance_type, min_count, max_count)
  File "/Users/classic/dev/redhat/nova/nova/compute/api.py", line 343, in _check_num_instances_quota
    cores=req_cores, ram=req_ram)
  File "/Users/classic/dev/redhat/nova/nova/quota.py", line 1301, in reserve
    user_id=user_id)
  File "/Users/classic/dev/redhat/nova/nova/quota.py", line 544, in reserve
    project_id=project_id, user_id=user_id)
  File "/Users/classic/dev/redhat/nova/nova/db/api.py", line 1143, in quota_reserve
    project_id=project_id, user_id=user_id)
  File "/Users/classic/dev/redhat/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
    return f(*args, **kwargs)
  File "/Users/classic/dev/redhat/nova/nova/db/sqlalchemy/api.py", line 202, in wrapped
    return f(*args, **kwargs)
  File "/Users/classic/dev/redhat/nova/nova/db/sqlalchemy/api.py", line 3130, in quota_reserve
    updates = sync(elevated, project_id, user_id, session)
  File "/Users/classic/dev/redhat/nova/nova/tests/test_quota.py", line 2101, in sync
    self.sync_called.add(res_name)
AttributeError: 'QuotaReserveSqlAlchemyTestCase' object has no attribute 'sync_called'

The symptom is cryptic here, but essentially the callables swapped in by this test refer to "self" as a QuotaReserveSqlAlchemyTestCase object, which has long since been torn down and no longer has the "sync_called" set associated with it.

I'd love to use mock.patch() for this kind of thing, but for the moment I'm going to submit a straightforward patch that restores QUOTA_SYNC_FUNCTIONS.

Mike Bayer (zzzeek)
Changed in nova:
assignee: nobody → Mike Bayer (zzzeek)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/99772

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/99772
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7168fcfe2d110eeca730e2ed966a5b5435b226b8
Submitter: Jenkins
Branch: master

commit 7168fcfe2d110eeca730e2ed966a5b5435b226b8
Author: Mike Bayer <email address hidden>
Date: Thu Jun 12 17:28:36 2014 -0400

    Ensure changes to api.QUOTA_SYNC_FUNCTIONS are restored.

    Add a local callable restore_sync_functions() that is
    established as an addCleanup() fixture callable,
    making use of a copy of the QUOTA_SYNC_FUNCTIONS
    dictionary to restore its previous contents in-place.

    Change-Id: I7fa69ea91e517121aba0ee22a71b0594591ae308
    Closes-Bug: #1329482

Changed in nova:
status: In Progress → Fix Committed
Changed in nova:
milestone: none → juno-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: juno-2 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.