Memory leak in unit tests

[Oleg Bondarev]

tests.unit.agent.ovsdb.native.test_connection.TestOVSNativeConnection calls Connection.start() which starts a daemon with a while True loop full of mocks. mock._CallList of those mocks start to grow very quick and finally eat all available memory.

mem_top output during unit tests run:

refs:
18118 <class 'mock.mock._CallList'> [call(1),
 call().get_nowait(),
 call().get_nowait().do_commit(),
 call().get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
 call().task_do
18117 <class 'mock.mock._CallList'> [call.get_nowait(),
 call.get_nowait().do_commit(),
 call.get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
 call.task_done(),
 call.get_no
17990 <class 'mock.mock._CallList'> [call(1),
 call().get_nowait(),
 call().get_nowait().do_commit(),
 call().get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
 call().task_do
17989 <class 'mock.mock._CallList'> [call.get_nowait(),
 call.get_nowait().do_commit(),
 call.get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
 call.task_done(),
 call.get_no
13592 <class 'mock.mock._CallList'> [call(),
 call().fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
 call().timer_wait(<Mock id='139952614993680'>),
 call().block(),
 call().fd_wait(<MagicMock name=
13591 <class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
 call.timer_wait(<Mock id='139952614993680'>),
 call.block(),
 call.fd_wait(<MagicMock name='TransactionQueue
13591 <class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
 call.timer_wait(<Mock id='139952614993680'>),
 call.block(),
 call.fd_wait(<MagicMock name='TransactionQueue
13588 <class 'mock.mock._CallList'> [call(),
 call().do_commit(),
 call().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
 call(),
 call().do_commit(),
 call().results.put(<MagicMock nam
13496 <class 'mock.mock._CallList'> [call(),
 call().fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
 call().timer_wait(<Mock id='139952687478800'>),
 call().block(),
 call().fd_wait(<MagicMock name=
13495 <class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
 call.timer_wait(<Mock id='139952687478800'>),
 call.block(),
 call.fd_wait(<MagicMock name='TransactionQueue
13495 <class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
 call.timer_wait(<Mock id='139952687478800'>),
 call.block(),
 call.fd_wait(<MagicMock name='TransactionQueue
13492 <class 'mock.mock._CallList'> [call(),
 call().do_commit(),
 call().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
 call(),
 call().do_commit(),
 call().results.put(<MagicMock nam
11550 <type 'dict'> {'keystoneclient.service_catalog': <module 'keystoneclient.service_catalog' from '/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/keystoneclient/service_catalog.pyc'>, 'oslo_messaging.r
9061 <class 'mock.mock._CallList'> [call(<Mock id='139952630613712'>, <Mock name='get_schema_helper()' id='139952630616016'>),
 call().wait(<MagicMock name='Poller()' id='139952641693968'>),
 call().run(),
 call().wait(<MagicMock name=
9060 <class 'mock.mock._CallList'> [call.wait(<MagicMock name='Poller()' id='139952641693968'>),
 call.run(),
 call.wait(<MagicMock name='Poller()' id='139952641693968'>),
 call.run(),
 call.wait(<MagicMock name='Poller()' id='13995264
9060 <class 'mock.mock._CallList'> [call.wait(<MagicMock name='Poller()' id='139952641693968'>),
 call.run(),
 call.wait(<MagicMock name='Poller()' id='139952641693968'>),
 call.run(),
 call.wait(<MagicMock name='Poller()' id='13995264
9059 <class 'mock.mock._CallList'> [call.do_commit(),
 call.results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
 call.do_commit(),
 call.results.put(<MagicMock name='TransactionQueue().get_
9059 <class 'mock.mock._CallList'> [call.do_commit(),
 call.results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
 call.do_commit(),
 call.results.put(<MagicMock name='TransactionQueue().get_
9059 <class 'mock.mock._CallList'> [call.get_nowait(),
 call.task_done(),
 call.get_nowait(),
 call.task_done(),
 call.get_nowait(),
 call.task_done(),
 call.get_nowait(),
 call.task_done(),
 call.get_nowait(),
 call.task_done(),
 call
8997 <class 'mock.mock._CallList'> [call(<Mock id='139952686333712'>, <Mock name='get_schema_helper()' id='139952686333840'>),
 call().wait(<MagicMock name='Poller()' id='139952689107984'>),
 call().run(),
 call().wait(<MagicMock name=

types:
460418 <class 'mock.mock._Call'>
79091 <type 'tuple'>
47269 <type 'function'>
45542 <type 'dict'>
30758 <type 'list'>
14696 <type 'weakref'>
8601 <type 'set'>
6579 <type 'cell'>
5639 <type 'type'>
4940 <class 'mock.mock.MagicProxy'>
3858 <type 'getset_descriptor'>
3291 <class 'unittest.suite.TestSuite'>
3275 <type 'itertools.count'>
3267 <class 'unittest2.case._TypeEqualityDict'>
2439 <type 'module'>
2304 <class '_weakrefset.WeakSet'>
2222 <type 'instancemethod'>
2219 <type 'builtin_function_or_method'>
1869 <type 'property'>
1424 <type 'wrapper_descriptor'>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/333827

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/333827
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a5d19b41a57141ef51208b8a821bb7d2e0b6c159
Submitter: Jenkins
Branch: master

commit a5d19b41a57141ef51208b8a821bb7d2e0b6c159
Author: Oleg Bondarev <email address hidden>
Date: Fri Jun 24 11:42:52 2016 +0300

    Mock threading.Thread to prevent daemon creation by unit tests

    tests.unit.agent.ovsdb.native.test_connection.TestOVSNativeConnection
    calls Connection.start() which starts a daemon with a while True loop
    full of mocks. mock._CallList of those mocks start to grow very
    quick and finally eat all available memory.

    Closes-Bug: #1595878
    Change-Id: Ie053a2248925ce5bb960207c16c23b261d1d458c

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 9.0.0.0b2 development milestone.