TestHeartbeatUDP.test_check intermittently fails tox tests

[Stephen Balukoff]

I've been noticing this intermittent error occurring with both the py27 and py34 tox environments, both running locally and within the gerrit jenkins system:

======================================================================
FAIL: octavia.tests.unit.amphorae.drivers.health.test_heartbeat_udp.TestHeartbeatUDP.test_check
tags: worker-1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sr/src/octavia/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "octavia/tests/unit/amphorae/drivers/health/test_heartbeat_udp.py", line 107, in test_check
    self.stats_update.update_stats.assert_called_once_with({'id': 1})
  File "/home/sr/src/octavia/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 947, in assert_called_once_with
    raise AssertionError(msg)
AssertionError: Expected 'update_stats' to be called once. Called 0 times.

To reproduce: Just run tox in a loop for a while and capture the output. :)

[Michael Johnson (johnsom)]

Can you include the branch and commit number you are running that reports this error?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/274294
Committed: https://git.openstack.org/cgit/openstack/octavia/commit/?id=54e81137b74260c65011b5408e9190f7cb7ecbf2
Submitter: Jenkins
Branch: master

commit 54e81137b74260c65011b5408e9190f7cb7ecbf2
Author: German Eichberger <email address hidden>
Date: Fri Jan 29 15:54:28 2016 -0800

    Fixes intermittend udp heartbeat test failure

    The test assumes that an asynchronous task finishes in time
    to trigger a mock. This is not always the case so I acdded
    code to shutdown the executor which waits until all concurrent
    tasks are finished.

    Change-Id: Iafadbc3f661431b0acb2390a575d1fb217b9aaa8
    Closes-Bug: #1539439