tests.alarm.test_rpc.TestRPCAlarmPartitionCoordination.test_ordination_allocate failed for timeout

[Liusheng]

http://logs.openstack.org/02/93702/2/check/gate-ceilometer-python27/e8ea708/console.html

Traceback (most recent call last):
  File "/home/jenkins/workspace/gate-ceilometer-python27/ceilometer/tests/alarm/test_rpc.py", line 235, in test_ordination_allocate
    self.coordinator_server.rpc.wait()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/oslo/messaging/server.py", line 143, in wait
    self._executor.wait()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/oslo/messaging/_executors/impl_eventlet.py", line 106, in wait
    self._thread.wait()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
    return self._exit_event.wait()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
    return hubs.get_hub().switch()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
    return self.greenlet.switch()
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 236, in run
    self.wait(sleep_time)
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/eventlet/hubs/poll.py", line 81, in wait
    sleep(seconds)
  File "/home/jenkins/workspace/gate-ceilometer-python27/.tox/py27/local/lib/python2.7/site-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler
    raise TimeoutException()
TimeoutException

[ZhiQiang Fan (aji-zqfan)]

another one: http://logs.openstack.org/69/83369/2/check/gate-ceilometer-python27/4e2715c/

[Mehdi Abaakouk (sileht)]

I have introduced the bug when I have updated the testr configuration

I think we can safely removed or increased the 'OS_TEST_TIMEOUT' in .testr.conf

[ZhiQiang Fan (aji-zqfan)]

there may be a further question, why 160s is not enough for our test?

[Anita Kuno (anteaya)]

message:"FAIL: tests.alarm.test_rpc.TestRPCAlarmPartitionCoordination.test_ordination_allocate"

https://review.openstack.org/#/c/98146/2/queries/1323524.yaml

[Matt Riedemann (mriedem)]

This is the change referred to in comment 2:

https://review.openstack.org/#/c/95199/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/98820

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/98820
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=f35a9a6b8c4158c7b060dcb9b3ced4367c46977d
Submitter: Jenkins
Branch: master

commit f35a9a6b8c4158c7b060dcb9b3ced4367c46977d
Author: Mehdi Abaakouk <email address hidden>
Date: Mon Jun 9 17:35:18 2014 +0200

    Removes per test testr timeout

    Because testr run tests in parallel and we use fake driver of
    oslo.messaging to make unittests and this one use time.sleep() monkey
    patched by eventlet, a test can take a lot of time, just because
    eventlet have decided to run other tests before instead of finish the
    started one.

    So, just remove the per test testr timeout, like we have before.

    Change-Id: I26f14937a16b23428f55fc7f42b4066b398c06a8
    Closes-bug: #1323524

[ZhiQiang Fan (aji-zqfan)]

after patch: Reenables the testr per test timeout Ifccef9040529ec1a0b21076ed17d91a4f9737096 is merged, this problem raises again, see http://logs.openstack.org/58/101558/1/check/gate-ceilometer-python27/de6a11d/testr_results.html.gz

[Mehdi Abaakouk (sileht)]

 @ZhiQiang Fan: yes this is the desired behavior, all ceilometer tests must run in less then 10 minutes if it take more, we have a issue.

We fixe many ceilometer unit tests timeout issues last week , the per timeout was to help to track which part of unit tests have issue instead of just reach the gate timeout than hide which test is broken.

One remaining issue is https://bugs.launchpad.net/ceilometer/+bug/1331453, you can use this one to recheck bug XXX.

[Boris Pavlovic (boris-42)]

This should be re opened

[Boris Pavlovic (boris-42)]

https://review.openstack.org/#/c/100239/ This patch failed with the similar reason