EventletContextManagerSpawnTest test cases fail

[Lance Bragstad]

When working on adding log_handler to oslo.messaging as part of https://wiki.openstack.org/wiki/Oslo/GraduationStatus#oslo.messaging

I noticed failures with tests.test_executor.EventletContextManagerSpawnTest.test_normal_run that I can't recreate locally (using tox --recreate -e py27,pep8) .

The patch to graduate log_handler.py to oslo.messaging is here: https://review.openstack.org/#/c/74804/ but keeps failing on http://logs.openstack.org/04/74804/8/check/gate-oslo.messaging-python27/afe5f4a/console.html which seems like an unrelated failure, and when looking at http://logs.openstack.org/04/74804/8/check/gate-oslo.messaging-python27/afe5f4a/testr_results.html.gz it almost seems like a race condition with Eventlet. This is just a guess and I'm open to suggestions.

I've checked the other open reviews for oslo.messaging but it doesn't look like it is effecting other patches.

[Matt Riedemann (mriedem)]

Looks like it's just your patch...

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiRkFJTFxcOiB0ZXN0cy50ZXN0X2V4ZWN1dG9yLkV2ZW50bGV0Q29udGV4dE1hbmFnZXJTcGF3blRlc3QudGVzdF9ub3JtYWxfcnVuXCIgQU5EIGZpbGVuYW1lOmNvbnNvbGUuaHRtbCIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5MjkyMDc2MDUzM30=

[Lance Bragstad (lbragstad)]

Ok, figured it out.. I think:

https://review.openstack.org/#/c/74804/8..9/oslo/messaging/notify/log_handler.py

Apparently using CONF = cfg.CONF the way I was doing it leaded to the error. The py27 tests are passing now so Jenkins should be ok with it once it finishes.

[Ihar Hrachyshka (ihar-hrachyshka)]

It failed the same way for https://review.openstack.org/#/c/77273/
You can find test results at: http://logs.openstack.org/73/77273/4/check/gate-oslo.messaging-python26/f0feed9/

The reason is probably some other greenthreads are still running when eventlet.sleep(0) is issued, so an incorrect thread is switched to, making test run fail.

I don't know whether there's a way to wait for greenthreads to complete before proceeding with the test.

[Mark McLoughlin (markmc)]

Fix proposed here: https://review.openstack.org/#/c/82151/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/82151
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=d4c04255f429ae22a08600db19212178ee6156bf
Submitter: Jenkins
Branch: master

commit d4c04255f429ae22a08600db19212178ee6156bf
Author: Ihar Hrachyshka <email address hidden>
Date: Fri Mar 21 18:57:39 2014 +0100

    Fixed inconsistent EventletContextManagerSpawnTest failures

    The test spawns a greenthread and expects it to complete after
    eventlet.sleep(0) call. The problem is that if there are other
    greenthreads currently running, sleep(0) may switch to unexpected
    thread, which may result in test thread not being invoked at all.
    Instead of assuming no other green threads are running at the moment
    of test execution, explicitly wait() for the test thread.

    Change-Id: Id45307860733658638b74ed3a713493df8c1080d
    Closes-Bug: #1282706