Eventlet seems to be incompatible with multiprocessing/pool.py

[Joshua Harlow]

The internals of multiprocessing/pool.py seem to use a thread attribute that doesn't exist when eventlet monkey patches the threading module. The error that I am seeing is the following:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/y/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/home/y/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/y/lib/python2.6/multiprocessing/pool.py", line 221, in _handle_tasks
    if thread._state:
AttributeError: '_MainThread' object has no attribute '_state'

Likely the eventlet monkey patching does not patch this attribute (is this attribute is a hidden internal of the threading module?). We likely either need to figure out a different pool and use that or pretty clearly document this case and create a threaded impl that will work with eventlet.

Easy way to hit this:

import eventlet
eventlet.monkey_patch(thread=True)

Then try to use the threaded engine.

[Joshua Harlow (harlowja)]

Upstream eventlet issue @ https://bitbucket.org/eventlet/eventlet/issue/156

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/46603
Committed: http://github.com/stackforge/taskflow/commit/86c60dfa60496eaccf16dc50d13ac39f0cae451b
Submitter: Jenkins
Branch: master

commit 86c60dfa60496eaccf16dc50d13ac39f0cae451b
Author: Joshua Harlow <email address hidden>
Date: Fri Sep 13 20:21:03 2013 -0700

    Use executors instead of pools

    A executor is a more generic concept than a pool and
    also seems to work better with eventlet. This is also
    a more supported model of performing concurrent operations
    than using the mostly unknown multiprocessing thread
    pool implementation.

    Fixes: bug 1225275

    Change-Id: I09e9a9000bc88cc57d51342b83b31f97790a62e9