Make IPython work with multiprocessing

Bug #315251 reported by Gael Varoquaux
4
Affects Status Importance Assigned to Milestone
IPython
Confirmed
Medium
Unassigned

Bug Description

Because the '.__module__' attribute of functions defined interactively in IPython is set to a virtual module non existant on the disk, distributing these functions with multiprocessing does not work:

resting ~ $ ipython
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import multiprocessing as mp

In [2]: p = mp.Pool(2)

In [3]: def foo(x):
   ...: return 2*x
   ...:

In [4]: p.map(foo, range(4))
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.5/threading.py", line 446, in run
    self.__target(*self.__args, **self.__kwargs)
  File "build/bdist.linux-i686/egg/multiprocessing/pool.py", line 225, in _handle_tasks
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Changed in ipython:
importance: Undecided → Medium
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.