IPython - Enhanced Interactive Python

Make IPython work with multiprocessing

Reported by Gael Varoquaux on 2009-01-08
4
Affects Status Importance Assigned to Milestone
IPython
Confirmed
Medium
Unassigned

Bug Description

Because the '.__module__' attribute of functions defined interactively in IPython is set to a virtual module non existant on the disk, distributing these functions with multiprocessing does not work:

resting ~ $ ipython
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import multiprocessing as mp

In [2]: p = mp.Pool(2)

In [3]: def foo(x):
   ...: return 2*x
   ...:

In [4]: p.map(foo, range(4))
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.5/threading.py", line 446, in run
    self.__target(*self.__args, **self.__kwargs)
  File "build/bdist.linux-i686/egg/multiprocessing/pool.py", line 225, in _handle_tasks
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

Changed in ipython:
importance: Undecided → Medium
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers