cPickle works in standard interpreter, but not in IPython

[reckoner]

for example:

In IPython:

>> class Mylist(list):
       def __init__(self,x=[]):
           list.__init__(self,x)

>> from cPickle import dumps
>> w=Mylist([1,2,3])
>> dumps(w)

PicklingError: Can't pickle <class '__main__.Mylist'>: attribute
lookup __main__.Mylist failed

However, using the standard Python interpreter:

Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from cPickle import dumps
>>> class Mylist(list):
... def __init__(self,x=[]):
... list.__init__(self,x)
...
>>> w=Mylist([1,2,3])
>>> dumps(w)
'ccopy_reg\n_reconstructor\np1\n(c__main__\nMylist\np2\nc__builtin__\nlist\np3\n
(lp4\nI1\naI2\naI3\natRp5\n.'
>>>

[avdd (avdd)]

Has nothing to do with pickle, but the FakeModule that is registered as __main__:

$ python
Python 2.5.2 (r252:60911, Oct 5 2008, 19:24:49)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.modules[__name__].foo = 1
>>> foo
1
>>>

a$ ipython

In [1]: import sys

In [2]: sys.modules[__name__].foo = 1

In [3]: foo
---------------------------------------------------------------------------
NameError Traceback (most recent call last)

/home/avdd/<ipython console> in <module>()

NameError: name 'foo' is not defined

In [4]:

[M Hickford (hickford)]

I have problem that neither pickle nor cPickle work in ipython (but work in python proper).

In my case pickle.dump works but pickle.

This is Jaunty, Python 2.6

My error:
AttributeError: 'FakeModule' object has no attribute 'Dictionary'

(I have a custom class inherited from list called Dictionary)

[Ville M. Vainio (villemvainio)]

On Sat, Jul 18, 2009 at 10:36 PM, Matt
Hickford<email address hidden> wrote:
> I have problem that neither pickle nor cPickle work in ipython (but work
> in python proper).
>
> In my case pickle.dump works but pickle.

I think you'll note that pickle works when the class of the pickleable
object is not declared interactively, but rather imported from module.

--
Ville M. Vainio
http://tinyurl.com/vainio

[reckoner (reckoner)]

If you get the class using the %run magic instead of importing, you DO
get the pickle failure. I don't know if doing %run is the same as
defining a class interactively, though.

Any guidance appreciated.

thanks.

On Sat, Jul 18, 2009 at 1:11 PM, Ville M. Vainio<email address hidden> wrote:
> On Sat, Jul 18, 2009 at 10:36 PM, Matt
> Hickford<email address hidden> wrote:
>> I have problem that neither pickle nor cPickle work in ipython (but work
>> in python proper).
>>
>> In my case pickle.dump works but pickle.
>
> I think you'll note that pickle works when the class of the pickleable
> object is not declared interactively, but rather imported from module.
>
> --
> Ville M. Vainio
> http://tinyurl.com/vainio
>
> --
> cPickle works in standard interpreter, but not in IPython
> https://bugs.launchpad.net/bugs/363115
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in IPython - Enhanced Interactive Python: Confirmed
>
> Bug description:
>
> for example:
>
> In IPython:
>
>
>>> class Mylist(list):
>       def __init__(self,x=[]):
>           list.__init__(self,x)
>
>>> from cPickle import dumps
>>> w=Mylist([1,2,3])
>>> dumps(w)
>
> PicklingError: Can't pickle <class '__main__.Mylist'>: attribute
> lookup __main__.Mylist failed
>
> However, using the standard Python interpreter:
>
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from cPickle import dumps
>>>> class Mylist(list):
> ...         def __init__(self,x=[]):
> ...             list.__init__(self,x)
> ...
>>>> w=Mylist([1,2,3])
>>>> dumps(w)
> 'ccopy_reg\n_reconstructor\np1\n(c__main__\nMylist\np2\nc__builtin__\nlist\np3\n
> (lp4\nI1\naI2\naI3\natRp5\n.'
>>>>
>

[eteq (erik-tollerud)]

I've encountered this in the past - I always just define any class I want to pickle in a file and then reload the module as necessary instead of pickling outputs from run or interactively-defined objects. That seems safer anyway - otherwise if you restart the interpreter, the methods on the object are forgotten.

It seems to work fine if you pickle from a file and then load that file with an interactively-defined or defined in "run" version, though.

[Zak Stone (zstone)]

For me, a similar problem arises when I am simply unpickling dictionaries in ipython -- no custom classes involved.

[ViktigLemma (jorgsk)]

This happened to me when I moved my scripts and my picked files to new folders. However, when I simulated the data again and pickled it again, I could unpickle without any errors.