Cannot step debugger in IPython 0.9.1/0.10 under Python 2.6.2/2.6.4: Pdb instance has no attribute 'curframe'

[Jon Olav Vik]

I cannot step into scripts with "run -d"; ipython halts before the
first line with "Pdb instance has no attribute 'curframe'". It fails
even when debugging very simple files and with no user-specific
customizations loaded, no .bash_profile and no PYTHONPATH, in a
pristine directory so none of my files should override any defaults.
"run" without -d option works, as does pdb.set_trace()1. Plain Python also works as usual.

I'm not sure when this problem arose, but it may have been after an
upgrade from the versions included with Ubuntu 8.04 (Python 2.5.2 and
IPython 0.8.1? wild guess).

Minimal example:
===============
$ cat > debug_me.py
x = 2 + 2
y = 3 + 3

$ ipython
/var/lib/python-support/python2.6/IPython/Magic.py:38:
DeprecationWarning: the sets module is deprecated
 from sets import Set
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: run -d debug_me.py
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)

.../<ipython console> in <module>()

/var/lib/python-support/python2.6/IPython/iplib.pyc in ipmagic(self, arg_s)
   951 else:
   952 magic_args = self.var_expand(magic_args,1)
--> 953 return fn(magic_args)
   954
   955 def ipalias(self,arg_s):

/var/lib/python-support/python2.6/IPython/Magic.pyc in magic_run(self,
parameter_s, runner)
  1613 maxtries = 10
  1614 bp = int(opts.get('b',[1])[0])
-> 1615 checkline = deb.checkline(filename,bp)
  1616 if not checkline:
  1617 for bp in range(bp+1,bp+maxtries+1):

/usr/lib/python2.6/pdb.pyc in checkline(self, filename, lineno)
   450 line or EOF). Warning: testing is not comprehensive.
   451 """
--> 452 line = linecache.getline(filename, lineno,
self.curframe.f_globals)
   453 if not line:
   454 print >>self.stdout, 'End of file'

AttributeError: Pdb instance has no attribute 'curframe'
===============

[Minjae (the-minjae)]

This error was also confirmed on a clean install of Ubuntu 9.04, IPython 0.10.bzr.r1163, and Python 2.6.2.

[Minjae (the-minjae)]

As shown above, the issue occurs when 'checkline' method of pdb is called upon. An immediate workaround is to install pydb and make ipython use pydb instead of pdb by calling ipython -pydb. This works because ipython with pydb defines 'checkline' using pydb's checkline method.

[Jon Olav Vik (jonovik)]

Minjae, thanks for the solution. For other newbies: I downloaded the package from http://sourceforge.net/projects/freshmeat_pydb/, e.g.

[jonvi@stallo-1 ~]$ wget http://sunet.dl.sourceforge.net/sourceforge/bashdb/pydb-1.26.tar.gz
[jonvi@stallo-1 ~]$ tar xvzf pydb-1.26.tar.gz

Since I'm just a regular user, I installed with
[jonvi@stallo-1 ~]$ cd pydb-1.26
[jonvi@stallo-1 pydb-1.26]$ ./configure --prefix=$HOME/usr/ && make && make install

Verifying installation (assuming ~/usr/bin is in $PATH):
[jonvi@stallo-1 ~]$ pydb
(Pydb) print 1+1
2
(Pydb) exit
Use exit() or Ctrl-D (i.e. EOF) to exit
<Ctrl-D>

(interestingly, exiting with exit() causes an error 8-)

My original example now works fine:

[jonvi@stallo-1 ~]$ cat > debug_me.py
x = 2 + 2
y = 3 + 3
<Ctrl-D>
[jonvi@stallo-1 ~]$ ipython -pydb
/global/apps/python/2.6.2/lib/python2.6/site-packages/ipython-0.9.1-py2.6.egg/IPython/Magic.py:38: DeprecationWarning: the sets module is deprecated
  from sets import Set
Python 2.6.2 (r262:71600, Apr 27 2009, 23:59:28)
IPython 0.9.1 -- An enhanced Interactive Python.
In [2]: run -d debug_me.py
No stack.
NOTE: Enter 'c' at the ipydb> prompt to start your script.
(<string>:1): <module>
ipydb> s
----------------Call level 7 > /home/jonvi/debug_me.py(1)<module>()
      0 y = 3 + 3
(/home/jonvi/debug_me.py:1): <module>
1 x = 2 + 2
ipydb> n
(/home/jonvi/debug_me.py:1): <module>
1 x = 2 + 2
ipydb>
(/home/jonvi/debug_me.py:2): <module>
2 y = 3 + 3
ipydb>
----------------Return from level 7 (<type 'NoneType'>)
--------------Return from level 6 (<type 'NoneType'>)

[Richard Musil (richard-musil)]

Immediate workaround suggested by #2 seems to be available (easily) only on linux. Anyone has idea how to workaround it on Windows?

[Jon Olav Vik (jonovik)]

Richard, could you specify which version you are using of Windows, Python and IPython? Do you get the exact same problem described above? I do not see this problem on my system, using the Enthough Python Distribution (EPD) with Python 2.5.2 and IPython 0.9.1 under Windows XP SP3.

Example: I put the following in c:\temp\temp.py
x = 2 + 2
y = 3 + 3

Then start IPython (with the "Pylab" shortcut installed with EPD).
In [3]: run -d temp.py
Breakpoint 1 at c:\temp\temp.py:1
NOTE: Enter 'c' at the ipdb> prompt to start your script.
> <string>(1)<module>()
ipdb> s
--Call--
> c:\temp\temp.py(1)<module>()
      0 y = 3 + 3
ipdb> n

...and this works fine.

However, the pydb solution also worked. I had already installed Mingw and Msys (www.mingw.org), downloaded the pydb tar.gz file, extracted it to c:\msys\1.0\home\jonvi, started msys, navigated to the pydb installation directory, and ran
$ ./configure
$ make
$ make install

I discovered that the HOME environment variable needed to be set before pydb would work (otherwise, I got KeyError: HOME). Fhe Windows command line (Start menu > Run > cmd), I did:
set HOME=c:\msys\1.0\home\jonvi
(you'll need to adjust this to your system).

Then, running
C:\Python25\Scripts\ipython.exe -pydb
worked fine:
In [3]: cd c:\temp
c:\temp
In [4]: run -d temp.py
No stack.
NOTE: Enter 'c' at the ipydb> prompt to start your script.
(<string>:1): <module>
ipydb> s
----------------Call level 7 > c:\temp\temp.py(1)<module>()
      0 y = 3 + 3
(c:\temp\temp.py:1): <module>
1 x = 2 + 2
ipydb> n
(c:\temp\temp.py:1): <module>
1 x = 2 + 2
ipydb>
(c:\temp\temp.py:2): <module>
2 y = 3 + 3
ipydb>
----------------Return from level 7 (<type 'NoneType'>)
--------------Return from level 6 (<type 'NoneType'>)

Hope this helps,
Jon Olav

[Richard Musil (richard-musil)]

ad #5) Sorry to miss that. I have moved recently to Python 2.6.2, indeed on 2.5 version there was no problem.

Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)]
IPython 0.10 -- An enhanced Interactive Python
Windows XP / SP2

[Jon Guyer (guyer)]

I'm afraid I don't see how pydb is even a workaround. It is at least an order of magnitude slower than pdb (probably more like two orders of magnitude), rendering it effectively unusable for debugging my computationally intensive code.

Is there a "don't be so slow" option I can set somewhere?

Mac OS X 10.5.8
Python 2.6.2 (r262:71600, Jun 13 2009, 08:58:56)
IPython 0.8.3.svn.r3001
pydb 1.26

[Jon Olav Vik (jonovik)]

#6: So, have you tried installing pydb? If that fails, please post the error message.

#7: On my Windows laptop with the example code below, pydb is about 2.5 times as slow as ipdb, using "%run -d" and "c" (continue, not single-stepping). That's about 170 and 60 times as slow as regular execution (as measured by %timeit), respectively. Example code in file "slow.py":
==
def slow(j):
 L = [0]
 for i in range(j):
  L.append(L[-1] + i)
slow(300000)
==
With pydb, "run -d slow" followed by "c" took about 35.6 s. With ipdb, it took 13.5 s. And %timeit slow(300000) reported 213 ms.

As for "don't be so slow" (with apologies if this is obvious):
* Use breakpoints (perhaps inside ad hoc "if" statements) to avoid single-stepping unproblematic parts of code.
* Use pydb.set_trace()? (Never tried that.)
* Are you using Numpy for the computationally intensive code?
* If you are, Cython (www.cython.org; available for Mac too) might speed up elementwise calculations. (Well, _debugging_ is actually more complicated with compiled code, but perhaps compiling the unproblematic portions would ease your debugging of the rest.)

Hope this helps,
JOV

[Richard Musil (richard-musil)]

#8: No, I did not install msys and I do not plan to do this either. I may sound like heretic, but on Windows I use solely MS VS to build extensions and I do not want to add "yet another toolchain" and mix them together just for the sake of this workaround. Also my personal experience with msys is rather love-hate relationship ;-).

[SebastienBarthelemy (barthelemy)]

Architecture: i386
DistroRelease: Ubuntu 9.10
NonfreeKernelModules: nvidia
Package: ipython 0.10-1
PackageArchitecture: all
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, user)
 LANG=fr_FR.UTF-8
ProcVersionSignature: Ubuntu 2.6.31-13.44-generic
Uname: Linux 2.6.31-13-generic i686
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare
XsessionErrors:
 (gnome-settings-daemon:4955): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (nautilus:5023): Eel-CRITICAL **: eel_preferences_get_boolean: assertion `preferences_is_initialized ()' failed
 (polkit-gnome-authentication-agent-1:5029): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (gnome-panel:4990): Gdk-WARNING **: /build/buildd/gtk+2.0-2.18.2/gdk/x11/gdkdrawable-x11.c:952 drawable is not a pixmap or window

[Greg Finney (gfinney)]

I've also replicated this with Python 2.6.2 , IPython 0.10 (egg distribution) on WinXP

Note that ipython seems to work fine with pdb when triggered by an exception (with the magic %pdb on)

[Jon Guyer (guyer)]

#8: I'm seeing a very different breakdown with my FiPy scripts on Mac OS X. Using time.time() embedded in the startup sequence for my script:

$ python r1082.py
 ...
startup time: 4.54922485352

----

$ python -m pdb r1082.py
(Pdb) c
 ...
startup time: 4.59463787079

----

$ ipython -pydb
In [1]: run -d -b 43 r1082.py
ipydb> c
 ...
startup time: 856.767014027

----

In contrast, using ipdb (which I didn't know about before; thank you!):

$ ipython
 ...
In [1]: %run r1082.py
ipdb> c
 ...
startup time: 4.54875421524

----

ipdb is a little clumsy to set up, but is quite usable. pydb is insane.

> As for "don't be so slow" (with apologies if this is obvious):
> * Use breakpoints (perhaps inside ad hoc "if" statements) to avoid single-stepping unproblematic parts of code.

My trouble, as above, is even getting to the code I want to single-step.

> * Use pydb.set_trace()? (Never tried that.)

Seems to work well, although if I'm going to do that, ipdb seems nicer.

> * Are you using Numpy for the computationally intensive code?

Amongst other things.

> * If you are, Cython (www.cython.org; available for Mac too) might speed up elementwise calculations.

I'm trying to *reduce* my debugging headaches! 8^)

Cython is certainly on our list of optimizations to explore, but is not the solution here. My immediate problem is not with the speed of my own code, nor with normal debugging of my code; it's this 'curframe' business and pydb that's causing me grief. Now that I know about ipdb, I can get on with things, thanks.

[dr who (dr-who)]

have this same error on python 2.6.4 and ipython 0.10 (amended topic to that effect).

/usr/lib/pymodules/python2.6/IPython/Magic.pyc in magic_run(self, parameter_s, runner, file_finder)
   1631 maxtries = 10
   1632 bp = int(opts.get('b',[1])[0])
-> 1633 checkline = deb.checkline(filename,bp)
   1634 if not checkline:
   1635 for bp in range(bp+1,bp+maxtries+1):

/usr/lib/python2.6/pdb.pyc in checkline(self, filename, lineno)
    450 line or EOF). Warning: testing is not comprehensive.
    451 """
--> 452 line = linecache.getline(filename, lineno, self.curframe.f_globals)
    453 if not line:
    454 print >>self.stdout, 'End of file'

AttributeError: Pdb instance has no attribute 'curframe'

[Jon Olav Vik (jonovik)]

In case anyone wants to hack away at this, here is a relevant diff (pdb.py from Python 2.6.2 vs 2.5.2):

http://svn.python.org/view/python/tags/r262/Lib/pdb.py?r1=60915&r2=71601

As Minjae pointed out, the error is in the checkline() method. Unfortunately, I don't have a Python 2.6 installation to hack, but maybe someone could test this in pdb.py?

Replace:
    def checkline(self, filename, lineno):
        """Check whether specified line seems to be executable.

        Return `lineno` if it is, 0 if not (e.g. a docstring, comment, blank
        line or EOF). Warning: testing is not comprehensive.
        """
        line = linecache.getline(filename, lineno, self.curframe.f_globals)
        if not line:

with:
    def checkline(self, filename, lineno):
        """Check whether specified line seems to be executable.

        Return `lineno` if it is, 0 if not (e.g. a docstring, comment, blank
        line or EOF). Warning: testing is not comprehensive.
        """
        try:
            line = linecache.getline(filename, lineno, self.curframe.f_globals)
        except AttributeError:
            line = 0
        if not line:

[dr who (dr-who)]

no, doesn't help really:

run -d ...
End of file
End of file
End of file
End of file
End of file
End of file
End of file
End of file
End of file
End of file
End of file
ERROR:
I failed to find a valid line to set a breakpoint
after trying up to line: 11.
Please set a valid breakpoint manually with the -b option.

and the code in question has valid lines starting from line 5 onwards.

[Minjae (the-minjae)]

This error is confirmed on Python 2.6.4, iPython 0.10 with Ubuntu 9.10.

[Minjae (the-minjae)]

I have a proposed solution, albeit rather barbarian.

The offending line (line 452 in pdb.py of Python 2.6):

       line = linecache.getline(filename, lineno, self.curframe.f_globals)

Replace the above line with the following:

        try:
            line = linecache.getline(filename, lineno, self.curframe.f_globals)
        except AttributeError:
            line = linecache.getline(filename, lineno)

Basically, the method in this madness is that when there is an error, replace it with Python 2.5 line, which does not contain lineno. This "solution" seems to do the trick for me. Please comment when this causes more problem.

[Minjae (the-minjae)]

Correction to above post:
"Python 2.5 line which does not contain lineno"

should read

the corresponding line in pdb.py from Python 2.5 which does not contain self.curframe.f_globals

Sorry for the inconvenience.

[dr who (dr-who)]

@minjae: yep, that does the trick for me as well. excellent! thx!

[Jon Olav Vik (jonovik)]

#18: Your fix works for IPython 0.10 under Python 2.6.4 on 32-bit Windows XP SP3. Thanks!

[kmdouglass (kyle-m-douglass)]

#18: Fix also works for IPython 0.10, Python 2.6.4, and 32-bit Ubuntu 9.10.

[Fernando Perez (fdo.perez)]

It would be great if someone could report this as a Python bug, which is what it really is: the checklines() method in pdb is broken, and this doesn't appear to have been reported.

I've had my bugs.python account messed up for a while so I can't report it myself right now, sorry.

[Fernando Perez (fdo.perez)]

Never mind: looking further, the bug is in our end... Stay tuned.

[Fernando Perez (fdo.perez)]

Fix committed here: http://bazaar.launchpad.net/~ipython-dev/ipython/0.10.1/revision/1214

Will be part of 0.10.1 release, but packagers (ubuntu and others) are welcome to include it now if they want to issue a quicker fix, because I'm not sure when 0.10.1 will be actually released.

[Mike Hearne (mhearne808)]

I'm using EPD6.0.0 with IPython 0.10 on Mac OS 10.5.8. I'm experiencing this bug, so I thought I'd edit "IPython/Debugger.py" as shown in page pointed to by the above link. However, I can't find a method called "checkline" in that file in my distribution, so I'm guessing there have been other changes since 0.10. What's the easiest way for an end-user such as myself to download this patch?

[Jon Olav Vik (jonovik)]

#26: Mike, try _adding_ the checkline() method in IPython/Debugger.py. Clicking the IPython/Debugger.py link in
http://bazaar.launchpad.net/~ipython-dev/ipython/0.10.1/revision/1214
will display added lines in dark green.

The IPython Pdb class inherits from the standard Pdb class ("from pdb import Pdb as OldPdb", followed later by "class Pdb(OldPdb):"). Fernando's fix overrides the checkline() method, whereas Minjae's original solution modified the standard Pdb.checkline() to tolerate the way IPython does things.

[elpargo (jorge-vargas)]

Ahhh I just got bitten by this one. Looking at the milestone https://launchpad.net/ipython/+milestone/0.10.1 all bugs seems to be fixed. Will it be possible to release 0.10.1 to pypi?

I just discovered how useful ipython is (I finally sat down to learn it) but I can't live without pdb.

[Jon Olav Vik (jonovik)]

#28: This particular bug only requires that you replace the file
site-packages/ipython-0.10-py2.6.egg/IPython/Debugger.py
(or wherever it is on your system)
with the latest version at
http://bazaar.launchpad.net/~ipython-dev/ipython/0.10.1/download/head%3A/debugger.py-20080216095032-xb0is4a97lmosv2z-24/Debugger.py

The only change is the addition of a checkline() method, as explained in #27.

[Pavel Snopok (snopok)]

Thanks a lot! #29 works perfectly with py2.6.egg. Finally, I can debug in color!