py-up/down-exception finds wrong comint/py-shell buffer

[Richard Everson]

Hi,

Thanks for the great work on python-mode.el, especially integration of IPython into the main python-mode. I find one of the most useful functions from the I/Python shell py-up-exception, which unfortunately works only with the Python inferior shell and not IPython or the dedicated shells.

The IPython exception reporting is differently formatted. Changing py-traceback-line-re to "\\(^[^\t >].+?\\.py\\).*\n.*\n.*\n-+> \\([0-9]+\\)" gets the exception matching right. However, py-up-exception always looks for the exception in the *Python* buffer; here's the current code:

(defun py-up-exception (&optional top)
  "Go to the previous line up in the traceback.

With \\[universal-argument] (programmatically, optional argument TOP)
jump to the top (outermost) exception in the exception stack."
  (interactive "P")
  (let* ((proc (get-process "Python"))
         (buffer (if proc "*Python*" py-output-buffer)))
    (if top
        (py-find-next-exception 'bob buffer 're-search-forward "Top")
      (py-find-next-exception 'bol buffer 're-search-backward "Top"))))

Hacking "Python" to "IPython" and *Python* to *IPython* makes py-up-exception work for IPython, but of course breaks it for Python, and neither works for dedicated py-shells. Just making buffer (current-buffer) works for the *first* exception, but py-up-exception is bound to C-c - python code buffers too, so it doesn't work for subsequent invocations. I can't provide a patch (sorry) because I don't understand the logic for getting hold of the correct comint buffer for the various py-shells.

It seems potentially complicated: maybe a solution would be to remember the last "active" py-shell or the last comint buffer from which py-up/down-exception was invoked?

Thanks again,

Richard.

[Andreas Roehler (a-roehler)]

Am 02.01.2012 11:58, schrieb Richard Everson:

Hi,

beside the whole bunch of bugs you pointed at, looks like a bug lives
already in

py-jump-to-exception, at

     (pop-to-buffer buffer)
     ;; Force Python mode

setting an Emacs lisp buffer here onto python-mode...

In order to fix things one-by-one, could you send me an example which
works with common Python shell?

Thanks,

Andreas

[Richard Everson (r-m-everson)]

Hi Andreas,

Here's a simple example in the attached demo1.py. To see the anticipated behaviour I started Python with C-c ! and then (pasting from the *Python* buffer):

Python 2.7.2 (default, Sep 3 2011, 01:14:13)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import demo1
>>> demo1.f()
Hello from f
Hello from g
Hello from h -- just before exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "demo1.py", line 4, in f
    g()
  File "demo1.py", line 8, in g
    h()
  File "demo1.py", line 12, in h
    1./0.
ZeroDivisionError: float division by zero
>>>

Now C-c - jumps to the exception on line 12 of demo1.py (actually in the minibuffer it says "Jumping to exception in file demo1.py on line 12", but the cursor ends up on line 13 -- I've seen this off-by-one reported in other bug reports). Pressing C-c - again (now in the demo1.py buffer), and again jumps to the lines 8 and 4 (well, 9 and 5). Successive C-c = allows you to go back down the stack.

A second demo is in demo2:

>>> import demo2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "demo2.py", line 2, in <module>
    import demo3
  File "demo3.py", line 4
    if a != 25
             ^
SyntaxError: invalid syntax
>>>

Now pressing C-c - jumps to the syntax error in demo3.py (off by one) and another C-c - takes you to the import statement in demo2.py (off by one).

Sadly IPython reports these two slightly differently:

Python 2.7.2 (default, Sep 3 2011, 01:14:13)
Type "copyright", "credits" or "license" for more information.

IPython 0.12 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX].
For more information, type 'help(pylab)'.
Welcome to rlcompleter2 0.98
for nice experiences hit <tab> multiple times

In [1]: import demo1

In [2]: demo1.f()
Hello from f
Hello from g
Hello from h -- just before exception
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
/Users/reverson/projects/edge/<ipython-input-2-1cebb13502c5> in <module>()
----> 1 demo1.f()

/Users/reverson/projects/edge/demo1.pyc in f()
      2 def f():
      3 print 'Hello from f'
----> 4 g()
      5
      6 def g():

/Users/reverson/projects/edge/demo1.pyc in g()
      6 def g():
      7 print 'Hello from g'
----> 8 h()
      9
     10 def h():

/Users/reverson/projects/edge/demo1.pyc in h()
     10 def h():
     11 print 'Hello from h -- just before exception'
---> 12 1./0.
     13
     14

ZeroDivisionError: float division by zero

(The pylab and rlcompleter2 messages in the header are irrelevant and don't affect this.) The hack I mentioned in the bug report allows C-c - and C-= to work for these sorts of exceptions, but the regexp doesn't match the syntax error reportin...

[Richard Everson (r-m-everson)]

Hi Andreas,

Thanks for the fix; it's much better. I think there is a small typo, which was causing py-down-exception not to work (and also a setq as well as a defvar for py-ipython-exception-line-re). Here's a diff against bar revno 766:

$ bzr diff
=== modified file 'python-mode.el'
--- python-mode.el 2012-01-06 13:07:11 +0000
+++ python-mode.el 2012-01-07 21:01:52 +0000
@@ -891,7 +891,6 @@
 (defvar py-ipython-traceback-line-re
   "^\\([^ \t>]+>[^0-9]+\\)\\([0-9]+\\)"
   "Regular expression that describes tracebacks.")
-(setq py-ipython-traceback-line-re "^\\([^ \t>]+>[^0-9]+\\)\\([0-9]+\\)")

 (defconst py-assignment-re "\\<\\w+\\>[ \t]*\\(=\\|+=\\|*=\\|%=\\|&=\\|^=\\|<<=\\|-=\\|/=\\|**=\\||=\\|>>=\\|//=\\)"
   "If looking at the beginning of an assignment. ")
@@ -6050,7 +6049,7 @@
 BOTTOM), jump to the bottom (innermost) exception in the exception
 stack."
   (interactive "P")
- (py-find-next-exception-prepare 'down (when (eq 4 (prefix-numeric-value top)) "BOTTOM")))
+ (py-find-next-exception-prepare 'down (when (eq 4 (prefix-numeric-value bottom)) "BOTTOM")))

 (defun py-up-exception (&optional top)
   "Go to the previous line up in the traceback.

Most things work well, there are couple of strange things.

First, if you repeatedly press C-c - to go up the exception stack. With regular Python, the cursor ends up jumping to line 1 of the file with the exception and in the minibuffer it says: "Jumping to exception in file <stdin> on line 1"; it might be better just to stay at the outermost frame in the stack as line 1 might be a long way away. With IPython the behaviour is a bit worse because the cursor in the *IPython* buffer goes to the lines like

----> 1 demo1.f()

(This is for the demo1.py file I sent before and attached again.) And Emacs opens a new buffer called "---->" which is a bit disconcerting. It's worse if you use the universal argument to C-c =

For the same file I get a new buffer "----->" associated with the file "/Users/reverson/projects/edge/<ipython-input-1-b03658927958> in <module>()" which I guess comes from the IPython traceback lines:

In [5]: demo1.f()
Hello from f
Hello from g
Hello from h -- just before exception
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
/Users/reverson/projects/edge/<ipython-input-5-1cebb13502c5> in <module>()
----> 1 demo1.f()

Secondly, the SyntaxError exceptions in the IPython buffer don't get matched. Pressing C-c - does nothing with the following:

In [1]: import demo2
  File "demo3.py", line 4
    if a != 25
              ^
SyntaxError: invalid syntax

I'm afraid I don't understand exactly what you've done for the buffer switching and the way the string matching is now supposed to work, so I haven't been able to come up with a fix. Sorry.

One thing that would be very useful would be to limit the searching for exception reporting lines to the *last* set of exceptions. If you have a long Python or IPython session there will probably be several exceptions and pressing C-c - repeatedly, not only goes up the stack for the last exception, but then revisits the ...

[Andreas Roehler (a-roehler)]

Am 07.01.2012 22:57, schrieb Richard Everson:

made two new reports out of this

thanks,

Andreas