Line entry edge case error

Bug #290981 reported by Stephan Peijnik on 2008-10-30
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
IPython
Confirmed
Low
Ville M. Vainio
ipython (Debian)
Fix Released
Unknown

Bug Description

This bug came in via bugs.debian.org and I am forwarding it to you as it seems to be an IPython issue. The Debian bug in question is http://bugs.debian.org/473018.
Could you please have a look at the bug report and comment on it? This problem seems to also affect IPython 0.9.1 which I am currently packaging for Debian.

The original problem description can be found below.

Regards,

Stephan

-- original problem description --
Interactively entering the following, allowing ipython
to automatically provide indentation:

def a(b):
  b()
 return True

causes an indentation error (as expected). I press "up"
to retrieve the history, and go back to add in the
additional space before "return". Pressing enter causes:

NameError: name 'e' is not defined

Weirder still: if True is replaced with 'q' and some value
is assigned to the variable q, ipython returns that value
as the next "out" result. Find transcript below.

The "call" b() appears to not matter (it can be replaced
with other code, for instance "b=b").

Reproduced. Seems like readline only gets "e" from the input line
'return True' when you edit it. I'll investigate.

On Thu, Oct 30, 2008 at 8:37 AM, Stephan Peijnik <email address hidden> wrote:
> Public bug reported:
>
> This bug came in via bugs.debian.org and I am forwarding it to you as it seems to be an IPython issue. The Debian bug in question is http://bugs.debian.org/473018.
> Could you please have a look at the bug report and comment on it? This problem seems to also affect IPython 0.9.1 which I am currently packaging for Debian.

--
Ville M. Vainio
http://tinyurl.com/vainio

Ville M. Vainio (villemvainio) wrote :

On Wed, Nov 5, 2008 at 12:01 PM, Ville M. Vainio <email address hidden> wrote:

> Reproduced. Seems like readline only gets "e" from the input line
> 'return True' when you edit it. I'll investigate.

It appears that the problem is all the way down in readline - we get
the "e" from raw_input(). It seems this can't be fixed in ipython
side, unfortunately.

--
Ville M. Vainio
http://tinyurl.com/vainio

Stephan Peijnik (speijnik) wrote :

OK, I'm thinking about marking this bug (in dbts) as "wontfix" then.

Changed in ipython:
status: Unknown → Confirmed
Stephan Peijnik (speijnik) wrote :

I know it's been a while, but I again tested against this bug and I'm not sure it's really readline-related.

Without doing a check on this I expect that ipython, and not readline, executes the python code entered. If that's the case it sure looks like ipython causing the problem, and not readline.

This is the result of another test of mine:

--begin test--

In [1]: def a(b):
   ...: q=5
   ...: return q
------------------------------------------------------------
IndentationError: unindent does not match any outer indentation level (<ipython console>, line 3)

In [2]: def a(b):
    q=5
   return q
---------------------------------------------------------------------------
NameError Traceback (most recent call last)

/home/sp/<ipython console> in <module>()

NameError: name 'q' is not defined

In [3]: def a(b):
    q=5
   return q
------------------------------------------------------------
IndentationError: unindent does not match any outer indentation level (<ipython console>, line 3)

In [6]: def a(b):
    q=5
   return q
---------------------------------------------------------------------------
NameError Traceback (most recent call last)

/home/sp/<ipython console> in <module>()

NameError: name 'q' is not defined
--end test--

Note how every command with an odd line number works correctly and how ones with an even line number do not work.
Also, I did copy&paste the complete sets of commands I entered, why does the fourth input end up at "In [6]"?

So again, I am suspecting an ipython-internal issue, rather than a readline one. Could someone please try to reproduce this?

-- Stephan

On Tue, Feb 24, 2009 at 1:28 PM, Stephan Peijnik <email address hidden> wrote:

> Without doing a check on this I expect that ipython, and not readline,
> executes the python code entered. If that's the case it sure looks like
> ipython causing the problem, and not readline.

Try "%hist -r" to see what IPython thinks you actually typed.

--
Ville M. Vainio
http://tinyurl.com/vainio

Stephan Peijnik (speijnik) wrote :

On Tue, 2009-02-24 at 13:24 +0000, Ville M. Vainio wrote:
> On Tue, Feb 24, 2009 at 1:28 PM, Stephan Peijnik <email address hidden>
> wrote:
>
> > Without doing a check on this I expect that ipython, and not readline,
> > executes the python code entered. If that's the case it sure looks like
> > ipython causing the problem, and not readline.
>
> Try "%hist -r" to see what IPython thinks you actually typed.

Okay, looking at this makes me think it's readline again, thanks for the
hint.

--begin: %hist -r output--
1:
def a(b):
    q=5
   return q
2: q
3: %hist -r
--end: %hist -r output--

-- Stephan

Stephan Peijnik (speijnik) wrote :

On Tue, 2009-02-24 at 13:24 +0000, Ville M. Vainio wrote:
> On Tue, Feb 24, 2009 at 1:28 PM, Stephan Peijnik <email address hidden>
> wrote:
>
> > Without doing a check on this I expect that ipython, and not readline,
> > executes the python code entered. If that's the case it sure looks like
> > ipython causing the problem, and not readline.
>
> Try "%hist -r" to see what IPython thinks you actually typed.

Oh, and yet another test:

--begin: test--
In [1]: def a(b):
   ...: q=5
   ...: return q
------------------------------------------------------------
IndentationError: unindent does not match any outer indentation level
(<ipython console>, line 3)

In [2]: %hist -r
Executing: def a(b):
    q=5
   return q
------------------------------------------------------------
IndentationError: unindent does not match any outer indentation level
(<ipython console>, line 3)
--end: test--

Now this looks even weirder than my previous test with '%hist -r' as you
suggested.

-- Stephan

Ville M. Vainio (villemvainio) wrote :

Ok, this makes me think it may be ipython bug after all.

I can't debug this properly now, but iplib.py has this snippet in raw_input:

                if self.has_readline: # and some config option is set?
                    try:
                        histlen = self.readline.get_current_history_length()
                        if histlen > 1:
                            newhist = self.input_hist_raw[-1].rstrip()
                            self.readline.remove_history_item(histlen-1)
                            self.readline.replace_history_item(histlen-2,
                                            newhist.encode(self.stdin_encoding))
                    except AttributeError:
                        pass # re{move,place}_history_item are new in 2.4.

Try disabling that code (if 0:). It's used to support multiline history editing, which may be buggy.

Stephan Peijnik (speijnik) wrote :

I disabled multiline history editing, as you suggested, but that doesn't change the result.

I am somewhat wondering why "%hist -r" executes the piece of code in its history again, as it shouldn't do that.
Right now I am wondering whether this bug isn't more severe than I initially thought. If %hist executes commands this could cause bad things to happen (think of a call to os.remove being executed again, after changing the working directory).

Ville M. Vainio (villemvainio) wrote :

On Fri, Feb 27, 2009 at 7:56 PM, Stephan Peijnik <email address hidden> wrote:

> I am somewhat wondering why "%hist -r" executes the piece of code in its history again, as it shouldn't do that.

It doesn't - it's just that readline is feeding our raw_input
characters that it shouldn't. You should run %hist -r (and %hist
without -r) to get a hint about what's happening.

> Right now I am wondering whether this bug isn't more severe than I initially thought. If %hist executes commands this could cause bad things to
happen (think of a call to os.remove being executed again, after
changing the working directory).

It's certainly a very serious issue - unfortunately it's something we
probably can't easily get rid of, since we are blindly relying on the
hope that we get sensible input from the terminal.

--
Ville M. Vainio
http://tinyurl.com/vainio

Brian Granger (ellisonbg) wrote :

Ville,

Have you decided that this is a readline issue? If so, can we close this ticket?

Changed in ipython:
assignee: nobody → villemvainio
importance: Undecided → Low
status: New → Confirmed
Changed in ipython (Debian):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.