IPython calls an object's __getattr__ incorrectly

[Angelo Bonet]

IPython seems to call an object's __getattr__ function inconsistently and at times when regular Python does not. According to the Python documentation on __getattr__: "if the attribute is found through the normal mechanism, __getattr__() is not called."

In the following examples, myobj has a bound method named "myfunc" defined and as well as a "__getattr__" method defined.

1) "myobj.myfunc()"
  In this example, IPython first calls myobj.__getattr__, before calling myfunc.

2) "myobj.myfunc ()"
  Introducing white-space between "myfunc" and "()" eliminates IPython's __getattr__ call

3) "myobj.myfunc( )"
  Introducing white-space between the parens "( )" eliminates IPython's __getattr__ call

4) "x = myobj.myfunc()"
  In this assignment statement, IPython does not call __getattr__.

In all of the above examples, regular Python *never* calls the object's __getattr__ routine, only myfunc is called.

[Robert Kern (robert-kern)]

What version of IPython are you using? With 0.8.3 and the trunk, I do not see this behavior. Can you show us the code that displays the behavior you see?

In [1]: !cat foo.py
IPython system call: cat foo.py
class foo(object):
    def __getattr__(self, attr):
        print 'Getting %r' % attr

    def myfunc(self):
        print 'myfunc()'

In [2]: import foo

In [3]: f = foo.foo()

In [4]: f.myfunc()
myfunc()

In [5]: f.myfunc ()
myfunc()

In [6]: f.myfunc( )
myfunc()

In [7]: x = f.myfunc()
myfunc()

In [8]: f.something
Getting 'something'

[Angelo Bonet (abonet)]

That's interesting. I'm running 0.8.4 w/Python 2.4.3. I've also experienced the same behavior with 0.7.2

Using your same example:

~/notes/ipython> ipython
Python 2.4.3 (#1, Jan 14 2008, 18:32:40)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.4 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: !cat foo.py
class foo(object):
    def __getattr__(self, attr):
        print 'Getting %r' % attr

    def myfunc(self):
        print 'myfunc()'

In [2]: import foo

In [3]: f = foo.foo()

In [4]: f.myfunc()
Getting 'myfunc()'
myfunc()

In [5]: f.myfunc ()
myfunc()

[Angelo Bonet (abonet)]

Stack-trace showing culprit of __getattr__ call:

In [1]: !cat foo.py

class foo(object):
    def __getattr__(self, attr):
        print 'Getting %r' % attr
        import traceback
        traceback.print_stack()

    def myfunc(self):
        print 'myfunc()'

In [2]: import foo

In [3]: f = foo.foo()

In [4]: f.myfunc()
Getting 'myfunc()'
  File "/home/ab/local/bin/ipython", line 27, in ?
    IPython.Shell.start().mainloop()
  File "/home/ab/local/pythonlib/IPython/Shell.py", line 84, in mainloop
    self.IP.mainloop(banner)
  File "/home/ab/local/pythonlib/IPython/iplib.py", line 1575, in mainloop
    self.interact(banner)
  File "/home/ab/local/pythonlib/IPython/iplib.py", line 1773, in interact
    line = self.raw_input(prompt,more)
  File "/home/ab/local/pythonlib/IPython/iplib.py", line 2196, in raw_input
    lineout = self.prefilter(line,continue_prompt)
  File "/home/ab/local/pythonlib/IPython/iplib.py", line 2279, in multiline_prefilter
    out.append(self._prefilter(l, continue_prompt))
  File "/home/ab/local/pythonlib/IPython/iplib.py", line 2261, in _prefilter
    return prefilter.prefilter(line_info, self)
  File "/home/ab/local/pythonlib/IPython/prefilter.py", line 149, in prefilter
    handler = check(line_info, ip)
  File "/home/ab/local/pythonlib/IPython/prefilter.py", line 277, in checkAutocall
    oinfo = l_info.ofind(ip) # This can mutate state via getattr
  File "/home/ab/local/pythonlib/IPython/prefilter.py", line 72, in ofind
    self._oinfo = ip._ofind(self.iFun)
  File "/home/ab/local/pythonlib/IPython/Magic.py", line 234, in _ofind
    obj = getattr(obj,part)
  File "foo.py", line 6, in __getattr__
    traceback.print_stack()
myfunc()

In [5]: f.myfunc ()
myfunc()

[Angelo Bonet (abonet)]

It appears this behavior is an artifact of having %autocall turned on -- turning it off makes it behave "correctly".

Apparently, someone was aware of the behavior given this comment in the LineInfo::ofind method defined in prefilter.py:

        Note: can cause state changes because of calling getattr, but should
        only be run if autocall is on and if the line hasn't matched any
        other, less dangerous handlers.

At any rate, I think this behavior is questionable. At a minimum it should be documented in %autocall? so users understand the risks of having it enabled.

[Robert Kern (robert-kern)]

To be clear, this is the output:

In [10]: %autocall
Automatic calling is: Smart

In [11]: f.myfunc()
Getting 'myfunc()'
myfunc()

Note that attribute being requested is 'myfunc()', not 'myfunc', so the rules about when __getattr__ gets called are being obeyed.

But yes, %autocall is risky. We could probably do a better parsing of the name and avoid going through ofind() if the name isn't just a valid.python.dotted name.

[Angelo Bonet (abonet)]

I think to say that "the rules about when __getattr__ gets called are being obeyed" is a bit of an overstatement considering that 'myfunc()' [with parens] is not a *legal* attribute name: (http://docs.python.org/ref/attribute-references.html -- http://docs.python.org/ref/identifiers.html).

It appears that %autocall has a couple of nasty side-effects regarding attribute access. In this case, it's fabricating a false attribute name -- one that was never requested. Another problem is that it allows syntax-errors to propagate way further than standard-Python would allow:

For example, in IPython:

  In [15]: f.'blah()' ## Notice the quotes around the attribute -- this is a syntax error
  Getting "'blah()'" ## Still calls __gettattr__
  ------------------------------------------------------------
     File "<ipython console>", line 1
       f.'blah()'
              ^
  SyntaxError: invalid syntax ## Finally produces a syntax-error

Standard-Python recognizes the syntax-error and never passes it to __getattr__:
  >>> f.'blah()'
    File "<stdin>", line 1
      f.'blah()'
             ^
  SyntaxError: invalid syntax

[Brian Granger (ellisonbg)]

Robert,

You have been active on this ticket. Do you think this issue os something we need to look into further?

[Robert Kern (robert-kern)]

Yes. Check the form of the name before passing it through ofind() per
my last message.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco