Allow embedded shells to work in doctest

[Fernando Perez]

Embedded shells should work in doctest. They fail currently because doctest captures stdout. A workaround is to redirect output to stderr. For 0.11 we should either implement the workaround out of the box, or see if a diffderent solution is better.

Bug reported on-list by Philip Gatt, see for further details:

http://mail.scipy.org/pipermail/ipython-user/2010-January/006765.html

Example code that works around the problem, for 0.10:

#!/usr/bin/env python

import sys

from IPython.Shell import IPShellEmbed

class IPShellDoctest(IPShellEmbed):
    def __call__(self, *a, **kw):
        sys_stdout_saved = sys.stdout
        sys.stdout = sys.stderr
        try:
            IPShellEmbed.__call__(self, *a, **kw)
        finally:
            sys.stdout = sys_stdout_saved

def some_function():
   """
   >>> some_function()
   'someoutput'
   """
   # now try to drop into an ipython shell to help
   # with development
   IPShellDoctest()()
   return 'someoutput'

if __name__ == '__main__':
   import doctest
   print "Running doctest . . ."
   doctest.testmod()

[Brian Granger (ellisonbg)]

> Embedded shells should work in doctest.  They fail currently because
> doctest captures stdout.  A workaround is to redirect output to stderr.
> For 0.11 we should either implement the workaround out of the box, or
> see if a diffderent solution is better.

Just off the top of my head, redirecting stdout->stderr seems like it
might have undesired
side effects. But, if we go this route, the try/finally should be
replace by a context manager/with
statement.

Cheers,

Brian

> Bug reported on-list by Philip Gatt, see for further details:
>
> http://mail.scipy.org/pipermail/ipython-user/2010-January/006765.html
>
> Example code that works around the problem, for 0.10:
>
> #!/usr/bin/env python
>
> import sys
>
> from IPython.Shell import IPShellEmbed
>
> class IPShellDoctest(IPShellEmbed):
>    def __call__(self, *a, **kw):
>        sys_stdout_saved = sys.stdout
>        sys.stdout = sys.stderr
>        try:
>            IPShellEmbed.__call__(self, *a, **kw)
>        finally:
>            sys.stdout = sys_stdout_saved
>
>
> def some_function():
>   """
>   >>> some_function()
>   'someoutput'
>   """
>   # now try to drop into an ipython shell to help
>   # with development
>   IPShellDoctest()()
>   return 'someoutput'
>
> if __name__ == '__main__':
>   import doctest
>   print "Running doctest . . ."
>   doctest.testmod()
>
> ** Affects: ipython
>     Importance: Medium
>     Assignee: Fernando Perez (fdo.perez)
>         Status: Confirmed
>
> ** Changed in: ipython
>   Importance: Undecided => Medium
>
> ** Changed in: ipython
>     Assignee: (unassigned) => Fernando Perez (fdo.perez)
>
> --
> Allow embedded shells to work in doctest
> https://bugs.launchpad.net/bugs/503698
> You received this bug notification because you are a member of IPython
> Developers, which is subscribed to IPython.
>
> Status in IPython - Enhanced Interactive Python: Confirmed
>
> Bug description:
> Embedded shells should work in doctest.  They fail currently because doctest captures stdout.  A workaround is to redirect output to stderr.  For 0.11 we should either implement the workaround out of the box, or see if a diffderent solution is better.
>
> Bug reported on-list by Philip Gatt, see for further details:
>
> http://mail.scipy.org/pipermail/ipython-user/2010-January/006765.html
>
> Example code that works around the problem, for 0.10:
>
> #!/usr/bin/env python
>
> import sys
>
> from IPython.Shell import IPShellEmbed
>
> class IPShellDoctest(IPShellEmbed):
>    def __call__(self, *a, **kw):
>        sys_stdout_saved = sys.stdout
>        sys.stdout = sys.stderr
>        try:
>            IPShellEmbed.__call__(self, *a, **kw)
>        finally:
>            sys.stdout = sys_stdout_saved
>
>
> def some_function():
>   """
>   >>> some_function()
>   'someoutput'
>   """
>   # now try to drop into an ipython shell to help
>   # with development
>   IPShellDoctest()()
>   return 'someoutput'
>
> if __name__ == '__main__':
>   import doctest
>   print "Running doctest . . ."
>   doctest.testmod()
>
>
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Fernando Perez (fdo.perez)]

On Wed, Jan 6, 2010 at 6:11 AM, Brian Granger <email address hidden> wrote:
>
> Just off the top of my head, redirecting stdout->stderr seems like it
> might have undesired
> side effects.  But, if we go this route, the try/finally should be
> replace by a context manager/with
> statement.

Good points both, thanks. We may have to live with the weird side
effects of stderr reuse, since doctest is already hijacking stdout for
its own purposes, but at least we should point it out.

And a context manager is indeed a good approach for this type of
problem. It may be a bit of overkill if all we end up needing is
really the above code, but we'll see: if we need that pattern more
than once or it gets more complicated, we'll use a CM, otherwise I
think a simply try/finally is still OK for cases like this.

Thanks again for the feedback!

[Brian Granger (ellisonbg)]

> And a context manager is indeed a good approach for this type of
> problem.  It may be a bit of overkill if all we end up needing is
> really the above code, but we'll see: if we need that pattern more
> than once or it gets more complicated, we'll use a CM, otherwise I
> think a simply try/finally is still OK for cases like this.

I may already have context managers that do things like this (we do
this type of thing all over IPython).

Brian

> Thanks again for the feedback!
>
> --
> Allow embedded shells to work in doctest
> https://bugs.launchpad.net/bugs/503698
> You received this bug notification because you are a member of IPython
> Developers, which is subscribed to IPython.
>
> Status in IPython - Enhanced Interactive Python: Confirmed
>
> Bug description:
> Embedded shells should work in doctest.  They fail currently because doctest captures stdout.  A workaround is to redirect output to stderr.  For 0.11 we should either implement the workaround out of the box, or see if a diffderent solution is better.
>
> Bug reported on-list by Philip Gatt, see for further details:
>
> http://mail.scipy.org/pipermail/ipython-user/2010-January/006765.html
>
> Example code that works around the problem, for 0.10:
>
> #!/usr/bin/env python
>
> import sys
>
> from IPython.Shell import IPShellEmbed
>
> class IPShellDoctest(IPShellEmbed):
>    def __call__(self, *a, **kw):
>        sys_stdout_saved = sys.stdout
>        sys.stdout = sys.stderr
>        try:
>            IPShellEmbed.__call__(self, *a, **kw)
>        finally:
>            sys.stdout = sys_stdout_saved
>
>
> def some_function():
>   """
>   >>> some_function()
>   'someoutput'
>   """
>   # now try to drop into an ipython shell to help
>   # with development
>   IPShellDoctest()()
>   return 'someoutput'
>
> if __name__ == '__main__':
>   import doctest
>   print "Running doctest . . ."
>   doctest.testmod()
>
>
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Fernando Perez (fdo.perez)]

On Wed, Jan 6, 2010 at 2:28 PM, Brian Granger <email address hidden> wrote:
>
> I may already have context managers that do things like this (we do
> this type of thing all over IPython).

Great, thanks!