py-find-imports can't handle multiline imports or comma separated imports

[Russell Sim]

Hi,

I have been messing about with the autocomplete and in the process i found that the py-find-imports function didn't support, comma separated imports or mulitline imports. I have had a first run at implementing this functionality, if you have any feedback i will gladly adjust the patch.

I apologize for not including tests, I have made a test import file, but adding the test was harder then i expected.

Thank you,
Russell

[Russell Sim (russell-sim)]

test import file

[Russell Sim (russell-sim)]

Here is an updated patch that includes a test :) . I should have done this the first time.

[Andreas Roehler (a-roehler)]

Am 11.07.2012 04:38, schrieb Russell Sim:
> Here is an updated patch that includes a test :) . I should have done
> this the first time.
>
> ** Patch added: "py-find-mulitline-imports.patch"
> https://bugs.launchpad.net/python-mode/+bug/1023236/+attachment/3219255/+files/py-find-mulitline-imports.patch
>

Hi Rusell,

thanks a lot.

Seeing your code dealing with backslashed lines --correctly so far--, there is a request still: as import is a statement, suggest the use of
py-end-of-statement to determine its borders.

BTW that part of code also is dealt with in branch

lp:~ufleisch/python-mode/improve-completion

Please be invited to join the team, so your will get all mail around,

thanks again,

Andreas

[Urs Fleisch (ufleisch)]

Hi Russell, Andreas,

This improves the parsing of import statements. However, things like this should not go into the imports result:

import sys, os; os.remove('do/something/nasty')

So it would be probably better to stop at semicolons rather than go to the end of the statement list with py-end-of-statement.

Regards,
Urs

[Andreas Roehler (a-roehler)]

Am 14.07.2012 17:40, schrieb Urs Fleisch:
> Hi Russell, Andreas,
>
> This improves the parsing of import statements. However, things like
> this should not go into the imports result:
>
> import sys, os; os.remove('do/something/nasty')
>
> So it would be probably better to stop at semicolons rather than go to
> the end of the statement list with py-end-of-statement.
>
> Regards,
> Urs
>

okay, thanks. Which signals a bug in py-end-of-statement for me.

IMO a semicolon ends a statement. (?)

Andreas

[Urs Fleisch (ufleisch)]

According to http://docs.python.org/reference/compound_stmts.html, a statement list is a statement:

statement ::= stmt_list NEWLINE | compound_stmt
stmt_list ::= simple_stmt (";" simple_stmt)* [";"]

This would mean that a suite of semicolon separated simple statements is still a statement and py-end-of-statement is correct.

[Andreas Roehler (a-roehler)]

you are referring to compound statements, while py-end-of-statement deals with simple statement.

Rusell provided a solution for a special case, which is okay. However, seen from this point, it's probable, there might be more and other special cases: blank lines, comments , lists over multiple lines etc.

That's the notion of statement is for.

Cheers,

Andreas

[Russell Sim (russell-sim)]

Andreas Roehler <email address hidden> writes:

> you are referring to compound statements, while py-end-of-statement
> deals with simple statement.

> Rusell provided a solution for a special case, which is okay. However,
> seen from this point, it's probable, there might be more and other
> special cases: blank lines, comments , lists over multiple lines etc.

We could perhaps copy the data out to a temporary buffer and use find
and replace to change the ; characters to newlines. That should make it
work for *most* other cases.

I agree, probably one of the harder to parse forms will be.

from os import (mknod, # import for later
     open, # another interesting import
     path) # import completed

So perhaps copy interesting lines to temporary buffer:
1. Trim comments
2. split the lines with a ;
3. Delete the string \\\n from the buffer
4. Parse with a simple regex for most of the imports.
4. Parse with a regex that will work over multiple lines with ()

The only thing that wont be handled will be conditional import, so I
guess it's best to leave them out for the time being.

I'll make a branch see what I can put together....

[Andreas Roehler (a-roehler)]

Am 16.07.2012 04:34, schrieb Russell Sim:
> Andreas Roehler <email address hidden> writes:
>
>> you are referring to compound statements, while py-end-of-statement
>> deals with simple statement.
>
>> Rusell provided a solution for a special case, which is okay. However,
>> seen from this point, it's probable, there might be more and other
>> special cases: blank lines, comments , lists over multiple lines etc.
>
> We could perhaps copy the data out to a temporary buffer and use find
> and replace to change the ; characters to newlines. That should make it
> work for *most* other cases.
>
> I agree, probably one of the harder to parse forms will be.
>
> from os import (mknod, # import for later
> open, # another interesting import
> path) # import completed
>
>
> So perhaps copy interesting lines to temporary buffer:
> 1. Trim comments
> 2. split the lines with a ;
> 3. Delete the string \\\n from the buffer
> 4. Parse with a simple regex for most of the imports.
> 4. Parse with a regex that will work over multiple lines with ()

should be implemented, basically inside py-end-of-statement

>
> The only thing that wont be handled will be conditional import, so I
> guess it's best to leave them out for the time being.

or rather go on herewith :)

May you point me onto some example?

thanks

Andreas

>
> I'll make a branch see what I can put together....
>

[Russell Sim (russell-sim)]

A simple example

try:
    import broken__
except:
    import os

But I have seen examples with if statements. I'll grab one when I get home.
On Jul 16, 2012 6:10 PM, "Andreas Roehler" <email address hidden>
wrote:

> Am 16.07.2012 04:34, schrieb Russell Sim:
> > Andreas Roehler <email address hidden> writes:
> >
> >> you are referring to compound statements, while py-end-of-statement
> >> deals with simple statement.
> >
> >> Rusell provided a solution for a special case, which is okay. However,
> >> seen from this point, it's probable, there might be more and other
> >> special cases: blank lines, comments , lists over multiple lines etc.
> >
> > We could perhaps copy the data out to a temporary buffer and use find
> > and replace to change the ; characters to newlines. That should make it
> > work for *most* other cases.
> >
> > I agree, probably one of the harder to parse forms will be.
> >
> > from os import (mknod, # import for later
> > open, # another interesting import
> > path) # import completed
> >
> >
> > So perhaps copy interesting lines to temporary buffer:
> > 1. Trim comments
> > 2. split the lines with a ;
> > 3. Delete the string \\\n from the buffer
> > 4. Parse with a simple regex for most of the imports.
> > 4. Parse with a regex that will work over multiple lines with ()
>
> should be implemented, basically inside py-end-of-statement
>
>
> >
> > The only thing that wont be handled will be conditional import, so I
> > guess it's best to leave them out for the time being.
>
> or rather go on herewith :)
>
> May you point me onto some example?
>
> thanks
>
> Andreas
>
>
> >
> > I'll make a branch see what I can put together....
> >
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1023236
>
> Title:
> py-find-imports can't handle multiline imports or comma separated
> imports
>
> Status in An Emacs mode for editing Python code:
> Fix Committed
>
> Bug description:
> Hi,
>
> I have been messing about with the autocomplete and in the process i
> found that the py-find-imports function didn't support, comma
> separated imports or mulitline imports. I have had a first run at
> implementing this functionality, if you have any feedback i will
> gladly adjust the patch.
>
> I apologize for not including tests, I have made a test import file,
> but adding the test was harder then i expected.
>
> Thank you,
> Russell
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/python-mode/+bug/1023236/+subscriptions
>

[Andreas Roehler (a-roehler)]

okay, may you file a new report mentioning "conditional" or so?

thanks,

Andreas

[Russell Sim (russell-sim)]

Ok, i wasn't able to find any other real examples but there were other edge cases like,

suds = __import__("suds")

[Andreas Roehler (a-roehler)]

Am 16.07.2012 10:46, schrieb Russell Sim:
> Ok, i wasn't able to find any other real examples but there were other
> edge cases like,
>
> suds = __import__("suds")
>

Hi Russel,

can't see anything wrong related to python-mode.

BTW before looking at edge cases, please permit me to ask your assistance
at some real painstaking issues.

at large a couple of bugs following M-x ipython RET

IPython handling is far from being perfect.

A sub-problem with some reduced complexity is its history-handling.

If interested, see comint-input-ring-file-name inside defun py-shell for example.

Best,

Andreas

[Russell Sim (russell-sim)]

Oh, Whoops, I don't think I ever got this message Andreas. I will gladly look at the IPython history handling. And I agree the pycomplete edge cases can wait.

I had actually been avoiding IPython in emacs because it was so hard to use. :)

Cheers,
Russell