python-mode very slow

[Georg Brandl]

Since the last few commits, python-mode is unbearably slow on nontrivial files. Even just moving around in the file makes Emacs use 100% CPU for a few seconds.

If this is due to the fix for lp:1102011, I would rather live with the highlight bug :)

[Andreas Roehler (a-roehler)]

The related commit extended a regexp only slightly, it's rather not the cause.
Having some example file displaying slowness will be helpful.

[Georg Brandl (gbrandl)]

Try the file below. I have narrowed the problem to the fix for lp:1102011 -- the regex *must* have pathological behavior (which wouldn't surprise me, such backtracking problems are very hard to fix). In general, for that many lines between module start and docstring, I think it is quite fine for python-mode not to color the docstring as such. I think the number of permitted comment lines should be restricted to 2, in order to accomodate a shebang line and a coding declaration.

# -*- coding: utf-8 -*-
# *****************************************************************************
# <Name of software>
# Copyright (c) 2009-2012 by the contributors (see AUTHORS)
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software
# Foundation; either version 2 of the License, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
# details.
#
# You should have received a copy of the GNU General Public License along with
# this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
# Module authors:
# Georg Brandl <email address>
#
# *****************************************************************************

"""Some docstring."""

__version__ = "$Revision$"

[Andreas Roehler (a-roehler)]

made some small API-change: forward commands will reach EOB if not succesful

Need some more info - what command from which point - should you still get the bug.

[Georg Brandl (gbrandl)]

The latest version feels better when moving around, but changing the buffer in the header area is still slow (seconds to insert a newline).

For example, try removing the empty line before "Some docstring." and putting it in again.

[Andreas Roehler (a-roehler)]

hmm, not the slighest delay here

[Georg Brandl (gbrandl)]

Strange. All I can say is that if I revert the regex to the one before the original fix, it works fine as it always did.

[Andreas Roehler (a-roehler)]

Then `py-use-font-lock-doc-face-p' set to nil should make it gone, right?

[Andreas Roehler (a-roehler)]

Hi Tom,

assigned this bug for you, as can't affirm it and have no idea what might be wrong.
 Maybe you can comment?

Cheers,

Andreas

[Georg Brandl (gbrandl)]

Confirmed, setting `py-use-font-lock-doc-face-p' to nil (and calling "python-mode" again) makes it go away. Recommend reverting the original patch until this is cleared up.

[Andreas Roehler (a-roehler)]

may confirm it meanwhile

[Tom Willemse (c-tom)]

I will look into this tomorrow or the day after

[Tom Willemse (c-tom)]

I've uploaded a commit to lp:~c-tom/python-mode/doc-face-speedup. I haven't been able to see any slow-down in the given test file when fontifying the font-lock-doc-face, even when making it bigger, but Andreas suggested using some functions from python-mode which, as a side-effect, also allow empty lines between the docstring and it's definition, so I've tried that.

Since I can't reproduce it (on either my PC or netbook) I would ask you, Georg, if you could please look if this helps.

I don't know what the usual etiquette for such changes is, so please forgive me if I'm doing something wrong and let me know so I can keep it in mind in the future.

[Georg Brandl (gbrandl)]

I've applied the change from your branch. I don't notice slowdown, but neither is the docstring in my example (see above) highlighted using the doc-face... :)

[Tom Willemse (c-tom)]

Sorry, in all the excitement I seem to have forgotten to add a check for the beginning of buffer. I've uploaded a new commit that should fix this. I don't understand how I thought I had fixed that...

[Andreas Roehler (a-roehler)]

Am 02.02.2013 10:26, schrieb Tom Willemsen:
> Sorry, in all the excitement I seem to have forgotten to add a check for
> the beginning of buffer. I've uploaded a new commit that should fix
> this. I don't understand how I thought I had fixed that...
>

IMHO it's not fixed that way, maybe just working per chance because of (bobp)

Severals errors inside.

Patch attached should fix it against previous commit.

Cheers

[Tom Willemse (c-tom)]

Andreas Roehler <email address hidden> writes:

> IMHO it's not fixed that way, maybe just working per chance because of
> (bobp)

It shouldn't, it's one of the possibilities, docstrings appear either at
the start of a function/method, the start of a class or the start of a
module (file). So after going to the beginning of the statement it
should just be at either at the start of def-or-class or at the
beginning of the file.

If there is something wrong with my reasoning please let me know.

> Severals errors inside.

Inside what? If they're in my code please let me know and I will try to
fix it.

> Patch attached should fix it against previous commit.

The patch broke all python font locking for me.

[Andreas Roehler (a-roehler)]

Am 02.02.2013 12:27, schrieb Tom Willemsen:
> Andreas Roehler <email address hidden> writes:
>
>> IMHO it's not fixed that way, maybe just working per chance because of
>> (bobp)
>
> It shouldn't, it's one of the possibilities, docstrings appear either at
> the start of a function/method, the start of a class or the start of a
> module (file). So after going to the beginning of the statement it
> should just be at either at the start of def-or-class or at the
> beginning of the file.
>

right. I'm correcting my previous post in such: check for (or looking-at ... (bopb)) is needed.

> If there is something wrong with my reasoning please let me know.
>
>> Severals errors inside.
>
> Inside what? If they're in my code please let me know and I will try to
> fix it.
>

it relats to the (beginning-of-def-or-class) call now, which is wrong IMHO as might reach BOB
  while just called from inside a string, needed is beginning-of-statement.

>> Patch attached should fix it against previous commit.
>
> The patch broke all python font locking for me.
>

Ohh, indeed.

So I'm awaiting your next post/commit :-)

[Tom Willemse (c-tom)]

Andreas Roehler <email address hidden> writes:

>> Inside what? If they're in my code please let me know and I will try
>> to fix it.
>
> it relats to the (beginning-of-def-or-class) call now, which is wrong
> IMHO as might reach BOB while just called from inside a string, needed
> is beginning-of-statement.

I agree that that would be wrong, but in `py-doc-string-p' I'm using
`py-beginning-of-statement' to go to the beginning of the current
statement and then check if where at beginning of buffer, def or class
by using `bobp' and `py-beginning-of-def-or-class-p', is this not
correct?

[Andreas Roehler (a-roehler)]

ah, sorry, it is. Didn't notice the final "-p".

[Andreas Roehler (a-roehler)]

local merge went fine. Test passed. Will upload ASAP