Spell checking

[sander]

Spell checking support on all platforms would be very nice. It should work similar as in Firefox. There should be an option in the preferences to disable spell checking called "Check spelling".

Interesting URLs:
http://alphatcl.sourceforge.net/wikit/186
http://wiki.tcl.tk/882

[Mats (matsben)]

Basic functionality implemented. Package to interface ispell and aspell command line tools. Component that interfaces spell package with application. Missing suggestions popup etc.

[sander (s-devrieze)]

2007/10/23, Mats <email address hidden>:
> Basic functionality implemented. Package to interface ispell and aspell
> command line tools. Component that interfaces spell package with
> application. Missing suggestions popup etc.

Does this also works on Mac OS X?

[Mats (matsben)]

Only tested on Mac. But I failed to make the aspell work since it complains:

Error: The file "nroff" does not contain any filters.

So I have ispell first in the prio list. aspell never tested but it runs in
a "ispell
compatibility mode" so it should work. Note that work remains here.

On 10/23/07, sander <email address hidden> wrote:
>
> 2007/10/23, Mats <email address hidden>:
> > Basic functionality implemented. Package to interface ispell and aspell
> > command line tools. Component that interfaces spell package with
> > application. Missing suggestions popup etc.
>
> Does this also works on Mac OS X?
>
> --
> Spell checking
> https://bugs.launchpad.net/bugs/144824
> You received this bug notification because you are a bug assignee.
>

[Mats (matsben)]

Most things should work now, at least using ispell. Tested on Mac and some on linux. Also popup with suggestions is working. Cute!
NB: It is surprisingly difficult to figure out when a word should be tested and which word. If you encounter problems you need to know *exactly* what you are doing, each key press etc.
There are many unclear interfaces issues, like encoding and dictionaries. And Windows remains...

[sander (s-devrieze)]

2007/10/25, Mats <email address hidden>:
> Most things should work now, at least using ispell. Tested on Mac and some on linux. Also popup with suggestions is working. Cute!
> NB: It is surprisingly difficult to figure out when a word should be tested and which word. If you encounter problems you need to know *exactly* what you are doing, each key press etc.
> There are many unclear interfaces issues, like encoding and dictionaries. And Windows remains...

It don't work on Kubuntu 7.10. Where do I have to enable this feature?

--
Mvg, Sander Devrieze.

[sander (s-devrieze)]

ok, I found it. Some feedback:
* How to change the language?
* The popup menu does not work here
* It only works in the message dialog, not in the whiteboard and chat dialogs
* Can you underline the word with a red dotted line like in Firefox
instead of making the word red? If you make the word red, this may
cause problems if the user sets the default text color to red for
whatever reason or when XHTML-IM is implemented.
* Is a menu entry to enable/disable spell checking really needed?
Isn't it enough to e.g. add an option to the plugins page in the
preferences dialog?
* Why isn't it enabled by default?
* What about remembering the language for each contact you chatted with?
* What about automatically detecting the language based on messages
from the contact (this may need a new XEP; I'll ask stpeter)?

[Mats (matsben)]

* How to change the language?

Don't know :-) You have to see 'man ispell' and 'man aspell', they work differently.
I haven't looked for a common interface yet. They have different naming conventions etc.

* The popup menu does not work here

Works both on my Mac and SUSE box. Note that it is not safe to assume that the speller gives
you suggestions (non popup) even if word is misspelled. You may also try to force load aspell
instead if ispell in contrib/spell.tcl:

    variable spellers [list ispell aspell]

Just change their order.
Update: there seems to be problems with aspell. Under investigation.

* It only works in the message dialog, not in the whiteboard and chat dialogs

Forgot to check in Chat and GroupChat.

* Can you underline the word with a red dotted line like in Firefox
instead of making the word red? If you make the word red, this may
cause problems if the user sets the default text color to red for
whatever reason or when XHTML-IM is implemented.

No because the internal limitations of the text widget :-(

* Is a menu entry to enable/disable spell checking really needed?

I think so. I often happen to change language between Swedish and English and it can be
so frustrating to have it all red.

Isn't it enough to e.g. add an option to the plugins page in the
preferences dialog?
* Why isn't it enabled by default?

Experimental.

* What about remembering the language for each contact you chatted with?

For the future.

* What about automatically detecting the language based on messages
from the contact (this may need a new XEP; I'll ask stpeter)?

xml:lang ?

[Mats (matsben)]

Two fixes:
1) Missing popup, bug unrelated to spell program
2) Spell on/off was not saved to prefs

Tried "iggen" using aspell. Lot of suggestions.

[sander (s-devrieze)]

2007/10/25, Mats <email address hidden>:
> * The popup menu does not work here
>
> Works both on my Mac and SUSE box. Note that it is not safe to assume that the speller gives
> you suggestions (non popup) even if word is misspelled. You may also try to force load aspell
> instead if ispell in contrib/spell.tcl:
>
> variable spellers [list ispell aspell]
>
> Just change their order.
> Update: there seems to be problems with aspell. Under investigation.

How should I see this popup menu? Right mouse click or something else?

> * It only works in the message dialog, not in the whiteboard and chat
> dialogs
>
> Forgot to check in Chat and GroupChat.

And the whiteboard?

> * Can you underline the word with a red dotted line like in Firefox
> instead of making the word red? If you make the word red, this may
> cause problems if the user sets the default text color to red for
> whatever reason or when XHTML-IM is implemented.
>
> No because the internal limitations of the text widget :-(

I know you are good at removing limitations! ;-)

> * Is a menu entry to enable/disable spell checking really needed?
>
> I think so. I often happen to change language between Swedish and English and it can be
> so frustrating to have it all red.

I like the way how it works in Firefox:
* RMB click in a text field opens a menu with amongst others a submenu
called "Languages". In this submenu people can select between all
installed spell check languages.
* This menu (not the submenu) also contains an option "Check spelling
in this field" (enabled by default)
* If this menu is opened when on a misspelled word, you see the
suggestions in bold at the top of this menu. Just under these
suggestions you also can select an option to add the word to the
dictionary.

> * What about remembering the language for each contact you chatted with?
>
> For the future.
>
> * What about automatically detecting the language based on messages
> from the contact (this may need a new XEP; I'll ask stpeter)?
>
> xml:lang ?

Yes, but then the sender has to specify the language he uses each
time. I was thinking about that Coccinella can see which languages
your contact understands and then asks you which language you will
speak to him.

--
Mvg, Sander Devrieze.

[Mats (matsben)]

On 10/25/07, sander <email address hidden> wrote:
> 2007/10/25, Mats <email address hidden>:
> > * The popup menu does not work here
> >
> > Works both on my Mac and SUSE box. Note that it is not safe to assume that
> the speller gives
> > you suggestions (non popup) even if word is misspelled. You may also try
> to force load aspell
> > instead if ispell in contrib/spell.tcl:
> >
> > variable spellers [list ispell aspell]
> >
> > Just change their order.
> > Update: there seems to be problems with aspell. Under investigation.
>
> How should I see this popup menu? Right mouse click or something else?
>

RMB on misspelled word. Not all words get a suggestion from the
spellchecker and no menu.

> > * It only works in the message dialog, not in the whiteboard and chat
> > dialogs
> >
> > Forgot to check in Chat and GroupChat.
>
> And the whiteboard?

Not possible. The text item there is really limited. Perhaps when we
switch to SVG.

>
> > * Can you underline the word with a red dotted line like in Firefox
> > instead of making the word red? If you make the word red, this may
> > cause problems if the user sets the default text color to red for
> > whatever reason or when XHTML-IM is implemented.
> >
> > No because the internal limitations of the text widget :-(
>
> I know you are good at removing limitations! ;-)
>

Yes but this one is hardcoded into the Tcl/Tk core code.

> > * Is a menu entry to enable/disable spell checking really needed?
> >
> > I think so. I often happen to change language between Swedish and English
> and it can be
> > so frustrating to have it all red.
>
> I like the way how it works in Firefox:
> * RMB click in a text field opens a menu with amongst others a submenu
> called "Languages". In this submenu people can select between all
> installed spell check languages.

See spell:alldicts
Right now I want it to mature a bit before fiddling with this.

> * This menu (not the submenu) also contains an option "Check spelling
> in this field" (enabled by default)

If spelling is on (Info menu) spell checking works while typing or editing.
This was the hard part. If you already have a message/chat and
switches on spellchecking,
it will automatically check the text.

> * If this menu is opened when on a misspelled word, you see the
> suggestions in bold at the top of this menu. Just under these
> suggestions you also can select an option to add the word to the
> dictionary.

Currently only suggestions are shown.
The problem is that I have to live with both aspell and ispell and
they work differently in some aspects.

>
> > * What about remembering the language for each contact you chatted with?
> >
> > For the future.
> >
> > * What about automatically detecting the language based on messages
> > from the contact (this may need a new XEP; I'll ask stpeter)?
> >
> > xml:lang ?
>
> Yes, but then the sender has to specify the language he uses each
> time. I was thinking about that Coccinella can see which languages
> your contact understands and then asks you which language you will
> speak to him.
>

Mats

[Mats (matsben)]

Now also on Windows!
This requires a dedicated installation from
http://aspell.net/win32/
including the necessary dictionaries. Perhaps a FAQ or something on this?

[antoniofcano (antoniofcano)]

Today I've update my CVS copy and discover this new feature ouoowwwwww!!!. Playing with it and goes really fine. I don't know if is something possible to include words into the dictionary of the ispell/aspell, when it doesn't recognize the word and is well write.

Great job

[Mats (matsben)]

I'm still working on it. I had hoped to finish it up today and write a
blog about it
but there are weird things going on.
Including words should be possible:

When in the -a mode, ispell will also accept lines of single words pre-
       fixed with any of '*', '&', '@', '+', '-', '~', '#', '!', '%', '`', or
       '^'. A line starting with '*' tells ispell to insert the word into the
       user's dictionary (similar to the I command). A line starting with '&'
       tells ispell to insert an all-lowercase version of the word into the
       user's dictionary (similar to the U command). A line starting with '@'
       causes ispell to accept this word in the future (similar to the A com-
       mand).

On 10/26/07, antoniofcano <email address hidden> wrote:
> Today I've update my CVS copy and discover this new feature
> ouoowwwwww!!!. Playing with it and goes really fine. I don't know if is
> something possible to include words into the dictionary of the
> ispell/aspell, when it doesn't recognize the word and is well write.
>
> Great job
>
> --
> Spell checking
> https://bugs.launchpad.net/bugs/144824
> You received this bug notification because you are a bug assignee.
>

[Mats (matsben)]

Added method to select dictionaries. Couldn't find a way to get current dict from aspell/ispell.
Added method to add word to dict.
I have seen some weirdness using Swedish dict but don't know if it depends on the dict or my code.

[antoniofcano (antoniofcano)]

In Linux using aspell you can do in a little hacky way.

ls /var/lib/aspell/*rws

/var/lib/aspell/en_CA-w_accents-only.rws /var/lib/aspell/en_US-w_accents-only.rws
/var/lib/aspell/en_CA-wo_accents-only.rws /var/lib/aspell/en_US-wo_accents-only.rws
/var/lib/aspell/en-common.rws /var/lib/aspell/en-variant_0.rws
/var/lib/aspell/en_GB-ise-w_accents-only.rws /var/lib/aspell/en-variant_1.rws
/var/lib/aspell/en_GB-ise-wo_accents-only.rws /var/lib/aspell/en-variant_2.rws
/var/lib/aspell/en_GB-ize-w_accents-only.rws /var/lib/aspell/es.rws
/var/lib/aspell/en_GB-ize-wo_accents-only.rws

This are the languages supported. Getting the list of files in that directory and taking the word between / and - (/en_CA-).

Little bug, I select the language and it does fine but next time I go to the menu the language doesn't appear selected in list.

[Mats (matsben)]

In spell.tcl I do:

proc spell::alldicts {} {
    variable static

    if {[info exists static(dicts)]} {
 return $static(dicts)
    } else {
 set L [list]
 set cmd [AutoExecOK aspell]
 if {[llength $cmd]} {
     set names [exec $cmd dicts]
     set L [lsort -unique [lapply {regsub {(-.+)}} $names [list ""]]]
     set static(dicts) $L
 }
 return $L
    }
}

Essentially: "aspell dicts"

The bug you say isn't implemented, see components/Spell.tcl:

 # Have no idea of how to get this.
 set dictname en

aspell didn't install right on my Mac and I can't do 'man aspell'.
There should be a command to get the current dict in use.

[sander (s-devrieze)]

"Yes, but then the sender has to specify the language he uses each
time. I was thinking about that Coccinella can see which languages
your contact understands and then asks you which language you will
speak to him."

Seems XEP-054 contains some sections that allow clients to add this information. See:
* section 6.9.7 Languages Known Less Well
* section 6.9.8 Languages Known Well
* Bug #144886

[Mats (matsben)]

On 10/29/07, sander <email address hidden> wrote:
> "Yes, but then the sender has to specify the language he uses each
> time. I was thinking about that Coccinella can see which languages
> your contact understands and then asks you which language you will
> speak to him."

Maybe but this is a slightly different matter.
I am thinking of how to create different instances of aspell.
Currently I don't have support for it but should be possible.
A key question is if aspell keeps its dicts per process or has an
overall settings.
I don't know. Once I get a working aspell I should test this.

>
> Seems XEP-054 contains some sections that allow clients to add this
> information. See:
> * section 6.9.7 Languages Known Less Well
> * section 6.9.8 Languages Known Well
> * Bug #144886
>

I don't see this in http://www.xmpp.org/extensions/xep-0054.html

[sander (s-devrieze)]

2007/10/30, Mats <email address hidden>:
> > Seems XEP-054 contains some sections that allow clients to add this
> > information. See:
> > * section 6.9.7 Languages Known Less Well
> > * section 6.9.8 Languages Known Well
> > * Bug #144886
>
> I don't see this in http://www.xmpp.org/extensions/xep-0054.html

I meant XEP-0154...Iprobably didn't pressed hard enough :o)

--
Mvg, Sander Devrieze.

[Mats (matsben)]

Seems like a mega-mega vcard...
Sidenote: I need to get spell.tcl more robust and failsafe.
Just discoverd apt-get aspell. Very handy. Now I've got a working aspell on Mac.

On 10/30/07, sander <email address hidden> wrote:
> 2007/10/30, Mats <email address hidden>:
> > > Seems XEP-054 contains some sections that allow clients to add this
> > > information. See:
> > > * section 6.9.7 Languages Known Less Well
> > > * section 6.9.8 Languages Known Well
> > > * Bug #144886
> >
> > I don't see this in http://www.xmpp.org/extensions/xep-0054.html
>
> I meant XEP-0154...Iprobably didn't pressed hard enough :o)
>
> --
> Mvg, Sander Devrieze.
>
> --
> Spell checking
> https://bugs.launchpad.net/bugs/144824
> You received this bug notification because you are a bug assignee.
>

[sander (s-devrieze)]

2007/10/30, Mats <email address hidden>:
> Seems like a mega-mega vcard...

idd, it's meant to replace the current vcard-temp XEP.

--
Mvg, Sander Devrieze.

[antoniofcano (antoniofcano)]

Little suggestion. Right click over a bad spell word. This should launch a menu with three options: Word suggestion, Include into the active dictionary and Language select for the bad word.

This let us write in spanish and put in the chat english words.

[Mats (matsben)]

I currently limit spell.tcl to use only a single 'aspell' process and
that is limited to a single language (dict). I presume that dict
selection is possible per process and not system wide. My intention is
to make spell.tcl more OO and keep an aspell process for each dict.
Then keep a set of spell instances, one for each dict, where new
processes (dicts) are created on demand. This requires some work,
though.
Testing:

sudo apt-get install aspell-de OK
sudo apt-get install aspell-sv
E: Couldn't find package aspell-sv

:-(

[sander (s-devrieze)]

2007/11/2, Mats <email address hidden>:
> sudo apt-get install aspell-sv
> E: Couldn't find package aspell-sv
>
> :-(

What OS?

Ubuntu: http://packages.ubuntu.com/cgi-bin/search_packages.pl?keywords=aspell-sv&searchon=names&subword=1&version=all&release=all
Debian: http://packages.debian.org/search?keywords=aspell-sv&searchon=names&suite=all&section=all
Compile manually: ftp://ftp.gnu.org/gnu/aspell/dict/0index.html
Darwin ports: http://aspell-dict-sv.darwinports.com/

--
Mvg, Sander Devrieze.

[Mats (matsben)]

Darwin, but I think I use Fink by using 'apt-get'. Seems not there:
http://pdb.finkproject.org/pdb/browse.php?summary=aspell
Guess I need a manual install somehow.

/Mats

On 11/2/07, sander <email address hidden> wrote:
> 2007/11/2, Mats <email address hidden>:
> > sudo apt-get install aspell-sv
> > E: Couldn't find package aspell-sv
> >
> > :-(
>
> What OS?
>
> Ubuntu:
> http://packages.ubuntu.com/cgi-bin/search_packages.pl?keywords=aspell-sv&searchon=names&subword=1&version=all&release=all
> Debian:
> http://packages.debian.org/search?keywords=aspell-sv&searchon=names&suite=all&section=all
> Compile manually: ftp://ftp.gnu.org/gnu/aspell/dict/0index.html
> Darwin ports: http://aspell-dict-sv.darwinports.com/
>
> --
> Mvg, Sander Devrieze.
>
> --
> Spell checking
> https://bugs.launchpad.net/bugs/144824
> You received this bug notification because you are a bug assignee.
>

[Mats (matsben)]

Added workarounds for the text widgets wordstart/wordend bugs which wont work for utf-8 characters. Also discovered that the dictionary switches wont behave properly since I read output line by line and that screws up the state array when getting more than one line.

[Mats (matsben)]

Despite many attempts I have not been able to handle async processing of the pipe method used. My comments:

# @@@ Despite many attempts I haven't been able to find a robust way to
# spell test more than a single word in an interactive mode.
# There is no way to relate what comes out with what was put in
# when it works async like this.
# I can only see the C api as a viable solution.
# Note that in such a case most of this code can still be used.

The C interface is described in: http://aspell.net/man-html/Through-the-C-API.html#Through-the-C-API

[sander (s-devrieze)]

any update?

[Mats (matsben)]

There are some C code wrapped in critlib, attached. This is work in progress and is not yet running.