Better spam filtering for evolution

[Tom Arnold]

I trained spammassassin for weeks now, but it still only filters just a few
mails. I just want
a working bayesfilter like spambayes or the one in thunderbird.
Can i tune spammassasin to not suck or is there something wrong? ( Because some
mails arent marked
with spam information in their headers )

[Matt Zimmerman (mdz)]

What we discussed prior to the release was integrating bogofilter with
evolution, but this could not be stabilized in time. It would be a good idea to
have it in Hoary.

[Tom Arnold (g0tt)]

I have spambayes with postfix and procmail on my Gentoobox and it beats the
living crap out of spamassassin anytime.
It is python and it has that neat UNSURE feature.
I think it is much better than bogofilter!

[Matt Zimmerman (mdz)]

ubuntu-devel is the place to discuss the details; so far there is at least a
consensus that we should provide a spam filtering solution by default, and that
it should not be spamassassin.

[Tom Arnold (g0tt)]

OK,

when i have time i will join the list and donate my 0.02 €.

Thanks for all the help and keep up the good work!

[Jeff Waugh (jdub)]

This is not fixed.

[Sebastien Bacher (seb128)]

*** Bug 14014 has been marked as a duplicate of this bug. ***

[Sebastien Bacher (seb128)]

*** Bug 13372 has been marked as a duplicate of this bug. ***

[Sebastien Bacher (seb128)]

*** Bug 22504 has been marked as a duplicate of this bug. ***

[Daniel Holbach (dholbach)]

*** Bug 26742 has been marked as a duplicate of this bug. ***

[Hiroyuki Ikezoe (ikezoe)]

What about bogofilter plugin
<http://mail.gnome.org/archives/evolution-list/2005-October/msg00090.html> ?

I have been using this plugin for a few days, it's perfectly nice. I finally get
relief from spam.

[Daniel Holbach (dholbach)]

That patch has been applied to evolution in Dapper. I'm closing the bug - if anybody disagrees feel free to reopen it and explain why.

[Irios (irios)]

I'm running an up-to-date Dapper, have spamassassin installed, and have checked the kill-the-spam boxes in Evolution 2.6.0

Still, Evolution won't catch a single spam (even in cases like, recevie 100 messages, all of them spam, none caught)

[Daniel Holbach (dholbach)]

Nacho: Did you use the "mark as spam" button to teach spamassassin what is spam and what not? You might want to try the bogofilter plugin (install bogofilter and check it in the plugins section) as it is reported to work better.

[Maciej Katafiasz (mathrick)]

Reopening, since as is explained by the mail, bogofilter needs bootstrapping with ham, that's not possible within Evolution at all, and you need to invoke bogofilter manually, not to mention extremely confusing for users. This pretty much kills all the usability of the plugin until that issue is fixed.

[Hezekiah Carty (hez)]

I would like to second this. The bogofilter is a much simpler plugin to setup than spamassassin initially, but I have marked 230 messages as spam/junk, and Evolution still is doing nothing to automatically mark messages as spam.

[lp-void (lp-void)]

This bug has been opened since 2004. Isn't there some kind of interim fix that can be presented like putting some kind of spam filtering proxy in as an option that circumvents Evolution's nasty way of doing this? Or perhaps a preconfigured fetchmail/spambayes pairing that is installed together with Evolution? Or something?

[Jan Claeys (janc)]

It's not true that you can't bootstrap bogofilter with ham from within evolution, it's just not easily discoverable. What you have to do is mark at least one ham message as spam, then go to your spam folder and mark it as ham. From then on it will work.

[PeterShinners (pete-shinners)]

I would guess it should be easy to take a small cross section of users that have been bogofiltering for awhile and collect all their ham info. Then this could be shipped with the bogofilter as a reasonable starting point.

Then again, I haven't looked into this sort of thing at all.

[Anthony Edwards (anthony-fishcat)]

In respect of SpamAssassin, the SpamAssassin version currently
packaged with Dapper (and Edgy!) is 3.1.0 - now ten months (and four releases!) out of date.

This makes it (sadly) to all intents and purposes useless as a spam filtering solution for real world deployment. The spam/spam filtering arena is effectively an arms race; the only way to filter spam effectively using SpamAssassin is to get current and stay current.

I have raised this as a bug in and of itself:

https://launchpad.net/distros/ubuntu/+source/spamassassin/+bug/54480

[Tom Arnold (g0tt)]

As the initial poster it is nice to see that this is still concerning people. I will try to take matters in my own hand and describe the solution with the python written spambayes. It is really neat and the classification in spam, ham and UNSURE is really great, coz you only have to care for the few unsure messages. I think this solution would be best suited for Ubuntu. I´ll be back.

[Tom Arnold (g0tt)]

I did not succeed in integrating spam-bayes into evolution in a easy manor, coz i am using IMAP and things became very tricky.
I think bogofilter is the way to, because it is faster and it already has a evolution plugin. But from what i have read it is not always working right.
And there has to be an easy way to train bogofilter from within evolution and it should be installed with evolution.

[Craig Sampson (ubuntu-psi-aus)]

All this training, spam, ham, blah blah - I've got to ask, why we don't just find a tool which uses RBL's and be done with the whole thing?

In windows I use something called spampal which has a bunch of functionality but I use it basically as a turnkey method of accessing RBL's - virtually no spam gets through and I don't need to train anything.

Spamassassin is meant to be able to use RBL's but its not at all obvious/or its borked on how to make it work effectively.

Using RBL's does rely (obviously) upon having a internet connection whilst obtaining your new mail, so I can see some corner cases where it wouldn't work for people, but why not set up spam assassin by default to use RBL's and -also- use a trained filter if thats whats required to capture those corner cases?

Craig

[Jan Claeys (janc)]

> why we don't just find a tool which uses RBL's and be done with the whole thing?

Because self-learning filters generally perform a lot better after being trained, when considering both false negatives (no problem if there aren't many) and false positives (even 1 might get you in trouble). As (almost) every RBL I know causes lots of false positives, I don't think they are a good idea.

Also, it would make it possible for a third party to hurt Ubuntu users badly (those RBLs are not under our control).

[Chris Lord (cwiiis)]

bogofilter ham can be trained with POP/local mail, or with IMAP mail if you choose to keep local computer synced by cd-ing into the directory ($HOME/.evolution/mail/local/Inbox or $HOME/.evolution/mail/imap/<account>/folders/INBOX) and using either 'bogofilter -n </home/user/.evolution/mail/local/Inbox' or 'for file in [123456789]*; do bogofilter -n < $file; done' as appropriate.

[Sebastien Bacher (seb128)]

what is the interest to add an evolution tag to an evolution bug?

[Todd Mansfield (toddmansfield)]

As a user, I'm more than happy to train my own spam filter. For me, this is perfectly understandable and reasonable. Problem is, I've been trying to train Evolution in Dapper for months now and it still lets a preposterously large amount of spam through. What a patsy of a spam tool! And yes, I have configured the right options in Evolution.

I took an hour today to research and work on better spam filtering tools. I chose bogofilter. I installed it, (using aptitude) read the manual, and trained it by hand on the command line using both spam and good messages I've accumulated over the last few months. These steps all worked fine.

But integrating bogofilter with Evolution was a most confusing experience, and even now I'm still not sure I've got it right.

Integrating spam tools with email is where a good distribution like Ubuntu really _should_ come through. Make it a humane experience to hook up the spam filter, and we humans will happily train our own.

Best!
todd

[Rasmus Toftdahl Olesen (halfdan)]

Hi Ubuntu guys

I have created an Evolution plugin that uses SpamBayes for filtering, it has not been tested that much yet, so i'm hoping from the adventorous of you to help test it.

It is available as Debian packages from here (in the etch distribution):
http://halfdans.net/wiki.py/DebianRepository

The packages hopefully also works on Ubuntu, otherwise i'm sure some smart Ubuntu guy will create packages for it.

There is a list of open issues here:
http://halfdans.net/wiki.py/EvolutionSpamBayesPlugin

Again, this is still experimental, so try at your own risk.

[Ben Edwards (funkytwig)]

OK - not sure if this help much but spamassasin used to work great. I think it got broke when I went to Edgy.

[fheible (forum-heible)]

Hello enveryone,

I'm surprised by this bug and it's "severe" qualification. I do use Evolution/spamassassin since 4 years now, and it's alsways been fully efficient, once I had correctly set it up, of course.

Remember you have to:

- set up spamassassin the way you want ut to work: as a resident service (spamd), or as a command-line called by evolution (as I do)? Which mail quotation will mark it as a spam? Should some foreing languages be considered as spam by default (probably the languages you don't speak...)? Do you want to use some complementary filters such as Pyzor and Razor insisde spamassassin?
- tell Evolution that you WANT it to move your spam in designed folder: excuse me if the name of the command are a bit different (I do use Evolution in french). In Evolution 2.10.1 (an other recent versions), it's become very easy: just try Edit/Mail filters, ad a test (and precise the condition: "the mail is a spam"), action to do: "move the mail to folder XXXX").
- enshure the firts time thact spamassin is correctly called when I receive your mail (under Linux: try the commande-line "top": the spamc or spamd process should be working and using a significant part of the processor time).

On my computer, Evolution calls spammassin in command-line (advantage: low resident memory loose; incovenient: it's slower). I activated every options, and so mail checking is quite slow. But the result is great: about 200 spams a day correctly designed an moved to the spam folder, no false positive. Only one false negative (then, I must check it 'spam') a week!

PS: please forgive the poor english of a french Linux lover who doesn't trains it's English regularly...

[Jan Claeys (janc)]

Op donderdag 17-05-2007 om 19:08 uur [tijdzone +0000], schreef fheible:
> - set up spamassassin the way you want ut to work: as a resident
> service (spamd), or as a command-line called by evolution (as I do)?
> Which mail quotation will mark it as a spam? Should some foreing
> languages be considered as spam by default (probably the languages you
> don't speak...)? Do you want to use some complementary filters such as
> Pyzor and Razor insisde spamassassin?

And now try to explain this to your uncle who never used linux (or maybe
even a computer) before, and see where the bug is... ;-)

--
Jan Claeys

[fheible (forum-heible)]

On my latest Ubuntu install (6.10), spamassassin was set up by default, and worked. I just had to create the mail filter.

Previously (and particularly on other distros), it was a pain, of course!

[Mark Edgington (edgimar)]

For the record, I am running Ubuntu 7.04 on an AMD64 machine, and use Evolution to access mail via IMAP. Spam filtering with Bogofilter only works when a group of messages is selected, and the Message->Check for Junk menu item is chosen. So, it does not check automatically upon receiving new mail. I have also tried to set up filters to do things with new incoming mail, but again, this does not work -- only selecting messages manually, and choosing Message->Apply Filters will cause the filters to be executed on the selected messages. I have tried all combinations of options on the "Receiving Options" account-preferences tab (e.g. "Apply filters to new messages in INBOX on this server", which sounds like the right thing), but nothing seems to work.

So, in short, I think that unless this problem (of being able to run filters/check for spam on new incoming messages) is resolved (am I the only one experiencing it?), spam filtering will not work (properly or easily) for Evolution users who have IMAP accounts.

[kseise (kevin-seise)]

Spam filtering under Gutsy Tribe 3 fails using the spamassassin plugin. It worked fine under 7.04.

[Jørgen Syvertsen (jorgg)]

I'm experiencing some strange warnings when I run Evolution from the command line. The warnings are:

(evolution-2.10:27539): evolution-mail-WARNING **: ignored this junk plugin: not enabled or we have already loaded one
(evolution-2.10:27539): e-utils-WARNING **: Plugin 'Spamassassin junk plugin' failed to load hook 'org.gnome.evolution.mail.junk:1.0'

I'm using Ubuntu 7.04 Feisty Fawn with Evolution 2.10.1. I haven't installed the spamassassin package and I've therefor unchecked the spamassassin junk filter plugin in Evolution. So it looks like Evolution tries to load the spamassassin plugin even though it's turned off in the UI checkbox. I have the bogofilter junk plugin turned on and installed the bogofilter package.

--
JS

[Anthony S (aaaantoine)]

I believe the fix for this has been released with Gnome 2.20:

http://www.gnome.org/start/2.20/notes/en/index.html#rnusers-email-and-calendar

"Allows you to choose the Spam plugin (SpamAssassin or BogoFilter) via its preferences. And SpamAssassin will now actually learn when you mark emails as Junk or Not Junk."

Can someone with Gutsy please verify?

[Sebastien Bacher (seb128)]

right, and bogofilter in install by default on gutsy, closing

[Evgeny Kuznetsov (nekr0z)]

Gutsy here, bogofilter installed, and it seems to work, but the spam messages don't make it to the spam folder, just stay in inbox.

[Kai Schroeder (kai-schroeder)]

Evolution still selects spamassasin as the default junk filter in gutsy, although it should select bogofilter which is installed.

[Murali (mshanker)]

I am using Gutsy, and have bogofilter installed, but not spamassassin. The filter as such works, but doesn't seem to be applied to the inbox, or new mail at all. I have to select All Messages, and then apply Check for Junk for the filtering to take place. If Fiesty it worked as it showed, but not now. This seems to be similar to http://bugzilla.gnome.org/show_bug.cgi?id=444503