amavisd-new + spamassassin: cronjob spams root user

[Matthias Andree]

Binary package hint: amavisd-new

TO REPRODUCE

1. install amavisd-new and spamassassin
2. edit /etc/default/spamassassin to read CRON=1
3. edit /etc/spamassassin/local.cf to read use_bayes 0
4. execute the cronjob manually: "sudo -u amavis /usr/sbin/amavisd-new-cronjob sa-sync"

PROBLEM

Cron then starts "/usr/sbin/amavisd-new-cronjob sa-sync" which in turn runs "/usr/bin/sa-learn --sync" which then complains by mail:
ERROR: configuration specifies 'use_bayes 0', sa-learn disabled

SUGGESTION

Avoid running sa-learn if use_bayes is set to 0. I'm however not sure how this can queried from the spamassassin configuration.

[Matthias Andree (matthias-andree)]

Package versions for this report:

amavisd-new 1:2.4.2-6.2ubuntu1
spamassassin 3.2.3-0ubuntu1

I'm filing this as amavisd-new report since the cron scripts are part of amavisd-new rather than spamassassin.

[Daniel Hahler (blueyed)]

Related, but with an updated amavisd-new from Hardy (on Feisty).
After having upgraded amavisd-new, I'm now getting cron mails with the following line:
bayes: synced databases from journal in 3 seconds: 210 unique entries (350 total entries)

This gets sent from "test -e /usr/sbin/amavisd-new-cronjob && /usr/sbin/amavisd-new-cronjob sa-sync".

This is not the same bug, but related enough IMHO.

amavisd-new 2.5.2-2ubuntu1
spamassassin 3.2.1-1ubuntu1~feisty1

[Daniel Holbach (dholbach)]

Server Team: can somebody of you take a look at this one?

[Scott Kitterman (kitterman)]

I've looked at this briefly. I'm not convinced this is the correct
solution.

[Matthias Andree (matthias-andree)]

I'd second that simply discarding output (what Daniel Hahler's debdiff does) is the wrong way.

[Daniel Hahler (blueyed)]

Ok, I agree that errors should not get discarded.

What's the better approach then?
1. Fix spamassassin to not handle/output "configuration specifies 'use_bayes 0', sa-learn disabled" as an error?
2. Try to find out if use_bayes is 0 before calling sa-learn?

The first one seems easier, because in the second case the only solution I see is grepping "/etc/spamassassin/" for the use_bayes setting.
The sa-learn tool could still exit with an error code, but would just print a warning to STDOUT.

Please note, that discarding STDOUT is supposed to be done (but fails so in the case where the script is run as user amavis). Therefor I'm getting the "bayes: synced databases from journal in 3 seconds: 210 unique entries (350 total entries)" spam.

There are two similar issues:
1. STDOUT does get handled differently for invocation by "root" and "amavis" - this should get fixed so that both code paths are the same. It does not make sense to have STDOUT in the cron mail. Agreed?
2. spamassassin/sa-learn causes output on STDERR, if use_bayes is disabled.

[Scott Kitterman (kitterman)]

Spamassassin 3.2.4 is just uploaded to Hardy. I've asked to have it backported to Gutsy and Feisty. Once you get 3.2.4, please see if this is still a problem as there are some sa-learn related changes in the new release.

[Daniel Hahler (blueyed)]

There are no relevant changes from 3.2.1 (which I'm currently using) to 3.2.4 (in sa-learn).
The "error" is still outputted using WARN.

Do you think we should patch this in the spamassassin package?

[Scott Kitterman (kitterman)]

No. I think it should either be managed by changing amavisd-new or you configuring your mail delivery agent to dump stuff you don't want to see. I'm not sure which. I just wanted to check.

[Scott Kitterman (kitterman)]

Unchecked patch as I don't think dumping to /dev/null is the right way to solve this.

[Pavel Mlčoch (pavkamlc)]

Ive this problem too, but I dont have amavisd-new installed:

pavka@pavka:~$ dpkg -l amavisd-new
rc amavisd-new

but cron.daily script is installed after remove this package without purge

pavka@pavka:~$ cat /etc/cron.daily/amavisd-new
#!/bin/sh
#
# Daily maintenance for amavisd-new
# $Id: amavisd-new.cron.daily 930 2006-08-10 13:38:45Z hmh $
#
test -e /usr/sbin/amavisd-new-cronjob && exec /usr/sbin/amavisd-new-cronjob sa-clean
pavka@pavka:~$

I have daily delivered mail with "run-parts: /etc/cron.daily/amavisd-new exited with return code 1" but on my filesystem isnt file /usr/sbin/amavisd-new-cronjob

pavka@pavka:~$ ls /usr/sbin/amavisd*
ls: cannot access /usr/sbin/amavisd*: No such file or directory

[Martijn Heemels (yggdrasil)]

I have the same issue as Daniel has at his comment https://bugs.launchpad.net/ubuntu/+source/amavisd-new/+bug/165184/comments/2

Amavisd-new and spamassassin are working fine in Hardy, but still outputting the informational messages from the sa-sync cronjob:
"bayes: synced databases from journal in 1 seconds: 1019 unique entries (1343 total entries)"

I'd rather not simply drop the output in case something actually goes wrong.

[Scott Kitterman (kitterman)]

Pavel, I suspect you had it installed on that box at one point. You'll just need to edit your crontab to get rid of the cron job.

[Pavel Mlčoch (pavkamlc)]

I think, when is this package removed, this cron script should be
removed too.

Scott Kitterman napsal(a):
> Pavel, I suspect you had it installed on that box at one point. You'll
> just need to edit your crontab to get rid of the cron job.
>
>

[Jon Skanes (jon-skanes)]

IMHO, this bug should be added as a security issue. In my setup, the cron script emails the amavis user. There was no alias to root added by the installation scripts, therefore, the user who set up the system will not receive the mail. This is a potential denial of service vulnerability. By flooding the account with mail, disk space could be consumed unchecked. Anyone setting up amavis with any of the docs i've seen for Ubuntu won't even know the account has mail.

[Scott Kitterman (kitterman)]

DoSing yourself because you let the hard drive get full is an
administration problem not a security problem. I'll look at the docs and
see what I can do.

[Jon Skanes (jon-skanes)]

My worry is that someone setting up a mail server without knowing about this could end up receiving mail to that account without knowing.

[Scott Kitterman (kitterman)]

I understand. I think documentation is the best way to address it. I'm
open to a patch that's more nuanced than sending everything to dev/null.

[Jon Skanes (jon-skanes)]

In my case, if the install scripts had added an alias for amavis to root it should solve that worry.

[Matthias Andree (matthias-andree)]

So, I see three options to fix this, which one to use isn't my call to decide:

1. patch spamassassin package, to make sure sa-learn doesn't complain if use_bayes is set to 0

2. patch amavisd-new package, to not run sa-learn if use_bayes is set to 0. There should be a way to query SpamAssassin's configuration from a script - if there is, just use it.

3. add /etc/default/* parameter to disable spamassassin's Bayes filtering, which would require subsequent changes to amavisd-new (don't run sa-learn - but beware of remaining cleanups when this option is set to 0 after some use) and spamassassin (to translate the /etc/default/* to use_bayes 0).

I personally prefer #2 as the cleanest solution: amavisd-new owns the offending cronjob, so amavisd-new is the package that also has to deal with undesired side effects.

[Scott Kitterman (kitterman)]

I'd buy:
> 2. patch amavisd-new package, to not run sa-learn if use_bayes is set to
> 0. There should be a way to query SpamAssassin's configuration from a
> script - if there is, just use it.

Patches welcome.

[Tuomas Jormola (tjormola)]

In order to silence the bayes database sync messages that are frequently sent via cron, could you consider fix along the lines of

--- /usr/sbin/amavisd-new-cronjob 2008-10-11 07:17:30.000000000 +0300
+++ /tmp/amavisd-new-cronjob 2009-01-12 09:20:53.960773192 +0200
@@ -29,7 +29,7 @@

 case $1 in
        sa-sync)
- do_amavis_cmd "/usr/bin/sa-learn --sync >/dev/null"
+ do_amavis_cmd "/usr/bin/sa-learn --sync 2>&1 | egrep -v '^bayes: synced databases from journal in [0-9]+ seconds: [0-9]+ unique entries \([0-9]+ total entries\)$'"
                ;;
        sa-clean)
                do_amavis_cmd "/usr/bin/sa-learn --sync --force-expire >/dev/null"

This would discard the annoying and totally useless sync messages but keep other possible error messages.

[Matthias Andree (matthias-andree)]

That's not a complete solution; since the job keeps complaining if bayes
is disabled.

[Tuomas Jormola (tjormola)]

You're absolutely correct, of course, it's not the complete solution, but would fix the unwanted mails if bayes is enabled.

[Sylvain Vallerot (lulu-fdn)]

Small patch, seems to solves the problem properly (my mind). Just checks about a
line containing something like "use_bayes 0" in Spamassassin config, and then exists.

[Mark Edwards (mark-antsclimbtree)]

I think Sylvain Vallerot's patch (comment #26) addresses this properly. Works well on my system. Please commit!

[John Clarke (jrc61)]

To get rid of the "bayes: synced databases from journal" emails reported in comment 2, try this patch to /usr/bin/sa-learn. It turns off verbose mode (forced on by --sync) which is what causes the output.

[Thierry Carrez (ttx)]

Accepted as a papercut, for a documentation fix.

[Thierry Carrez (ttx)]

Back on track as part of the Maverick papercuts cycle

[David Futcher (bobbo)]

The Debian bug reports this was fixed in package version 2.6.2-1, which has been in Ubuntu since Jaunty. Is this still a bug on recent Ubuntu installs?

[Adam Sommer (asommer)]

I can confirm that the issue is still in Maverick. I've added a subsection to the Amavisd-new section of the Server Guide to add the bayes 0 check from Sylvain's patch on comment #26.

If this information solves the bug please let me know. You can see the new section by checking out the ubuntu-docs source, or checking: http://doc.ubuntu.com/ubuntu/serverguide/C/mail-filtering.html

I believe the site is updated from the latest source daily so it may not be there until tomorrow.

Thanks,
Adam

[Thierry Carrez (ttx)]

@Adam: I can't see the new section...

[Dave Walker (davewalker)]

@sommer: Thanks for updated the documentation to a work around for reducing the unwanted mail. Is there an intention to get an out of the box fix for Maverick?

Thanks.

[rduke15 (rduke15)]

See also Debian bug 447040. In my opinion, the simplest fix is http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=447040#51 :

Add ">dev/null" directly into /etc/cron.d/amavisd-new, at the end of the cron line.

[Daniel Hahler (blueyed)]

This appears to be fixed in 2.6.4-2 (Debian), which needs to get merged for Ubuntu.

I am updating the bug remote watch accordingly.

[webworm (rudy-webworm)]

In fact the problem is not fixed. It's workaround that doesn't give fix the underlying problem.

The real solution would be to check if use_bayes is set to 0. If it is set like that, it shouldn't run the cleaning job.

[webworm (rudy-webworm)]

/usr/sbin/amavisd-new-cronjob could be changed to check if use_bayes is set to 0:

        sa-clean)
                grep -q "use_bayes 0" /etc/spamassassin/local.cf || do_amavis_cmd "/usr/bin/sa-learn --sync --force-expire"
                ;;

Or it can be checked for in cron.daily/amavisd-new in a similar way.