spamd stops autostarting after a release upgrade switch to systemd

[Sorin Sbarnea]

After upgrading Ubuntu from LTS to 15.04, spamassassin daemon does not start anymore because it does not have a proper systemd configuration. Attempts to enable it using `systemctl enable spamassassin` do also fail.

root@atlas:/var/log# systemctl status -l spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
   Loaded: loaded (/lib/systemd/system/spamassassin.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
root@atlas:/var/log# systemctl enable spamassassin.service
Synchronizing state for spamassassin.service with sysvinit using update-rc.d...
Executing /usr/sbin/update-rc.d spamassassin defaults
Executing /usr/sbin/update-rc.d spamassassin enable
root@atlas:/var/log# systemctl status -l spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
   Loaded: loaded (/lib/systemd/system/spamassassin.service; enabled; vendor preset: enabled)
   Active: inactive (dead)

[Ralph Corderoy (ralph-inputplus)]

After 14.10 to 15.04 upgrade, with amavis so spamassassin is installed,
but spamd shouldn't be running IIRC, root is receiving /etc/cron.daily
output.

    /etc/cron.daily/spamassassin:
    Job for spamassassin.service invalid.
    invoke-rc.d: initscript spamassassin, action "reload" failed.

Here, I've

    $ grep '^[^#]' /etc/default/spamassassin
    ENABLED=0
    OPTIONS="--create-prefs --max-children 5 --helper-home-dir"
    PIDFILE="/var/run/spamd.pid"
    CRON=1
    $

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774380 looks related.

[Tony Middleton (ximera)]

I think there might be two separate things here.

I've just had the same problem with a Debian system with spamassassin not loading at boot time. My "systemctl status" was similar to the above ie "disabled". "systemctl enable" didn't start the service but changed the status to enabled and gave output as above. "systemctl start" started the service OK and it now starts at boot.

The second post above refers to a different problem which looks like it is covered by the bug mentioned.

[Christian Ehrhardt  (paelzer)]

The second bug is fixed in the mentioned Debian bug, and the recent merge that went in for Xenial has this one fixed for Ubuntu as well.

The first reported issue is something else:
I retried this on the following paths

In each I configured spam assassin checked that it works as intended.
Trusty:
The trusty systems needed ENABLED=1 in /etc/default/spamassassin as intended.
Note: This was not an upstart job yet, only sysv in /etc/init.d/spamassassin

Wily:
On Wily as the config file states "systemctl enable spamassassin.service" is required - it will then properly start on either next reboot or a manual "systemctl start spamassassin.service". Afterwards on reboots the service came up just correctly.

Everything fine and correct so far.
This defined the starting scenario for the upgrades.
Spamassassin after upgrade
14.04 -> 15.10: not started
14.04 -> 16.04: not started
15.10 -> 16.04: started ok

It seems we "just" need a postinst that transfers the old config, if ENABLED was set to one runs "systemctl enable spamassassin.service".

Since an easy workaround is available it is "only" medium severity, yet it is a regression on upgrading so not lower.

[chris pollock (cpollock)]

I believe I'm seeing the same issue. As far as I know SA is supposed to start at boot time. I've just noticed after a restart that in fact it's not running so I started it by running /etc/init.d/spamassassin start. According to:

chris@localhost:~$ systemctl status spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
   Loaded: loaded (/lib/systemd/system/spamassassin.service; disabled; vendor pr
   Active: active (running) since Mon 2016-08-22 11:52:34 CDT; 9min ago
  Process: 6793 ExecStart=/usr/sbin/spamd -d --pidfile=/var/run/spamassassin.pid
 Main PID: 6796 (/usr/sbin/spamd)
   CGroup: /system.slice/spamassassin.service
           ├─6796 /usr/sbin/spamd -d --pidfile=/var/run/spamassassin.pid --creat
           ├─6803 spamd chil
           └─6804 spamd chil

Aug 22 11:52:34 localhost spamd[6796]: pyzor: [6801] error: TERMINATED, signal 1
Aug 22 11:52:34 localhost spamd[6796]: rules: meta test L_UNVERIFIED_YAHOO has d
Aug 22 11:52:34 localhost spamd[6796]: rules: meta test L_UNVERIFIED_GMAIL has d
Aug 22 11:52:34 localhost spamd[6796]: spamd: server started on IO::Socket::IP [
Aug 22 11:52:34 localhost spamd[6796]: spamd: server pid: 6796
Aug 22 11:52:34 localhost spamd[6796]: spamd: server successfully spawned child
Aug 22 11:52:34 localhost spamd[6796]: spamd: server successfully spawned child
Aug 22 11:52:34 localhost spamd[6796]: prefork: child states: IS
Aug 22 11:52:34 localhost spamd[6796]: prefork: child states: II
Aug 22 11:52:34 localhost systemd[1]: Started Perl-based spam filter using text

it is running. I looked at the /etc/default/spamassassin file

# /etc/default/spamassassin
# Duncan Findlay

# WARNING: please read README.spamd before using.
# There may be security risks.

# Change to one to enable spamd
ENABLED=1

and if I understand correctly it should start at boot.

chris@localhost:~$ lsb_release -rd
Description: Ubuntu 16.04.1 LTS
Release: 16.04

chris@localhost:~$ apt-cache policy spamassassin
spamassassin:
  Installed: 3.4.1-3
  Candidate: 3.4.1-3
  Version table:
 *** 3.4.1-3 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu xenial/main i386 Packages
        100 /var/lib/dpkg/status

[Tim Riker (timriker)]

should /etc/default/spamassassin have ENABLED=1 after an upgrade to 16.04 using systemd? or does this entry no longer matter? /etc/init.d/spamassassin seems to still look at the value.

$ systemctl status spamassassin

showed enabled, but not running. I manually started it.

[Christian Ehrhardt  (paelzer)]

That is really odd.
Maybe it has an issue with its dependencies, fails to start at boot due to that but otherwise runs just fine.

Just installed again, after install/upgrade it seems fine:
systemctl status spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
   Loaded: loaded (/lib/systemd/system/spamassassin.service; disabled; vendor preset: enabled)
   Active: active (running) since Fri 2016-12-16 08:15:48 UTC; 25s ago
 Main PID: 15901 (/usr/sbin/spamd)
   CGroup: /system.slice/spamassassin.service
           ├─15901 /usr/sbin/spamd -d --pidfile=/var/run/spamassassin.pid --create-prefs --max-children 5 --helpe
           ├─15902 spamd child
           └─15903 spamd child

Dec 16 08:15:46 zesty-test-commands systemd[1]: spamassassin.service: Failed to set invocation ID on control grou
Dec 16 08:15:46 zesty-test-commands systemd[1]: Starting Perl-based spam filter using text analysis...
Dec 16 08:15:47 zesty-test-commands spamd[15899]: logger: removing stderr method
Dec 16 08:15:48 zesty-test-commands spamd[15901]: spamd: server started on IO::Socket::IP [::1]:783, IO::Socket::
Dec 16 08:15:48 zesty-test-commands spamd[15901]: spamd: server pid: 15901
Dec 16 08:15:48 zesty-test-commands spamd[15901]: spamd: server successfully spawned child process, pid 15902
Dec 16 08:15:48 zesty-test-commands spamd[15901]: spamd: server successfully spawned child process, pid 15903
Dec 16 08:15:48 zesty-test-commands spamd[15901]: prefork: child states: IS
Dec 16 08:15:48 zesty-test-commands systemd[1]: Started Perl-based spam filter using text analysis.
Dec 16 08:15:48 zesty-test-commands spamd[15901]: prefork: child states: II

Then rebooting said system:
systemctl status spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
   Loaded: loaded (/lib/systemd/system/spamassassin.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

A "systemctl start spamassassin.service" gets it running again.
So something really is broken.

Note: all this can be done in a container - so easy to reproduce.

Raising priority as this could mean loosing spam protection or failing mail config.

[Christian Ehrhardt  (paelzer)]

It doesn't seem like dependencies so much:
  WantedBy=multi-user.target

That looks pretty safe.

But I saw that on status it is:
disabled; vendor preset: enabled
Check with:
$ systemctl is-enabled spamassassin
disabled

So maybe the postinst misses to enable it properly on systemd?

[Christian Ehrhardt  (paelzer)]

I just checked on Debian-sid, there the case is just the same.
So it is not an issue special to Ubuntu (spamassassin has no delta itself, but that also means that no other bits in the system trigger it).

[Christian Ehrhardt  (paelzer)]

Note: as suggested before - yes the systemd service file seems not to care about ENABLED in /etc/default/spamassassin (it considers the OPTIONS thou).

Cron option is considered by /etc/cron.daily/spamassassin - so "only" ENABLED seems lost.

And more important for this issue, as installed by default the service gets started but not enabled. A simple
$ systemctl enable spamassassin.service
fixes that up for all of you as workaround now.

This bug is present in Debian too, and Ubuntu currently doesn't make any changes over the Debian package. So this bug would be best fixed directly in Debian, and then Ubuntu will pick up the fix automatically.

Would you mind filing a bug with Debian please?
Doing so you can focus on just "spamassassin service not enabled after installation and therefore not started after reboot"

[Christian Ehrhardt  (paelzer)]

Note:
the config as delivered these days also has this comment right above the ENABLED statement:
# If you're using systemd (default for jessie), the ENABLED setting is
# not used. Instead, enable spamd by issuing:
# systemctl enable spamassassin.service
# Change to "1" to enable spamd on systems using sysvinit:

It also is now enabled by default after a fresh install on version 3.4.1-6 - I haven't tracked down the change and if a upgrade from pre-systemd to this would keep it enabled (or just always be enabled).
If this fixes the issue (need to identify the change) we might want to backport to Xenial as all pre-systemd upgrade paths will go through there.

Oherwise we really "just" need to fix that if an old config has "ENABLED=1" it does enable the service in the postinst on upgrade - and that fix we should share with (or better directly create in) Debian - see the last post.

[Robie Basak (racb)]

13:25 <rbasak> cpaelzer: I think I follow. Let me check. In the past, the user was supposed to set ENABLED=1 in /etc/default/spamassassin. Now, the user is supposed to run "systemctl enable spamassassin.service" instead. So, on upgrade, the state goes to enabled to disabled as there is no upgrade path. Correct?

13:25 <cpaelzer> rbasak: exactly

[Robie Basak (racb)]

I've sent this up to Debian. Also:

spamd is not enabled by default, so this is an unusual end-user configuration

On upgrade you should see a conffile prompt containing a comment telling you what you need to do for systemd, unless you are upgrading such that this message is suppressed.

The workaround is trivial

@Sorin: in your original report, you said that "systemctl enable spamassassin.service" doesn't work, but you did not attempt to start the service after you enabled it which is a separate action. So I don't think there's a bug there. Run "systemctl start spamassassin.service" and I think you should be fine. Or, if that doesn't work, please file a separate bug since in that case there are two issues and we need to track them independently.

So I think Importance: Low is appropriate for this particular issue now that we have figured out what is going on. I also don't think this warrants an upgrade path fix in Ubuntu by diverging from Debian. So unless Debian decides to do otherwise, I don't think any action for Ubuntu is appropriate, so I'll set the bug to Won't Fix for now. This is open to discussion and change if someone disagrees and is prepared to propose a patch.

[Sorin Sbarnea (ssbarnea)]

I have no further interest in pursuing this bug, feel free to close it or to allow someone else to take care of it. I muted it.