maradns init script silently fails during the boot, but works when run manually

[Uqbar]

Binary package hint: maradns

In 9.04, the maradns init script silently fails during the boot.
At the end of the boot sequence, from the command line interface, the init script works when run manually.
This happens 100% of the times. This is how to reproduce the bug.

0.
Description: Ubuntu 9.04
Release: 9.04

1.
maradns:
  Installato: 1.3.07.09-3
  Candidato: 1.3.07.09-3
  Tabella versione:
 *** 1.3.07.09-3 0
        500 http://it.archive.ubuntu.com jaunty/universe Packages
        100 /var/lib/dpkg/status

2. Install maradns (sudo apt-get install maradns)
2.1 the boot init script is set up as S19maradns in /etc/rc[2-5].d, check with "ls -l /etc/rc[2-5].d/*maradns"

3. Reboot the system.

4. Check the maradns daemon is not running but will start gracefully after the boot:
4.1 "sudo /etc/init.d/maradns restart"

[Michael W. Koehler (mkoehle1)]

I am able to reproduce this bug 100% of the time. Thank you for your report.

[Uqbar (uqbar)]

Keep in mind that in Ubuntu 8.04 it was working fine.
I'm not sure, but I think the binary was the same version.

[Uqbar (uqbar)]

I meant Ubuntu 8.10, not 8.04.

[Uqbar (uqbar)]

By the way, this bug tracking system has a bug triggered by this bug!
In Michael's comment the string "bug 100%" triggers an "automatic link detection". I presume the same will happen in this comment.

[Uqbar (uqbar)]

Back to maradns.
In my syslog (daemon) I've found this line:

maradns.etc_maradns_mararc: /usr/sbin/maradns already running.

The use of "_" instead of "/" makes me think that the problem source is in the start-stop-daemon (invocation).
Before doing a manual start, I've tried a manual stop which yields to this message:

Stopping maradns: start-stop-daemon: warning: failed to kill 2623: No such process

Under the previous Ubuntu version (8.10) the package used to work fine. The software version appear to be "almost" the same:

http://packages.ubuntu.com/search?suite=default&section=all&arch=any&searchon=names&keywords=maradns

[Uqbar (uqbar)]

I changed the bug description to a more correct one.

[Uqbar (uqbar)]

By rising the log level, at the boot I find these diagnostics messages in /var/log/daemon:

May 14 09:59:24 g1s maradns.etc_maradns_mararc: Adding root nameserver icann for zone .
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: Root directory changed
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: Binding to address 127.0.0.1
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: Socket opened on UDP port 53
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: Root privileges dropped
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: All RRs have been loaded
May 14 09:59:24 g1s maradns.etc_maradns_mararc: Log: Awaiting data on port 53

I can confirm these are the messages sent to the syslog system at the very boot.
It seems that the daemon is started and then dies somehow.
At the end of the boot I start maradns manually with "sudo /etc/init.d/maradns start" and I found exactly the very same lines in the logs but the last line:

May 14 10:05:02 g1s maradns.etc_maradns_mararc: Adding root nameserver icann for zone .
May 14 10:05:02 g1s maradns.etc_maradns_mararc: Log: Root directory changed
May 14 10:05:02 g1s maradns.etc_maradns_mararc: Log: Binding to address 127.0.0.1
May 14 10:05:02 g1s maradns.etc_maradns_mararc: Log: Socket opened on UDP port 53
May 14 10:05:02 g1s maradns.etc_maradns_mararc: Log: Root privileges dropped
May 14 10:05:02 g1s maradns.etc_maradns_mararc: Log: All RRs have been loaded

So I could argue the problem is not in the script itself but in the binary process.
Any hint?

[Uqbar (uqbar)]

I've done some investigation with the sourcecode.
There's an endless loop in server/MaraDNS.c from which you can exit only with and exit() or a couple of break instructions.
I'v added some extra mlog() calls to identify the exiting point, recompiled the package and substituted to the official binary.
At the reboot I've got the usual syslog messages and maradns failed to start. If I'm right, maradns died sonn after process initialisation.
Maybe this is a real bug in the application but somehow was not happening in Ubuntu 8.10 (same maradns version number!) .

[Uqbar (uqbar)]

Some other tests.
maradns is not starting if I run the initscript in /etc/rc.local which is run/linked as S99rc.local in the rc directories.
This is really weird to me.

[Adamantios (adamand-cc)]

An ugly workaround might be to remove all rc.d references
(update-rc.d -f maradns remove) and then start it from
/etc/network/if-up.d/ .

[Mihnea-Costin Grigore (mihnea-zulu)]

I can confirm this bug happening on Kubuntu 9.04... The logs do not show any strange messages, and in my case they are identical between the "failed" initial start and the subsequent start at the command line. I think the issue is related to intermittent connections -- probably wireless which is started later in the boot, possibly even after login if useing Networkmanager. Can the previous posters confirm they have such a setup? It is true in my case... Any other ideas and debugging info would be welcome.

[Uqbar (uqbar)]

I have been using almost only wireless (but not only).
In 8.10 (https://bugs.launchpad.net/ubuntu/+source/maradns/+bug/370932/comments/2) maradns was working fine.
Later on it stopped working properly.

[Felix Geyer (debfx)]

For me maradns only fails to start when booting with usplash.

[EdC (edc-cce)]

I am also seeing this bug but I don't use wireless networking.

However, as for Felix, removing "splash" from the appropriate kernel line in /boot/grub/menu.lst seems to fix the problem.

Very strange.

[Miika Komu (miika-iki)]

I am one of the maintainers of software called HIP for Linux (HIPL). I tripped into this same issue with HIPL. I solved the problem as follows:

http://<email address hidden>/hipl--userspace--2.6--patch-2564/test/packaging/hipl-deb.spec.diff?diff

Notice that we're using debbuild building system instead of "normal" debian one, but I hope you got an idea. We're starting the service in rcS.d instead of rc2.d. This is really a workaround, not a real solution, but it's better than removing splash.

Is this stuff occurring because of "upstart"?

[Tom Worley (tom-worley)]

This bug also seems to affect 9.10 Karmic.
I can start maradns by hand without any issue, but when I reboot it doesn't come back up.
I've temporarily hacked this by adding this to /etc/rc.local:
service maradns restart

Far from a fix =(

[Mekk (marcin-kasperski)]

I found this bug because I started to observe similar behaviour (10.04): maradns fails to start after reboot, but works flawlessly if started manually later. However, in my case it logs differently, so maybe my case is different:

Mar 2 03:08:20 linode maradns.etc_maradns_mararc: Using default ICANN root servers
Mar 2 03:08:20 linode maradns.etc_maradns_mararc: Log: Root directory changed
Mar 2 03:08:20 linode maradns.etc_maradns_mararc: Fatal error: Problem binding to port 53.
Mar 2 03:08:20 linode maradns.etc_maradns_mararc:
Mar 2 03:08:20 linode maradns.etc_maradns_mararc: System said: Cannot assign requested address

My system receives address via DHCP, so my bet is that maradns attempts to start too early, when system network configuration is not yet ready. Ubuntu version change can impact this by either network manager (which changes the way network starts noticeably) or upstart (which changes whole boot sequence).

Considering ideas above, maybe the best solution would be to convert maradns startup from /etc/init.d script to upstart job which could both be flagged with "start on net-device-up" or sth similar, and annotated with "respawn" just in case...

[Mekk (marcin-kasperski)]

My initial take on the above (to be used instead of /etc/init.d/maradns and /etc/rc* stuff). Warning: not yet tested, I can't reboot machine just now.

======================[ /etc/init/maradns.conf ]==================

# MaraDNS DNS server service

description "MaraDNS"
author "Whoever <[e-mail]>"

start on (net-device-up
          and local-filesystems
          and runlevel [2345])
stop on runlevel [016]

respawn

exec /usr/sbin/maradns -f /etc/maradns/mararc

======================

For completeness, similar file for zoneserver could be created, identical but with

  exec /usr/sbin/zoneserver -f /etc/maradns/mararc

[Phillip Susi (psusi)]

The usplash package has been superseded by plymouth and has been removed from the Ubuntu archive. Closing all related bugs.

[Dariusz Dwornikowski (dariusz-dwornikowski)]

It is fixed in new version.