"init: statd main process ended, respawning" upstart problem

[Cédric Dufour]

statd fails to start during upstart with following messages:
$ fgrep statd /var/log/boot.log
init: statd main process (937) terminated with status 1
[...]
init: statd main process ended, respawning
init: statd main process (1036) terminated with status 1
init: statd respawning too fast, stopped

The following change fixes the issue:
$ diff -u /etc/init/statd.conf{.orig,}
--- /etc/init/statd.conf.orig 2010-04-21 23:04:08.000000000 +0200
+++ /etc/init/statd.conf 2010-04-21 22:52:26.000000000 +0200
@@ -3,7 +3,7 @@
 description "NSM status monitor"
 author "Steve Langasek <email address hidden>"

-start on (started portmap or mounting TYPE=nfs)
+start on ((started portmap or mounting TYPE=nfs) and stopped networking)
 stop on stopping portmap

 expect fork

NB: I'm not sure if I understand it right, but the "stopped networking" condition seems to make upstart wait until all network interfaces are up before starting the service. Can someone confirm this?

Note that I have no NFS mount in /etc/fstab. NFS is mounted via autofs (for partitions that are NOT needed during the boot process).

$ lsb_release -rd; uname -a
Description: Ubuntu lucid (development branch)
Release: 10.04
Linux ced-workstation 2.6.32-21-generic #32-Ubuntu SMP Fri Apr 16 08:09:38 UTC 2010 x86_64 GNU/Linux

$ apt-cache policy nfs-common
nfs-common:
  Installed: 1:1.2.0-4ubuntu4

[Steve Langasek (vorlon)]

Thank you for taking the time to report this bug and help to improve Ubuntu.

The proposed change is wrong, rpc.statd doesn't require the network to be up before it starts. Do you happen to have your /var on a separate filesystem?

[Ali Onur Uyar (aouyar)]

After a recent Lucid upgrade I have been observing the same message at boot.

What is wrong with having a /var on a separate partition? Separating /var is a quite common recommended installation best practice for UNIX / Linux systems. I always try to use LVM to have a small root partition and separate /var, /usr, /home partitions.

[Steve Langasek (vorlon)]

I didn't say anything is *wrong* with having /var on a separate partition. There's simply a (difficult to solve) bug with the statd upstart job when you have your system configured this way.

[Cédric Dufour (cdufour-keyword-ubuntu-086000)]

Thanx Steve for feedback.
Yes, I do have a separate /var partition (LVM). So I guess all I achieved with the "networking" condition is to add enough delay so that /var is mounted when statd starts. Actually, I can confirm the fsck for this /var partition always comes after the "statd main process ended" errors. So I guess it makes this bug a duplicate of the other /var related bugs.
OT: can you point me to the way to enable debug/verbosity in the upstart process? I search for that feature a lot, to no avail.

[Steve Langasek (vorlon)]

Ok, marking as a duplicate of bug #525154 then, thanks.

> OT: can you point me to the way to enable debug/verbosity
> in the upstart process? I search for that feature a lot, to no avail.

You can increase verbosity of upstart itself by booting with '--verbose', or by running 'sudo initctl log-priority info' after boot. This wouldn't give you much more information about why statd was failing, though, which unfortunately doesn't seem to support any -v options.

[Eric Bursley (eric-bursley)]

I'm having this same issue. I also have a /var volume using LVM, but I am mounting two NFS shares in the fstab.
I performed a clean installation using the alternative CD for Ubuntu 10.04 LTS.

[jesse (jesse)]

This is killing me. Even with my NFS shares disabled in fstab, still hangs on boot about /dev/sda5 (/var) and telling me to run fsck manually. I have done it from a rescue CD, but still no help.

[Steve Langasek (vorlon)]

Jesse, that's unrelated to this bug. If you are not running the current Ubuntu 10.04 LTS, you should almost certainly upgrade; if you are running 10.04 LTS and have this problem, you should file a new bug report or inquire on ubuntuforums.org.

[jesse (jesse)]

Steve, no I'm definitely on 10.04. Sorry, actually running fsck manually from rescue again got me booting. I saw the stuff about /var above and thought it was related. I do still get errors on the NFS shares, but they do seem to be mounting on boot. Sorry.