karmic: nfs shares are not mounted at boot

[Alvin]

Binary package hint: mountall

After an update to karmic, NFS shares are no longer mounted at boot time.

Afterwards, filesystems can be mounted with:
$ sudo mount -t -a nfs

[Scott James Remnant (Canonical) (canonical-scott)]

Please supply output of "sudo mountall --debug" and send it SIGUSR1

[Alvin (alvind)]

Output of: sudo mountall --debug > mountall.log 2>&1
(mountpoints are in /media instead of /srv, as described in first /etc/fstab)

[Steve Langasek (vorlon)]

End of the log shows:

try_mount: /media/xyimages waiting for /media/xydata
One or more of the mounts listed in /etc/fstab cannot yet be mounted:
(ESC for recovery shell)
/media/archive: waiting for back:/archive
/media/backup: waiting for coolio:/backup
/home/roger/sh_home: waiting for coolio:/home/roger
/media/mac: waiting for coolio:/mac/es
/media/parkeer: waiting for coolio:/parkeer
/media/pdf: waiting for coolio:/spool/AC
/media/software: waiting for coolio:/software
/media/xchange: waiting for coolio:/Xchange
/media/xydata7: waiting for coolio:/xydata
/media/xydata: waiting for coolio:/xydata8
/media/xyimages: waiting for coolio:/xyimages
int_handler: Received SIGINT

mountall: Cancelled

Doesn't look like an nfs-utils bug? Reassigning back to mountall.

[Scott James Remnant (Canonical) (canonical-scott)]

The log doesn't show SIGUSR1 being received

[Jason Straight (jason-jeetkunedomaster)]

I think what's going on here is that the first mountall isn't omitting network filesystems like it should be. I could be wrong - I don't know how to monitor upstart in a way to debug which of the various mountalls are being run when it hangs.

However if the network and portmap aren't up then this is usually what happens.

[daemacles (daemacles)]

I'm having the same problem. At boot mountall hangs and drops me into a service shell. I can start portmap/statd manually, manually mount my NFS share. Manually remount / (root directory, a local disk) as read-write, then boot continues. If I don't manually remount / then mountall just sits there waiting for it to mount...which it already is.

[Paul de Vries (phs-de-vries)]

I have the same issue, nfs shares not mounted at boot.

Listing the /proc/filesystems does not show the nfs filesystem listed, which might be the reason the nfs shares are not mounted?

After running sudo mount -a -t nfs the shares are mounted. And running cat /proc/filesystems does now have nfs and nfs4 included.

[Albrecht Dreß (albrecht-dress)]

I can confirm this bug, on a box updated from 9.04 (which, needless to mention, booted flawlessly). As I'm also dropped to the boot service shell, I made all nfs drives "noauto" and wrote a shell script do perform it at the end of the boot process.

[flowerdealer (pixelcowboy79)]

We also get this, if we wait for a while the system boots correctly with the nfs mounts mounted, but it does take quite a while to do so.

[Alvin (alvind)]

Scott, I did sent SIGUSR1 (using htop). Maybe I'm doing it wrong though.

Meanwhile, I'm seeing this problem all networks under my care and I'm using Albrecht's workaround for them.
(I have now seen this problem on 5 different machines in 3 different networks.)

The big problem is: Sometimes /home (on NFS) does not get mounted and the boot process halts like in Bug #431248.
I did notice a difference when using dhcp or a static IP. DHCP seems to work. The error messages remain, but the shares get mounted.
When using a static address, NFS shares don't mount. (and in the case of /home, fail booting altogether.)

[Scott James Remnant (Canonical) (canonical-scott)]

I wonder whether this and bug #470776 are the same, ie. SIGUSR1 is being received but the device isn't ready due to underlying mountpoints - and then when the underlying mountpoints are ready, there's no more SIGUSR1

[Alvin (alvind)]

The issue of the static ip address is bug #446031, so I'm using dhcp on the servers now. I was also using a separate /boot that added to the confusion. (bug #462961, now fixed.)
Setting the NFS clients to dhcp did solve this issue! I rechecked by setting a static ip again.

I will use this as a workaround now, but I'd prefer the servers to use static ip's. Using dhcp will make sure the computer boots and NFS mounts are available, but the list of error messages remain during boot:
One or more of the mounts listed in /etc/fstab cannot yet be mounted:
 (ESC for recovery shell)
etc....
This slows down the boot process.

Also, when a netfs storage pool is defined for libvirtd, libvirtd does not succeed in starting that pool at boot. Could this be related? (restarting the libvirtd service after boot will mount the drives)

[Paul de Vries (phs-de-vries)]

Can confirm that setting the NFS clients to dhcp does solve this issue for me as well.

[mkalkbrenner (markus-kalkbrenner)]

I ran into the same problem as I switched from DHCP to a static IP.

[mkalkbrenner (markus-kalkbrenner)]

... and my mythtv backend doesn't start automatically anymore.

DHCP => nfs mounts via fstab and mythtv backend works
static IP => nfs shares need to be mounted by hand and mythtv-backend has to be started by hand

[Mikael Ståldal (mikaelstaldal)]

I also get this problem. However, it doesn't even work when setting to "noauto" in /etc/fstab and then try to mount it from /etc/rc.local, I get this error message:
mount.nfs: DNS resolution failed for myserver: Name or service not known

But it does work to mount it from /etc/gdm/PostLogin/Default.

I use DHCP. I suspect that NFS mounting does not wait for the DHCP lookup to finish as it should.

[Mikael Ståldal (mikaelstaldal)]

I use the setup described here:
http://www.staldal.nu/tech/2009/11/15/linux-with-mounted-read-only-2-0/

[Alvin (alvind)]

This bug report is marked for expiration. What information is missing?

[ldr (leon-toyos)]

I seem to have exactly the same problems. I first upgraded a server from jaunty to karmic, and this morning decided to give the machine a clean install (karmic server 32bits) - with only openssh, nfs-common and portmap packages installed.

I have only put the following line in /etc/fstab:

172.16.1.250:/staff /nfs/staff nfs rw,rsize=8192,wsize=8192,soft,bg,intr,nodev,nosuid,noexec 1 1

(this has always worked before)

--

The mountall program then hangs. When I kill it from the commandline and start it manually with arguments --verbose --debug, it ends with the following output:

...
swap finished
mounted: local 2/2 remote 0/0 virtual 11/11 swap 1/1
try_mount: /nfs/staff waiting for device 172.16.1.250:/staff
try_udev_device: block /dev/sda5 dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6 (null)
try_udev_device: none by link /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6
queue_fsck: /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6: already ready
run_swapon: /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6: already activated

And mountall hangs again.

--

I also tried stracing on pid the mountall program, to see what it's doing, but it just loops the following:

waitid(P_ALL, 0, 0xbf88b998, WNOHANG|WEXITED|WSTOPPED|WCONTINUED, NULL) = -1 ECHILD (No child processes)
clock_gettime(CLOCK_MONOTONIC, {12981, 496289054}) = 0
clock_gettime(CLOCK_MONOTONIC, {12981, 496350970}) = 0
select(8, [0 3 4 6], [], [3 7], {1, 0}) = 0 (Timeout)
read(4, 0xbf88bbff, 1) = -1 EAGAIN (Resource temporarily unavailable)

--

Does anyone know if this is the same problem, or what I should do to get the NFS mount working ?

I forgot to mention: when doing a 'mount -a' the nfs gets mounted properly..

Thanks,

Leon

[ldr (leon-toyos)]

I just tried sending the SIGUSR1 when mountall was running, and then it works:

root@envy:~# mountall --debug --verbose
mount / [15647] exited normally
mount /proc [15648] exited normally
mount /sys [15649] exited normally
mount /sys/fs/fuse/connections [15650] exited normally
mount /sys/kernel/debug [15651] exited normally
mount /sys/kernel/security [15652] exited normally
mount /dev [15653] exited normally
mount /dev/pts [15654] exited normally
mount /dev/shm [15655] exited normally
mount /var/run [15656] exited normally
mount /var/lock [15657] exited normally
mount /lib/init/rw [15658] exited normally
virtual finished
local finished
fhs mounted
activating /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6
swapon: /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6: swapon failed: Device or resource busy
mountall: swapon /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6 [15667] terminated with status 255
mountall: Problem activating swap: /dev/disk/by-uuid/dc57d7a3-15a6-48dd-b047-b6ce35e2b2c6
swap finished

<<< at this time, from a different console I sent the SIGUSR1 >>>

checking /nfs/staff
fsck from util-linux-ng 2.16
fsck /nfs/staff [15980] exited normally
mounting /nfs/staff
mount /nfs/staff [15981] exited normally

--

So question remains, what is the reason this doesn't work at boot time ?

[damianos (damian-linux)]

So in a nut shell you can't have a DHCP server on karmic that mounts NFS filesystems at startup.

This is biting me on three boxes. I could set fixed IP addresses in my DHCP server for the nodes in questions but this is just a workaround.

[Alvin (alvind)]

I can't judge the importance of this bug for others, but should it not be mentioned in ubuntu-release-notes?

[Scott James Remnant (Canonical) (canonical-scott)]

The difference between static IP and DHCP is all down to timing, which would strongly suggest you're all experiencing the bug described in #470776 - that mountall gets told the network is up, but isn't ready to mount the NFS partitions yet - then when it *is* ready, waits for the network to come up (ie. has forgotten that it already came up)

[bjo (bjo81)]

In my case, mountall mounted my nfsshares after establishing a network connection in karmic. Now in lucid, mountall does not mount any nfsshare. The shares are not needed at boottime.

/etc/fstab:
192.168.0.20:/home/bjo /mnt/bjo nfs4 defaults 0 0
192.168.0.20:/data /mnt/daten1 nfs4 defaults 0 0
192.168.0.20:/data2 /mnt/daten2 nfs4 defaults 0 0
192.168.0.20:/data3 /mnt/daten3 nfs4 defaults 0 0

dpkg -l mountall:
ii mountall 2.4 filesystem mounting tool