CIFS VFS Server is slowing down shutdown

[Ernst]

Binary package hint: samba

Since I upgraded to Natty, the computer takes a very long time to shut down (approximately 5 minutes). I made a movie from the shutdown messages and I think the CIFS server is not stopped correctly. I see the following message (for a split second):

CIFS VFS: Server xxx.xxx.xxx.xxx has not responded in 300 seconds. Reconnecting...

So, I guess the CIFS server does not quit and therefore the computer is not turned off. Can I do anything to stop CIFS server?

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: samba 2:3.5.8~dfsg-1ubuntu2
ProcVersionSignature: Ubuntu 2.6.38-8.42-generic 2.6.38.2
Uname: Linux 2.6.38-8-generic x86_64
Architecture: amd64
CIFSMounts: Error: command ['mount', '|', 'grep', 'cifs'] failed with exit code 1: mount: only root can do that
Date: Thu Apr 21 20:17:04 2011
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Alpha amd64 (20100122)
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SambaClientRegression: Yes
SourcePackage: samba
UpgradeStatus: Upgraded to natty on 2011-03-26 (26 days ago)

[Ernst (ernst-blaauw)]

The CIFS share is mounted in fstab. If I do "sudo umount /mnt/share", the computer will turn off normally.

[Clint Byrum (clint-fewbar)]

This is not samba, but the CIFS filesystem module in the kernel that you're using to mount the remote file server.

Reassigning to the kernel.

[SecuGuru (christopherthe1)]

I'm seeing similar behavior in my Natty system. Lots of these messages (every 5 minutes, actually) in the syslog:

scorpio kernel: [414810.732082] CIFS VFS: Server leo has not responded in 300 seconds. Reconnecting...

The indicated server "leo" is a Windows XP x64 SP2 machine, and the fstab line to mount it is:
//leo/storage /opt/media/leo cifs ro,credentials=path.to.credfile 0 0

Yes, that's a read-only mount.

[Clint Byrum (clint-fewbar)]

Marking Confirmed and setting priority to Medium now that there are multiple reports.

[Mike Perrin (mperrin)]

This seems to be a zombie bug (aka regression error) I encountered in 8.04 and 9.04 that has returned from the dead. Fortunately there is a fix by Sander Marechal available at
http://www.jejik.com/articles/2007/07/automatically_mounting_and_unmounting_samba_windows_shares_with_cifs/
Sander's script eliminated the shutdown delay on my 11.04 machine using K19umountcifs for the symlinks in rc0.d and rc6.d
I don't have this problem with 10.04 or 10.10 and I don't remember having it with 9.10. Maybe the developers will drive a stake through its heart this time.

[Mike Perrin (mperrin)]

Oops. This isn't as simple as I thought. The shutdown problem came back even with Sander's umountcifs script installed. I discovered that if I use a hard wire network connection shutdown is fine and umountcifs is unnecessary. The delayed shutdown occurs with a wireless network. I speculate that the wireless connection closes too soon and keeps the cifs virtual file system from shutting down cleanly. Can anyone confirm or correct this?

[Ernst (ernst-blaauw)]

I only use a wired network (although a wireless network is available, but I
do not connect to that wireless network), so that's not the cause.

[bdoe (bdoe-att)]

It gets worse than that! It would seem that /etc/init.d/umountnfs.sh is not completing execution, hanging somewhere after evaluating /etc/mtab for nfs/cifs mounts, but before the variable $DIRS itself can be evaluated and acted upon! I ended up taking matters into my own hands and modified umountnfs.sh by adding a line to explicitly unmount my network share, immediately after the mtab evaluation, as so:

exec 9<&0 </etc/mtab

umount //192.168.1.100/MyNET\ Shares/

DIRS=""

As a result, shutdown only takes a few seconds and no longer hangs.

[Clint Byrum (clint-fewbar)]

Excerpts from Ernst's message of Wed May 18 21:01:31 UTC 2011:
> I only use a wired network (although a wireless network is available, but I
> do not connect to that wireless network), so that's not the cause.
>

Is it possible though, that your wired connection is managed through
network-manager where the ones that work are static (as in, listed in
/etc/network/interfaces)?

It would seem that network-manager suffers from a common race that
other services do because it stops when dbus stops, and dbus stops on
runlevel [06]. This means on shutdown, stopping it races with umountnfs.

However, its not clear to me if and/or how NetworkManager de-configures
things on SIGTERM, so this may be a moot point.

[Mike Perrin (mperrin)]

Ernst is correct; after further testing this turns out *not* to be a wireless network issue for me. I am developing a background process that mounts/unmounts shares as defined by users in a configuration file in their home directory. My script interferes with umountcifs when user logout occurs on shutdown or restart. If I have umountcifs installed and I kill my script before shutting down the shutdown occurs normally. When I tested with hardwire I either did not have my script running or it was one of those lucky shots where the two second time delay in my infinite loop occurred at just the right moment to allow umountcifs to do its work. Now I just have to fix my code. My apologies for the red herring.

Ernst - Have you installed umountcifs to determine if it solves your problem?

[Mike Perrin (mperrin)]

It turns out that my three previous posts have been about half right. After further research and testing I have concluded that:
- This is a zombie bug. I recommend this bug be marked as yet another in the long list of duplicate bugs for #211631.
- Sander Marechal's umountcifs is not needed for a workaround.
- The root cause is that Network Manager, controlled by upstart, terminates early and disables the network interfaces before the cifs shares are unmounted. To validate this, I disabled network manager and manually brought up my wireless interface using iwconfig and ifconfig. With two shares mounted a normal shutdown was achieved on ten successive trials.
- Comment #139 in bug #211631 notes that Linux Mint 10 does not have this problem by virtue of relinking umountnfs to K15umountnfs in rc0.d and rc6.d. I validated this by achieving ten successive normal shutdowns with two shares mounted and my wireless connection controlled by Network Manager. (Note that Glen Robinson's comment #139 says umountnfs was originally linked as S43umountnfs whereas my 11.04 installation had umountnfs.sh linked as S31umountnfs.sh. I am unable to explain the discrepancy.) Glen also states that he did not have the problem with a hardwire network, as I reported in comment #7 above.
- Since this is a race (timing) problem, there is no guarantee that either of the above two workarounds would be effective for all situations.
- My cifs management script was interfering with shutdown, confusing my investigation of the original problem . I think I have that fixed now. How unusual - a bug in my code!

[Clint Byrum (clint-fewbar)]

Mike, very nice analysis, and I have to agree fully that this is indeed bug #211631 (its not a zombie though, because that bug has never been fixed).

Thank you to everyone who added to the discussion. Marking as duplicate.