Newer kernels are far slower with CIFS mounts

[jgreenso]

Ubuntu 10.04 with uname -a: Linux ubuntu 2.6.32-23-generic-pae #37-Ubuntu SMP Fri Jun 11 09:26:55 UTC 2010 i686 GNU/Linux

grep cifs fstab:
//10.0.0.84/ControlPanel /home/jamesg/ControlPanel cifs noauto,username=jgreen 0 0

The ControlPanel share is a PHP project. I can read the files within it over the CIFS share without any problem using less, vim, etc. However, when I run 'phpunit' nothing happens except on the console where I see:

CIFS VFS: Server not responding
CIFS VFS: No response for cmd 50 for mid 1234

Repeat the No response line every few seconds. If I ctrl+C the phpunit execution it takes a few seconds to quit.

The Ubuntu machine is actually a vmware-server instance. The vmware -server runs in the WinXP machine that hosts the samba share, so that CIFS link is running over a vmware local network. This link should be very fast and was running perfectly fine prior to this new kernel being installed this morning. I'm about to test with the previous kernel.

I have also tried with OplockEnabled set to zero without change. Sometimes (not always) if I try to access the web site itself on the Ubuntu VM, it works, sometimes it never responds.

[jgreenso (james-green-mjog)]

Linux ubuntu 2.6.32-22-generic-pae #36-Ubuntu SMP Thu Jun 3 23:14:23 UTC 2010 i686 GNU/Linux still hangs with console errors this time though I see:

CFS VFS: Send error in FindClose = -9

I can access the projected as an ubuntu-hosted site mounted via CIFS and I can get to the files no problem, but phpunit just hangs there with these console errors slowly trickling out.

[jgreenso (james-green-mjog)]

I tried to use dd if=/dev/zero bs=100 of=ControlPanel/test/foo and ctrl+c'd it after a few seconds. 144M file appeared under windows but dd hung completely. I couldn't even kill -9 the PID as root. I had to reboot Ubuntu with shutdown -r now.

I'm getting in syslog:
Jul 2 11:00:58 ubuntu kernel: [ 952.643403] CIFS VFS: No response for cmd 50 mid 2518
Jul 2 11:01:13 ubuntu kernel: [ 967.721554] CIFS VFS: server not responding
Jul 2 11:01:13 ubuntu kernel: [ 967.722125] CIFS VFS: No response for cmd 50 mid 2647
Jul 2 11:01:13 ubuntu kernel: [ 967.722873] CIFS VFS: Unexpected lookup error -112

This could be a CIFS problem and nothing to do with Ubuntu of course. It seems a little co-incidental that yesterday morning a kernel update come in and since then the share has been unusably slow when trying to operate phpunit.

I've just asked a colleague to mount this share from his XP machine - he managed to copy a multi-megabyte file in without any delay.

[jgreenso (james-green-mjog)]

Similar, if not the same:
http://ubuntuforums.org/showthread.php?t=1467999

[jgreenso (james-green-mjog)]

I've just tried switching back to smbfs in fstab (no change). Then back again to cifs. Then added noserverinfo (no change) then added nounix (no change).

[jgreenso (james-green-mjog)]

Using a remote 9.10 box I mounted the same Windows ControlPanel share and ran phpunit on it. The speed was back to how my VM used to be prior to it's kernel upgrade yesterday morning.

Feels like a regression that's hit my configuration spectacularly badly.

[jgreenso (james-green-mjog)]

I've switched my VM to run Linux ubuntu 2.6.32-23-generic #37-Ubuntu SMP Fri Jun 11 07:54:58 UTC 2010 i686 GNU/Linux and re-ran the phpunit test. It's still slow as a dog (2m30 to do 240 tests is insane).

syslog during the run:
Jul 2 12:41:46 ubuntu kernel: [ 100.435401] CIFS VFS: No response for cmd 50 mid 1185
Jul 2 12:42:01 ubuntu kernel: [ 115.484392] CIFS VFS: No response for cmd 50 mid 1326
Jul 2 12:42:16 ubuntu kernel: [ 130.420782] CIFS VFS: No response for cmd 50 mid 1523
Jul 2 12:42:31 ubuntu kernel: [ 145.485006] CIFS VFS: No response for cmd 50 mid 1731
Jul 2 12:42:46 ubuntu kernel: [ 160.421428] CIFS VFS: No response for cmd 50 mid 1976
Jul 2 12:43:01 ubuntu kernel: [ 175.501242] CIFS VFS: No response for cmd 50 mid 2222
Jul 2 12:43:16 ubuntu kernel: [ 190.422185] CIFS VFS: No response for cmd 50 mid 2430
Jul 2 12:43:31 ubuntu kernel: [ 205.449135] CIFS VFS: No response for cmd 50 mid 2509
Jul 2 12:43:46 ubuntu kernel: [ 220.422707] CIFS VFS: No response for cmd 50 mid 2686

Samba do talk about a 15s timeout when making server requests. But what on earth has happened to cause it?

[jgreenso (james-green-mjog)]

I've now tried with http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/linux-image-2.6.35-999-generic_2.6.35-999.201007021009_i386.deb which continues to suffer.

I've also used Wireshark to examine my XP machine's packets, and I repeatedly see lots of QUERY_PATH_INFO examining the various files in the project. I'm guessing these time out...

[jgreenso (james-green-mjog)]

http://ubuntuforums.org/showthread.php?t=1384092 is possibly another report of the same symptoms.

I have just switched to a much older kernel - 2.6.28-14-server and the problem is far less pronounced. I'm still seeing no response for cmd 50 but repeatedly phpunit is more responsive (48s rather than 4m30s as an example).

[Jeremy Foshee (jeremyfoshee)]

Hi jgreenso,

Please be sure to confirm this issue exists with the latest development release of Ubuntu. ISO CD images are available from http://cdimage.ubuntu.com/daily/current/ . If the issue remains, please run the following command from a Terminal (Applications->Accessories->Terminal). It will automatically gather and attach updated debug information to this report.

apport-collect -p linux 600565

Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[jgreenso (james-green-mjog)]

Already done upstream testing. A cd image of daily build will be more of a challenge.

[speeves (speeves)]

Possible duplicate?
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/471512

[speeves (speeves)]

Sorry, posted that to the wrong bug :P

[Jeremy Foshee (jeremyfoshee)]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu development release http://cdimage.ubuntu.com/daily-live/current/ . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]