CIFS errors on 4.4.0-98, but not on 4.4.0-97 with same config

Bug #1729337 reported by Jack Hardcastle on 2017-11-01
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
High
Joseph Salisbury
Trusty
High
Joseph Salisbury
Xenial
High
Joseph Salisbury
Zesty
High
Joseph Salisbury
Artful
High
Joseph Salisbury

Bug Description

== SRU Justification ==
The bug reporter stated they have a cluster of servers that applied Xenial updates
and then were unable to mount CIFS shares after upgrading to 4.4.0-98. The
same machines on 4.4.0-97 do not hit the regression. It was found that the
regression is fixed by mainline commit:
4587eee04e2a ("SMB3: Validate negotiate request must always be signed").

This fix is required in all Ubuntu supported releases. Commit 4587eee04e2a
landed in mailine as of 4.14-rc7. It was also cc'd to upstream stable,
but it has not landed in any stable releases yet, which is the reason for
this SRU.

== Fix ==
commit 4587eee04e2ac7ac3ac9fa2bc164fb6e548f99cd
Author: Steve French <email address hidden>
Date: Wed Oct 25 15:58:31 2017 -0500
    SMB3: Validate negotiate request must always be signed

== Regression Potential ==
This patch is to fix a regression. It was also cc'd to upstream stable, so
it received addition review upstream.

We have a cluster of servers that applied a security update overnight and were unable to mount CIFS shares after upgrading to 4.4.0-98. The same machines on 4.4.0-97 were fine the night before, and are fine after downgrading. The only error message CIFS would report, even on verbose, was:

[ 257.089876] CIFS VFS: validate protocol negotiate failed: -11
[ 257.089964] CIFS VFS: cifs_mount failed w/return code = -5

Rebooting did not help. Nor did attempting to mount the share manually using mount -t cifs.

Here's the offending line from our /etc/fstab (with hostnames sanitized):

//server/share /mnt/share cifs rw,user,credentials=/etc/samba/credentials.share,uid=33,gid=33,file_mode=0770,dir_mode=0770
,exec,soft,noserverino,vers=3.0 0 0

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-98-generic 4.4.0-98.121
ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90
Uname: Linux 4.4.0-98-generic x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Nov 1 07:56 seq
 crw-rw---- 1 root audio 116, 33 Nov 1 07:56 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.10
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Date: Wed Nov 1 08:49:47 2017
HibernationDevice: RESUME=/dev/mapper/ubuntu--template--vg-swap_1
InstallationDate: Installed on 2016-12-16 (319 days ago)
InstallationMedia: Ubuntu-Server 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb: Error: command ['lsusb'] failed with exit code 1:
MachineType: VMware, Inc. VMware Virtual Platform
PciMultimedia:

ProcFB: 0 svgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.4.0-98-generic root=/dev/mapper/ubuntu--template--vg-root ro
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-98-generic N/A
 linux-backports-modules-4.4.0-98-generic N/A
 linux-firmware 1.157.13
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/21/2015
dmi.bios.vendor: Phoenix Technologies LTD
dmi.bios.version: 6.00
dmi.board.name: 440BX Desktop Reference Platform
dmi.board.vendor: Intel Corporation
dmi.board.version: None
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 1
dmi.chassis.vendor: No Enclosure
dmi.chassis.version: N/A
dmi.modalias: dmi:bvnPhoenixTechnologiesLTD:bvr6.00:bd09/21/2015:svnVMware,Inc.:pnVMwareVirtualPlatform:pvrNone:rvnIntelCorporation:rn440BXDesktopReferencePlatform:rvrNone:cvnNoEnclosure:ct1:cvrN/A:
dmi.product.name: VMware Virtual Platform
dmi.product.version: None
dmi.sys.vendor: VMware, Inc.

CVE References

Jack Hardcastle (jwhardcastle) wrote :

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Changed in linux (Ubuntu):
importance: Undecided → High
Changed in linux (Ubuntu Xenial):
status: New → Incomplete
importance: Undecided → High
status: Incomplete → In Progress
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu):
assignee: nobody → Joseph Salisbury (jsalisbury)
status: Confirmed → In Progress
Jack Hardcastle (jwhardcastle) wrote :

@jsalisbury I've got one machine from our cluster that's not been downgraded so I can do any debugging or provide any additional information that you might need. Let me know if there's anything I can do to help.

Thanks.

Joseph Salisbury (jsalisbury) wrote :

I built a 16.04 test kernel with the folloiwng commit reverted:

f5c4ba816315 ("cifs: release auth_key.response for reconnect.")

This was the only cifs change between -97 and -98. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1729337/

Can you test this kernel and see if it resolves this bug?

Jack Hardcastle (jwhardcastle) wrote :

Sorry, it doesn't seem to have fixed the issue. I installed the image, extras, and both headers packages. No cloud or tools (they aren't installed in our system usually).

root@client:~# uname -a
Linux client 4.4.0-98-generic #121~lp1729337 SMP Wed Nov 1 14:29:16 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@client:~# dmesg
... snip ...
[ 9.177979] CIFS VFS: validate protocol negotiate failed: -11
[ 9.178594] CIFS VFS: cifs_mount failed w/return code = -5

root@client:~# mount.cifs --verbose //server/share /mnt/share -o rw,user,credentials=/etc/samba/credentials.share,uid=33,gid=33,file_mode=0770,dir_mode=0770,exec,soft,noserverino,vers=3.0
domain=ourdomain
mount.cifs kernel mount options: ip=1.2.3.4,unc=\\server\share,file_mode=0770,dir_mode=0770,soft,noserverino,vers=3.0,uid=33,gid=33,user=www-data,,domain=ourdomain,pass=********
mount error(5): Input/output error

Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
root@client:~#

Joseph Salisbury (jsalisbury) wrote :

Thanks for testing that. We can next perform a kernel bisect to identify the commit that introduced the regression.

First it would be good to know if this bug is already fixed upstream. If it is, we can perform a "Reverse" bisect to identify the commit that fixes the bug. Can you test v4.14-rc7? It can be downloaded from:

http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.14-rc7

FedeX (fedex) wrote :

Hi,

Can confirm I'm having the same issue with multiple servers after latest updates:

Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-98-generic x86_64)
Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-135-generic x86_64)

Also as a workaround changing from vers=3.0 to vers=2.1 fix the issue

Thanks

Jack Hardcastle (jwhardcastle) wrote :

Confirmed that 4.14-rc7 works fine, no error.

root@cf03:~# uname -a
Linux client 4.14.0-041400rc7-generic #201710292231 SMP Sun Oct 29 22:32:07 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@client:~# mount
... snip ...
//server/share on /mnt/share type cifs (rw,nosuid,nodev,relatime,vers=3.0,cache=strict,username=www-data,domain=ourdomain,uid=33,forceuid,gid=33,forcegid,addr=1.2.3.4,file_mode=0770,dir_mode=0770,soft,nounix,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,user)

root@client:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

FedeX (fedex) wrote :

Here are more details of two Ubuntu servers where I'm experiencing this issue:

uname -a
Linux 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty

uname -a
Linux 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

Ubuntu 16.04.3 LTS using kernel 4.4.0-97-generic works OK
Ubuntu 14.04.5 LTS using kernel 3.13.0-133-generic works OK

Let me know if there is extra info I can provide to help out, I've found this which seems related: https://bugzilla.redhat.com/show_bug.cgi?id=1502606

FedeX (fedex) wrote :

I've found another interesting fact by searching the mount.cifs man page, quote:

       sec=
           Security mode. Allowed values are:

           · none - attempt to connection as a null user (no name)

           · krb5 - Use Kerberos version 5 authentication

           · krb5i - Use Kerberos authentication and forcibly enable packet signing

           · ntlm - Use NTLM password hashing

           · ntlmi - Use NTLM password hashing and force packet signing

           · ntlmv2 - Use NTLMv2 password hashing

           · ntlmv2i - Use NTLMv2 password hashing and force packet signing

           · ntlmssp - Use NTLMv2 password hashing encapsulated in Raw NTLMSSP message

           · ntlmsspi - Use NTLMv2 password hashing encapsulated in Raw NTLMSSP message, and force packet signing

           The default in mainline kernel versions prior to v3.8 was sec=ntlm. In v3.8, the default was changed to sec=ntlmssp.

           If the server requires signing during protocol negotiation, then it may be enabled automatically. Packet signing may also be enabled automatically if it's enabled in /proc/fs/cifs/SecurityFlags.

With this in mind, if I go ahead and change the mount to use vers=3.0 again and add option sec=ntlmsspi it works like a charm, see below example:

//x.y.z.w/share /mnt/share cifs vers=3.0,iocharset=utf8,noperm,rw,uid=root,file_mode=0660,dir_mode=0770,credentials=/credentials.file,sec=ntlmsspi 0 0

Above fix works on both Xenial (using 4.4.0-98-generic) and Trusty (using 3.13.0-135-generic)

Jack Hardcastle (jwhardcastle) wrote :

FWIW I can confirm that adding sec=ntlmsspi fixed our issue with 4.4.0-98.

root@client:~# mount | grep share
//server/share on /mnt/share type cifs (rw,nosuid,nodev,relatime,vers=3.0,sec=ntlmsspi,cache=strict,username=www-data,domain=ourdomain,uid=33,forceuid,gid=33,forcegid,addr=10.4.0.30,file_mode=0770,dir_mode=0770,nounix,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,user)

root@client:~# uname -a
Linux client 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Changed in linux (Ubuntu Trusty):
status: New → In Progress
Changed in linux (Ubuntu Zesty):
status: New → In Progress
Changed in linux (Ubuntu Artful):
status: New → In Progress
Changed in linux (Ubuntu Trusty):
importance: Undecided → High
Changed in linux (Ubuntu Zesty):
importance: Undecided → High
Changed in linux (Ubuntu Artful):
importance: Undecided → High
Changed in linux (Ubuntu Trusty):
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Zesty):
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Artful):
assignee: nobody → Joseph Salisbury (jsalisbury)
Joseph Salisbury (jsalisbury) wrote :

I built Xenial, Zesty and Artful test kernels with the following commit:
4587eee SMB3: Validate negotiate request must always be signed

The test kernels can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1729337/

Can you test these kernels and see if that commit resolves this bug?

I tried to build a Trusty test kernel, but it failed to build. I'll do some backporting and post that shortly.

Joseph Salisbury (jsalisbury) wrote :

I also built a Trusty test kernel with commit 4587eee. It can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1729337/trusty

Joseph Salisbury (jsalisbury) wrote :

With all these test kernels, be sure to install both the linux-image and linux-image-extra .deb packages.

Thanks in advance!

Jack Hardcastle (jwhardcastle) wrote :

That did it for me on Xenial.

root@client:~# uname -a
Linux client 4.4.0-98-generic #121~lp1729337 SMP Thu Nov 2 20:53:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

root@client:~# mount | grep share
//server/share on /mnt/share type cifs (rw,nosuid,nodev,relatime,vers=3.0,sec=ntlmssp,cache=strict,username=www-data,domain=ourdomain,uid=33,forceuid,gid=33,forcegid,addr=1.2.3.4,file_mode=0770,dir_mode=0770,nounix,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,user)

Interestingly, as FedeX pointed out, the option (unspecified in my fstab) of sec=ntlmssp has appeared as a new explicit parameter. Makes no difference to me, and it works, just FYI.

description: updated
description: updated
Changed in linux (Ubuntu Artful):
status: In Progress → Fix Committed
Joseph Salisbury (jsalisbury) wrote :

@FedeX

It's interesting that reverting to 3.13.0-133 fixed the issue since there are no cifs related change from 3.13.0-133 to 3.13.0-135.

Also, according to upstream kernel bugzilla 197311, it's mentioned that this was introduced by:
0603c96f upstream ("SMB: Validate negotiate (to protect against downgrade) even if signing off")

However, this commit was not backported to Trusty. Would it be possible for you to assist with a kernel bisect to identify the specific commit that introduced this bug in trusty? It would require you testing 3 - 5 test kernels.

I started a kernel bisect between Ubuntu-3.13.0-133 and Ubuntu-3.13.0-135.

I built the first test kernel, up to the following commit:
5165d87e2242f7d253eca47f4a15ff2cf62eac53

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1729337

If you can assist with a bisect, would it be possible for you to test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Note, just be sure to install both the linux-image and linux-image-extra .deb packages for these kernels.

Thanks in advance

FedeX (fedex) wrote :

Hi @jsalisbury,

Sorry been flat out with work!

Sure can help with that, just to double check please, do I need to download and install all 8 deb packages in?: http://kernel.ubuntu.com/~jsalisbury/lp1729337

Because I don't think I've ever used/installed the cloud-tools in any of my servers though, they are all running in VMWare hypervisor if that helps

Cheers
FedeX

Joseph Salisbury (jsalisbury) wrote :

Only the linux-image and linux-image-extra .deb packages need to be installed.

Stefan Bader (smb) on 2017-11-20
Changed in linux (Ubuntu Xenial):
status: In Progress → Fix Committed
FedeX (fedex) wrote :

Hi Joseph,

Thanks for that

No worries, I'll test this now and report back to you

Cheers
FedeX

FedeX (fedex) wrote :

Hi Joseph,

It seems to be working fine with this kernel, is there any particular info you would like me to provide?

Cheers
FedeX

FedeX (fedex) wrote :

Ok this is very weird, it worked fine after the first reboot, but then I booted once again with 3.13.0-135-generic #184-Ubuntu just to confirm that the same mount options where failing (which they did). Then I rebooted again to 3.13.0-134-generic #183~lp1729337Commit5165d87e224 and the mount it's failing just like with 3.13.0-135-generic #184-Ubuntu

uname -a
Linux ruby2-staging03 3.13.0-134-generic #183~lp1729337Commit5165d87e224 SMP Thu Nov 16 21:22:11 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ sudo mount -o remount -a
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Not sure what to think of this now

FedeX (fedex) wrote :

I've done further tests, and found that with kernels 3.13.0-128, 3.13.0-133, 3.13.0-134 and 3.13.0-135, mounting cifs folder using vers=3.0 (with no security mode manually set, ie: no option sec=ntlmsspi used) sometimes work and sometimes it doesn't, unfortunately I can't figure out what is causing this; if anyone has any ideas please let me know.

It's so weird that sometimes after a fresh boot, I can see that the mount points from the fstab aren't mounted so if I do either sudo mount -a or sudo mount -o remount -a it sometimes crashes and sometimes it works and successfully mount the folders, see below example:

# Fresh boot
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty

# In this example I'm using 3.13.0-128, but same behaviour with newer kernels
$ uname -a
Linux ruby2-staging03 3.13.0-128-generic #177-Ubuntu SMP Tue Aug 8 11:40:23 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ ll /mnt/apps
total 8
drwxr-xr-x 2 root root 4096 Nov 19 2015 ./
drwxr-xr-x 9 root root 4096 Aug 10 17:54 ../

$ sudo mount -a
- OR -
$ sudo mount -o remount -a
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

# try again, could fail every time but sometimes it does succeeds and mount the files:
$ sudo mount -a
- OR -
$ sudo mount -o remount -a

$ ll /mnt/apps
total 25
drwxrwx--- 2 root root 4096 Aug 21 14:38 ./
drwxr-xr-x 9 root root 4096 Aug 10 17:54 ../
-rw-rw---- 1 root root 0 Aug 10 21:04 test
drwxrwx--- 2 root root 0 Oct 21 2015 tmp/

# Relevant entry from /etc/fstab:
//some_server/Applications /mnt/apps cifs vers=3.0,iocharset=utf8,noperm,rw,uid=root,file_mode=0660,dir_mode=0770,credentials=/credentials 0 0

Cheers
FedeX

Khaled El Mously (kmously) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial

Hi @FedeX and @jwhardcastle,

Could you please verify if the Xenial kernel currently in -proposed fixes the issue?

Thank you.

Launchpad Janitor (janitor) wrote :
Download full text (9.5 KiB)

This bug was fixed in the package linux - 4.4.0-103.126

---------------
linux (4.4.0-103.126) xenial; urgency=low

  * linux: 4.4.0-103.126 -proposed tracker (LP: #1736181)

  * CVE-2017-1000405
    - mm, thp: Do not make page table dirty unconditionally in touch_p[mu]d()

  * CVE-2017-16939
    - netlink: add a start callback for starting a netlink dump
    - ipsec: Fix aborted xfrm policy dump crash

linux (4.4.0-102.125) xenial; urgency=low

  * linux: 4.4.0-102.125 -proposed tracker (LP: #1733541)

  * tar -x sometimes fails on overlayfs (LP: #1728489)
    - ovl: check if all layers are on the same fs
    - ovl: persistent inode number for directories

  * NVMe timeout is too short (LP: #1729119)
    - nvme: update timeout module parameter type

  * Set PANIC_TIMEOUT=10 on Power Systems (LP: #1730660)
    - [Config]: Set PANIC_TIMEOUT=10 on ppc64el

  * Cannot pair BLE remote devices when using combo BT SoC (LP: #1731467)
    - Bluetooth: increase timeout for le auto connections

  * CIFS errors on 4.4.0-98, but not on 4.4.0-97 with same config (LP: #1729337)
    - SMB3: Validate negotiate request must always be signed

  * Plantronics P610 does not support sample rate reading (LP: #1719853)
    - ALSA: usb-audio: Add sample rate quirk for Plantronics P610

  * Invalid btree pointer causes the kernel NULL pointer dereference
    (LP: #1729256)
    - xfs: reinit btree pointer on attr tree inactivation walk

  * Samba mount/umount in docker container triggers kernel Oops (LP: #1729637)
    - ipv6: only call ip6_route_dev_notify() once for NETDEV_UNREGISTER
    - ipv6: fix NULL dereference in ip6_route_dev_notify()

  * [kernel] tty/hvc: Use opal irqchip interface if available (LP: #1728098)
    - tty/hvc: Use opal irqchip interface if available

  * Device hotplugging with MPT SAS cannot work for VMWare ESXi (LP: #1730852)
    - scsi: mptsas: Fixup device hotplug for VMWare ESXi

  * NMI watchdog: BUG: soft lockup on Guest upon boot (KVM) (LP: #1727331)
    - KVM: PPC: Book3S: Treat VTB as a per-subcore register, not per-thread

  * Attempt to map rbd image from ceph jewel/luminous hangs (LP: #1728739)
    - crush: ensure bucket id is valid before indexing buckets array
    - crush: ensure take bucket value is valid
    - crush: add chooseleaf_stable tunable
    - crush: decode and initialize chooseleaf_stable
    - libceph: advertise support for TUNABLES5
    - libceph: MOSDOpReply v7 encoding

  * Xenial update to 4.4.98 stable release (LP: #1732698)
    - adv7604: Initialize drive strength to default when using DT
    - video: fbdev: pmag-ba-fb: Remove bad `__init' annotation
    - PCI: mvebu: Handle changes to the bridge windows while enabled
    - xen/netback: set default upper limit of tx/rx queues to 8
    - drm: drm_minor_register(): Clean up debugfs on failure
    - KVM: PPC: Book 3S: XICS: correct the real mode ICP rejecting counter
    - iommu/arm-smmu-v3: Clear prior settings when updating STEs
    - powerpc/corenet: explicitly disable the SDHC controller on kmcoge4
    - ARM: omap2plus_defconfig: Fix probe errors on UARTs 5 and 6
    - crypto: vmx - disable preemption to enable vsx in aes_ctr.c
    - iio: trigger: free trigger...

Read more...

Changed in linux (Ubuntu Xenial):
status: Fix Committed → Fix Released
Jack Hardcastle (jwhardcastle) wrote :
Download full text (11.0 KiB)

I'm happy to create a new bug report for this, however before I do I wanted to follow up here first. I've been working on a bionic VM template this week and the issue has resurfaced. Client (18.04) reboots daily at 3:00 a.m., and somewhere between 30 minutes and 2 hours later, the CIFS mount point stops responding. Meanwhile other clients (16.04, and Windows) continue chugging along merrily. A reboot sometimes fixes the problem, and sometimes the problem has fixed itself by 8am when I arrive. Here's some syslog debug output after the machine finishes booting.

Yesterday it cleared up on its own. Today the server is still down 10 hours later.

I would blame Java, except that the whole mount point becomes non-responsive when this happens, not just for that one process.

Jun 6 03:00:37 localhost systemd[1]: Reached target Multi-User System.
Jun 6 03:00:37 localhost systemd[1]: Starting Execute cloud user/final scripts...
Jun 6 03:00:37 localhost systemd[1]: Reached target Graphical Interface.
Jun 6 03:00:37 localhost systemd[1]: Starting Update UTMP about System Runlevel Changes...
Jun 6 03:00:38 localhost systemd[1]: Started Update UTMP about System Runlevel Changes.
Jun 6 03:00:38 localhost cloud-init[1531]: Cloud-init v. 18.2 running 'modules:final' at Wed, 06 Jun 2018 03:00:38 +0000. Up 23.72 seconds.
Jun 6 03:00:38 localhost cloud-init[1531]: Cloud-init v. 18.2 finished at Wed, 06 Jun 2018 03:00:38 +0000. Datasource DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]. Up 23.84 seconds
Jun 6 03:00:38 localhost systemd[1]: Started Execute cloud user/final scripts.
Jun 6 03:00:38 localhost systemd[1]: Reached target Cloud-init target.
Jun 6 03:00:38 localhost systemd[1]: Startup finished in 2.806s (kernel) + 21.078s (userspace) = 23.885s.
Jun 6 03:00:39 localhost kernel: [ 24.927412] TCP: ens160: Driver has suspect GRO implementation, TCP performance may be compromised.
Jun 6 03:00:51 localhost systemd-timesyncd[574]: Synchronized to time server 91.189.91.157:123 (ntp.ubuntu.com).
Jun 6 03:14:28 localhost systemd[1]: Starting Message of the Day...
Jun 6 03:14:30 localhost 50-motd-news[1699]: * Meltdown, Spectre and Ubuntu: What are the attack vectors,
Jun 6 03:14:30 localhost 50-motd-news[1699]: how the fixes work, and everything else you need to know
Jun 6 03:14:30 localhost 50-motd-news[1699]: - https://ubu.one/u2Know
Jun 6 03:14:30 localhost systemd[1]: Started Message of the Day.
Jun 6 03:15:48 localhost systemd[1]: Starting Cleanup of Temporary Directories...
Jun 6 03:15:48 localhost systemd[1]: Started Cleanup of Temporary Directories.
Jun 6 03:17:01 localhost CRON[1770]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jun 6 04:00:01 localhost CRON[1878]: (root) CMD (/mnt/www/config/backup_config.sh)
Jun 6 04:17:01 localhost CRON[1916]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jun 6 04:17:33 localhost nslcd[1438]: [b141f2] <group/member="root"> failed to bind to LDAP server ldap://dc01.example.com: Can't contact LDAP server
Jun 6 04:17:33 localhost nslcd[1438]: [b141f2] <group/member="root"> connected to LDAP server ldap://dc02.example.com
Jun 6 04:3...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.