CIFS client: access problems after updating to kernel 4.4.0-29-generic

[Chris van Run]

SRU Justification

Impact: A regression in the xenial kernel causes EPERM to be returned when accessing dfs subfilesystems in a cifs mount.

Fix: Upstream cherry pick which uses the appropriate credentials for these automounts.

Regression Potential: The fix involves reverting a sauce patch to fix a similar problem in nfs, but it's replace with a more generic fix which fixes cifs as well. I have tested with both nfs and cifs submounts and confirmed that both work as expected.

---

Expected: being able to mount subfilesystems via cifs mount.
What happens: permission denied even though there is permission

I am denied access to the subfilesystems on a samba server. The top level filesystem itself is unaffected but dfs links seem to run into 'Operation not permitted' problems. When accessing the subfilesystem on the mount as root it mounts correctly. The standard user is then also able to access the subfilesystem.

This only happens when the client uses kernel 4.4.0-34 (#53). When I revert my client to an older kernel (4.4.0-21) everything works. Using the older kernel the normal user can directly ls into the subfilesystem.

This is bug is very similar to bug #1603719 (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1603719). However, the released fix (in 4.4.0-33.52) does not seem to fix it for cifs.

Extra info:
Ubuntu 16.04.1 LTS
cifs-utils version: 2:6.4-1ubuntu1
Mount options are minimal: sec=krb5,uid=XXXX,cruid=XXX

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: ubuntu-release-upgrader-core 1:16.04.15
ProcVersionSignature: Ubuntu 4.4.0-34.53-generic 4.4.15
Uname: Linux 4.4.0-34-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.1
Architecture: amd64
CrashDB: ubuntu
CurrentDesktop: KDE
Date: Thu Aug 11 10:32:20 2016
InstallationDate: Installed on 2016-04-28 (104 days ago)
InstallationMedia: Kubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.update-motd.d.91-release-upgrade: [deleted]

[Chris van Run (christehrun)]

Upstream Kernel v 4.4.14-040414 does not have this problem. Latest kernel (4.8-rc1 runs into other problems).

[Chris van Run (christehrun)]

Think this problem is related to the linux kernel rather than cifs-utils package.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1612135

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Chris van Run (christehrun)]

Report was done via apport and hence should contain all the logs required. Also, apport-collect does not like 'linux' as package name =/.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream 4.4 stable kernel? It can be downloaded from:

http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4.17

If it still has the bug, we can bisect between 4.4.14 and 4.4.15.

[Chris van Run (christehrun)]

Dear reader,

Tested the latest upstream 4.4 stable kernel. All seems well in upstream. It is the current LTS kernel version that seems to have the glitch.

To summarize:
4.4.17 : no problem. (Commit: 133cec911c639d2cdf544ed602442951f702e08c)
4.4.15 : no problem. (Commit: 35467dc7630af60abacc330f64029d081f160530)
4.4.14 : no problem. (Commit: 9ca1d50fa959cda1f04f43275f7930a70f1a631e)

Packaged LTS (xenial) versions:
4.4.0-34 #53: problem with cifs.
4.4.0-21 #37: no problem, everything is Dandy.

Cheers, Chris

[Joseph Salisbury (jsalisbury)]

Thanks for the update. It sounds like the bug is due to an Ubuntu SAUCE patch. Can you test the latest Xenial -proposed kernel?

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed.

If -proposed still has the bug, we can bisect between 4.4.0-21 and 4.4.0-34.

[Ian Gordon (ian-gordon)]

I have the same problem.

I do not experience the problem with kernel 4.4.0-28 but do with 4.4.0-31 and above.

If you access the CIFS share (which contains DFS referrals) as root it works fine - but not as a normal user.

[Ian Gordon (ian-gordon)]

I can confirm that the problem still exists in the latest proposed kernel 4.4.0-36

[Joseph Salisbury (jsalisbury)]

I'd like to start a bisect, but we need to narrow down the last good kernel version and the first bad one. Can you also test the 4.4.0-29 and 4.4.0-30 kernels?

4.4.0-29: https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/10184098

4.4.0-30: https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/10195224

Note, with these test kernels you need to install both the linux-image and linux-image-extra .deb packages.

If you find 4.4.0-29 has the bug, then there is no need to test 4.4.0-30.

Thanks in advance!

[Ian Gordon (ian-gordon)]

The bug is present in 4.4.0-29

Cheers,

Ian.

[Chris van Run (christehrun)]

I tested 4.4.0-29 and can confirm it has the bug. For good measure I also tested 4.4.0-28 (I did 22 previously) and can confirm that the bug is not present in version 28.

I think we found the culprit in what ever happened between 28 and 29.

Cheers, Chris

[Joseph Salisbury (jsalisbury)]

I started a kernel bisect between Ubuntu-4.4.0-28.47 and Ubuntu-4.4.0-29.48. The kernel bisect will require testing of about 7-10 test kernels.

I built the first test kernel, up to the following commit:
465a7e95502dd08f44d0c01e5dccf142bc75e1f2

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1612135

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance!

[Ian Gordon (ian-gordon)]

This kernel (465a7e95502dd08f44d0c01e5dccf142bc75e1f2) does NOT have the bug.
Next kernel please :)

Cheers.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
a5abdcb33ee353704e558a1368ccadb3c4d6cf78

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1612135

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance!

[Chris van Run (christehrun)]

This next test kernel (a5abdcb33ee353704e558a1368ccadb3c4d6cf78) does NOT have the bug. Looking forward to the next! Cheers.

[Chris van Run (christehrun)]

Looking forward to the next kernel test build.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
c5bf368a270f1ae468ae27f2e6527d40381dda5d

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1612135

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance!

[Ian Gordon (ian-gordon)]

This new test kernel (commit c5bf368a270f1ae468ae27f2e6527d40381dda5d) does have the bug.
You have found the culprit commit.

Cheers,

Ian.

[Joseph Salisbury (jsalisbury)]

@Ian, there are a couple more kernels to test before the exact commit is identified. I'll build the next kernel and post it shortly.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
302cabb739eb88e4c6c6ea50fce2a1f9201e2147

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1612135

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance!

[Ian Gordon (ian-gordon)]

This new test kernel (commit 302cabb739eb88e4c6c6ea50fce2a1f9201e2147) DOES have the bug.
Next test please...

[Joseph Salisbury (jsalisbury)]

The bisect reported 302cabb739eb88e4c6c6ea50fce2a1f9201e2147 as the first bad commit. I built a Xenial test kernel with this commit reverted. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1612135

Can you test that kernel and report back if it has the bug or not?

Note, you will need to install both the linux-image and linux-image-extra .deb packages for this kernel.

Thanks

[Ian Gordon (ian-gordon)]

This new test kernel is not there. The URL is an empty directory.

Cheers,

Ian.

[Joseph Salisbury (jsalisbury)]

Sorry, it should be there now.

[Ian Gordon (ian-gordon)]

The kernel in comment #24 does NOT have the bug. :)

Cheers,

Ian Gordon.

[Walter Mautner (retlaw-rentuam)]

I can confirm, I installed the kernel from #24 on two AD-joined ubuntu-14.04 machines (after running into the problem since 4.40-34, been "forced" to update since the latest 4.2.0-42 tells me it's EOL) and it appears to work just fine :)

Thanks, Greetings from Vienna
Walter

[Seth Forshee (sforshee)]

Please test this kernel and see if it fixes the issue. Thanks!

http://people.canonical.com/~sforshee/lp1612135/

[Seth Forshee (sforshee)]

I was able to reproduce the bug and test the fix, and it does work. Will send for Xenial SRU.

[Tim Gardner (timg-tpi)]

commit aeaa4a79ff6a5ed912b7362f206cf8576fca538b merged in v4.8-rc1

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-38.57

---------------
linux (4.4.0-38.57) xenial; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1620658

  * CIFS client: access problems after updating to kernel 4.4.0-29-generic
    (LP: #1612135)
    - Revert "UBUNTU: SAUCE: (namespace) Bypass sget() capability check for nfs"
    - fs: Call d_automount with the filesystems creds

  * apt-key add fails in overlayfs (LP: #1618572)
    - SAUCE: overlayfs: fix regression in whiteout detection

linux (4.4.0-37.56) xenial; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1618040

  * [Feature] Instruction decoder support for new SKX instructions- AVX512
    (LP: #1591655)
    - x86/insn: perf tools: Fix vcvtph2ps instruction decoding
    - x86/insn: Add AVX-512 support to the instruction decoder
    - perf tools: Add AVX-512 support to the instruction decoder used by Intel PT
    - perf tools: Add AVX-512 instructions to the new instructions test

  * [Ubuntu 16.04] FCoE Lun not visible in OS with inbox driver - Issue with
    ioremap() call on 32bit kernel (LP: #1608652)
    - lpfc: Correct issue with ioremap() call on 32bit kernel

  * [Feature] turbostat support for Skylake-SP server (LP: #1591802)
    - tools/power turbostat: decode more CPUID fields
    - tools/power turbostat: CPUID(0x16) leaf shows base, max, and bus frequency
    - tools/power turbostat: decode HWP registers
    - tools/power turbostat: Decode MSR_MISC_PWR_MGMT
    - tools/power turbostat: allow sub-sec intervals
    - tools/power turbostat: Intel Xeon x200: fix erroneous bclk value
    - tools/power turbostat: Intel Xeon x200: fix turbo-ratio decoding
    - tools/power turbostat: re-name "%Busy" field to "Busy%"
    - tools/power turbostat: add --out option for saving output in a file
    - tools/power turbostat: fix compiler warnings
    - tools/power turbostat: make fewer systems calls
    - tools/power turbostat: show IRQs per CPU
    - tools/power turbostat: show GFXMHz
    - tools/power turbostat: show GFX%rc6
    - tools/power turbostat: detect and work around syscall jitter
    - tools/power turbostat: indicate SMX and SGX support
    - tools/power turbostat: call __cpuid() instead of __get_cpuid()
    - tools/power turbostat: correct output for MSR_NHM_SNB_PKG_CST_CFG_CTL dump
    - tools/power turbostat: bugfix: TDP MSRs print bits fixing
    - tools/power turbostat: SGX state should print only if --debug
    - tools/power turbostat: print IRTL MSRs
    - tools/power turbostat: initial BXT support
    - tools/power turbostat: decode BXT TSC frequency via CPUID
    - tools/power turbostat: initial SKX support

  * [BYT] display hotplug doesn't work on console (LP: #1616894)
    - drm/i915/vlv: Make intel_crt_reset() per-encoder
    - drm/i915/vlv: Reset the ADPA in vlv_display_power_well_init()
    - drm/i915/vlv: Disable HPD in valleyview_crt_detect_hotplug()
    - drm/i915: Enable polling when we don't have hpd

  * [Feature]intel_idle enabling on Broxton-P (LP: #1520446)
    - intel_idle: add BXT support

  * [Feature] EDAC: Update driver for SKX-SP (LP: #1591815)
    - [Config] CONFIG_EDAC_SKX=m
    - EDAC, skx_edac: Ad...

[Nico (nicom66)]

Hello I've the kernel version 4.4.0-38 and I'm experiencing the same problem mounting CIFS:

ls: cannot open directory '.': Permission denied

But even with root I can't list the directory:

ls: reading directory '.': Object is remote

This is a bug with high impact and need to solve as soon as possible.

Thanks for help

[Seth Forshee (sforshee)]

@Nico: I just tested 4.4.0-38 and the cifs automount regression reported here does seem to be fixed. What you're experiencing must be different somehow, so please open a new bug by running 'ubuntu-bug linux' in a terminal. Include as much information as possible about your environment to help us determine the root cause. You can subscribe me to the bug. Thanks!

[Nico (nicom66)]

Hello @sforshee,

I'm sorry but in my case does not work. I tried with kernel 4.4.0.36 and it works correctly, so it's a kernel problem for sure.

The command for mounting is:

sudo mount -t cifs //windowsdomain.ad/carpetes/documents /home/nico/mnt/carpetes -o user=nico,password=mypassword,domain=windowsdomain.ad,sec=ntlm,file_mode=0777,dir_mode=0777

After run the command themount is done but the problem is that I can't access to the files, even when I can surf into the root directory /home/nico/mnt/carpetes.

Thanks a lot for your help,

Nico

[Seth Forshee (sforshee)]

@Nico: I'm not denying that you're seeing a problem, I just do not believe it to be the same as the problem originally reported on this bug. Your last comment confirms this - the problem reported here is accessing a dfs sub-filesystem, and you appear to be having problems accessing the top-level cifs mount.

That is why I am asking you to file a new bug. Please do so, and I will help you there.

[Nico (nicom66)]

Ok @sforshee,

I'm sorry. I'll open a new problem.

Thank you very much.

[Walter Mautner (retlaw-rentuam)]

Unfortunately - while the dfs subdirectories on one share do work now, mounting the homeshare doesn't ... "Object is remote".
Since obviously the patch for the mentioned dfs problem adversely affacts "homeshare" accessibility, it is at least a followup-regression.

[Seth Forshee (sforshee)]

I suspect that the regression is separate from the fix for this bug since the fix only applies to automounts, but let's find out for sure. I built 4.4.0-38 with the fix reverted. Please install the kernel at the link below and see whether or not the regression is still present. Thanks!

http://people.canonical.com/~sforshee/lp1612135/

[Robert Schetterer (robsche)]

Hi, confirmed cifs with automount ( krb in my case ) works with 4.4.0.36
its broken in 4.4.0.38

[Julian Einwag (jeinwag)]

I'm also experiencing the "object is remote" problem with 4.4.0-38 on trusty. 4.4.0-36 works fine.

@sforshee: The problem is still present in the test kernel with the fix reverted.

[Chris van Run (christehrun)]

OP here.

I can confirm the reported bug of dfs-referrals had been fixed in 4.4.0-36.55. However, the newer kernel 4.4.0-38.57 has another problem at a more fundamental level: no mount at all.

The fix-reverted version from #39 (4.4.0-38.57+lp1612135v201609211623) re-introduces the original dfs-referrals bug but does allow a mount to occur.

Cheers,

Chris

[Seth Forshee (sforshee)]

That's what I thought - this is a new bug not related to this one nor caused by it's fix. So someone please run 'ubuntu-bug linux' in a terminal after experiencing this problem to open a new bug. You'll have to do this with the official 4.4.0-38.57 and not my test build from comment #39. Once you've done that, please post the bug number here for others and subscribe me to the bug. Thanks!

[Nico (nicom66)]

Hello @sforshee

I did, here is the link:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1626112

Thanks!

[Chris van Run (christehrun)]

Hello,

Another bug has occurred that prevents CIFS automounts to function as they do in kernel 4.4.0-28. This is likely caused by the fixes suggested here.

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1629204

@sforshee, I subscribed you already.

Thanks!