Kernel 4.4.0-31 cd into NFS4 directory: Operation not permitted

[Reinhard Zierke]

Since apt-upgrading my Ubuntu 16.04 PC to kernel 4.4.0-31 I can't cd into NFS4 subvolumes any more:

zierke@rzpc100$ uname -a
Linux rzpc100 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
zierke@rzpc100$ kinit
Password for <email address hidden>:
zierke@rzpc100$ cd /informatik2
zierke@rzpc100$ cd rz
-bash: cd: /informatik2/rz: Operation not permitted

When I reboot my PC with the previous kernel 4.4.0-28 this works fine.

This problem has also been mentioned in
https://www.kubuntuforums.net/showthread.php?70499-Latest-16-04-bug-NFS-problem-no-access-unless-I-use-root-first-strange
https://ubuntuforums.org/showthread.php?t=2331137

[Paul White (paulw2u)]

As you've saying that this is a kernel problem, I'm reassigning the bug report against "linux".

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1604396

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream stable kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.4 stable kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4.23

[Reinhard Zierke (reinhardzierke)]

On Mon, Oct 03, 2016 at 07:36:35PM -0000, Joseph Salisbury wrote:
> Would it be possible for you to test the latest upstream stable kernel?
> Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the
> latest v4.4 stable kernel[0].

Works fine, thank you very much.

> If this bug is fixed in the mainline kernel, please add the following
> tag 'kernel-fixed-upstream'.

Done.

Cheers,
  Reinhard

[Joseph Salisbury (jsalisbury)]

That is good news. The Ubuntu Xenial kernel will get the fix for this bug, through the normal stable update process.

Xenial already has the 4.4.21 updates in the -proposed repository. Can you give that kernel a try to see if the fix is in a later kernel, such as .22 or .23?

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed.

[Torsten Harenberg (harenberg)]

Hi,

I cannot verify that the new upstream kernel fixes that.

Also suffering from not being able to log into an NFS4/Kerberoized directory with 16.04 anymore while the same works fine on 14.04. idmapd are configured identically.

Ubuntu 16.04 (with new upstream kernel)
------------

Could not chdir to home directory /common/home/harenber: Permission denied
groups: cannot find name for group ID 50094
-bash: /common/home/harenber/.bash_profile: Permission denied
harenber@bene:/$ mount | grep nfs
whep-nfs.pleiades.uni-wuppertal.de:/home on /common/home type nfs4 (rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=krb5i,clientaddr=132.195.104.97,local_lock=none,addr=132.195.125.8)
harenber@bene:/$ cd /common
harenber@bene:/common$ ls -l
total 4
drwxr-xr-x 88 root users 4096 Sep 30 13:08 home
harenber@bene:/common$ cd home
-bash: cd: home: Permission denied
harenber@bene:/common$ klist
Ticket cache: KEYRING:persistent:1548:krb_ccache_JGqN6o8
Default principal: <email address hidden>

Valid starting Expires Service principal
06.10.2016 10:08:11 07.10.2016 10:08:11 <email address hidden>
harenber@bene:/common$ kinit -f
Password for <email address hidden>:
harenber@bene:/common$ klist
Ticket cache: KEYRING:persistent:1548:krb_ccache_JGqN6o8
Default principal: <email address hidden>

Valid starting Expires Service principal
06.10.2016 10:09:10 07.10.2016 10:09:07 <email address hidden>
harenber@bene:/common$ ls -l
ls: cannot access 'home': Permission denied
total 0
d????????? ? ? ? ? ? home
harenber@bene:/common$ uname -a
Linux bene.physik.uni-wuppertal.de 4.4.23-040423-generic #201609300709 SMP Fri Sep 30 11:11:23 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
harenber@bene:/common$

whereas on 14.04:

[10:10] harenber@whep-nfs:~ $ klist
Ticket cache: FILE:/tmp/krb5cc_1548_S3S8IP
Default principal: <email address hidden>

Valid starting Expires Service principal
10/06/2016 10:10:07 10/07/2016 10:10:07 <email address hidden>
[10:10] harenber@whep-nfs:~ $ cd /common
[10:10] harenber@whep-nfs:/common $ ls
home
[10:10] harenber@whep-nfs:/common $ cd home
[10:10] harenber@whep-nfs:/common/home $ uname -a
Linux whep-nfs 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:42:33 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[10:10] harenber@whep-nfs:/common/home $

Best regards,

  Torsten

[Torsten Harenberg (harenberg)]

Sorry, the second half of my post was from the NFS server (running 16.04) where the file system is local.

This here is a a 14.04 NFS *CLIENT*

[10:26] harenber@whep-web:~ $ ls /common/home/
aquota.user ernis hamacher lehmann riegel tepel
arwa errenst harenber lenz roggel thartmann
becks fausten harenberg link sahu tsang
bergenth Finanzen hartbrich lisa sandhoff ugrin
bocprod fischer hirsch lost+found sanny umesh
boerner fleischm ilc maettig schick volkmer
christina flick israeli moeller scholz wagner
cmwagner fp jaekel mvogel schuelerp weiss
dcs fstroth jem neumann schuh wensing
debus getsimple kathrin ofir schulz wwwhome
drees ggilles kersten partimag seema yarr
duda glitza kind pataraia sekretariat yildirim
duelsen goetze kuechler physik2 springer zeitnitz
elkhalii grollius lang pro_ttbar_s sweber
elli haeusling lapitckii puellen tcorneli
[10:26] harenber@whep-web:~ $ uname -a
Linux whep-web 3.13.0-77-generic #121-Ubuntu SMP Wed Jan 20 10:50:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[10:26] harenber@whep-web:~ $ klist
Ticket cache: FILE:/tmp/krb5cc_1548_UcLLWT
Default principal: <email address hidden>

Valid starting Expires Service principal
10/06/2016 10:26:31 10/07/2016 10:26:31 <email address hidden>
[10:26] harenber@whep-web:~ $ mount | grep nfs
whep-nfs.pleiades.uni-wuppertal.de:/home on /common/home type nfs4 (rw,sec=krb5i,soft,addr=132.195.125.8,clientaddr=132.195.124.14)
[10:27] harenber@whep-web:~ $

So 14.04 client: works, 16.04 client does not work, neither with 4.4.0 nor with 4.4.23 kernel.

Best regards,

  Torsten

[Thomas Fili (tfili69)]

Similar problem here,

but ubuntu kernel 4.4.0-31 was not affected here, all kernels from there to the current 4.4.0-45-generic has the problem.

Today i found this thread and test the latest mainline, 4.4.27-040427-generic, this one works without problems :)

NFS Server is FreeBSD 11, but also OmniOS r151018

Thx

Thomas

[Joseph Salisbury (jsalisbury)]

The upstream 4.4.24 updates are now in the 4.4.0-46.67 Ubuntu kernel. This is the current kernel in the -proposed repository.

Can you test the -46 proposed kernel?

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed.

[Thomas Fili (tfili69)]

Hi,

for me the proposed kernel 4.4.0-46.67 do not work ... Permission denied ...

As mentioned above mainline 4.4.27 work without problem.

Maybe some surveillance will help.

In our enviroment the FreeBSD 11 server shares /apool/NFS/share/home as NFSv4 + Kerberos ... every directory is a zfs filesystem. The clients mount /home/nfs.

Users home is located under /home/nfs/staff/username ... staff and username are also zfs filesystems.

A user can accesss his homedirectory under /home/nfs/staff/username without error but the next level deeper will only work as a normal directory, not as another zfs filesystem or NFSv4 sub-share

If i look at the mounts as the affected user with mount or df i see the base mountpoint /home/nfs and the sub-shares /home/nfs/staff and /home/nfs/staff/username

With kernels having the problem, the next sub-share will not shown when i try to access it.

With kernel working without the "Permission denied" problem i am able to access the next sub-share (data for example ) and mount or df shows also /home/nfs/staff/username/data

This happend also on shares without kerberos.