After upgrade to 3.13.0-35.62, rpc.gssd complains about missing /run/rpc_pipefs/gssd/clntXX/info

[Sergio Gelato]

The following changes in 3.13.0-35.62:
 * sunrpc: create a new dummy pipe for gssd to hold open
   - LP: #1327563
 * sunrpc: replace sunrpc_net->gssd_running flag with a more reliable check
   - LP: #1327563
 * nfs: check if gssd is running before attempting to use krb5i auth in SETCLIENTID call
   - LP: #1327563
are causing rpc.gssd to fill syslog with messages of the form
ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory

The problem was discussed last December in https://bugzilla.redhat.com/show_bug.cgi?id=1037793
where the resolution was to include the following three patches:

http://marc.info/?l=linux-nfs&m=138624689302466&w=2
http://marc.info/?l=linux-nfs&m=138624684502447&w=2
http://marc.info/?l=linux-nfs&m=138624684502447&w=2

These patches are already in the upstream kernel (since 3.14). I suggest cherry-picking them for 3.13. Commit hashes from the 3.14 branch:
 3396f92f8be606ea485b0a82d4e7749a448b013b
 e2f0c83a9de331d9352185ca3642616c13127539
 23e66ba97127ff3b064d4c6c5138aa34eafc492f

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1365869

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Sergio Gelato (sergio-gelato)]

Won't run apport-collect. (Not really needed for this particular bug.)

[penalvch (penalvch)]

Marking Triaged for now, as regression bisect performed, and upstream fix commits identified.

[Tim Gardner (timg-tpi)]

The following changes since commit 9cbc4a7f74175a538f6d7263e81a30e6c12ddf85:

  UBUNTU: SAUCE: Fix nfs oops stable regression (2014-09-04 16:15:41 -0700)

are available in the git repository at:

  git://kernel.ubuntu.com/rtg/ubuntu-trusty.git lp1365869-rpc-gssd

for you to fetch changes up to fa5324d3e1169c2dc37d673ff7e0569b330fa74f:

  rpc_pipe: fix cleanup of dummy gssd directory when notification fails (2014-09-05 09:32:17 -0700)

----------------------------------------------------------------
Jeff Layton (3):
      rpc_pipe: remove the clntXX dir if creating the pipe fails
      sunrpc: add an "info" file for the dummy gssd pipe
      rpc_pipe: fix cleanup of dummy gssd directory when notification fails

 net/sunrpc/rpc_pipe.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)

[Sergio Gelato (sergio-gelato)]

I've built, installed and tested the kernel described in comment #4. It does what I expected it to do:
* file /run/rpc_pipefs/gssd/clntXX/info now exists;
* rpc.gssd no longer complains.

No adverse side effects so far.

[Tim Gardner (timg-tpi)]

Sergio - thanks for the feedback. These patches will appear in Ubuntu-3.13.0-37.64.

[Joni-Pekka Kurronen (joni-kurronen)]

I have nfs4 not working and guessing could it be same problem,... actually
after 12.04 to 14.04 upgrade it stopped to work and I belived it was
pam/ldap realatd problem,... can kindly look at and confirm that this is the
reason and I can wait kernel update ( or upgrade kerenel? ) to get NFS4
working.

Server:
  ApacheDS: ldap and kerberos
  MIT: kerberos client
  nfs-kernel

LOG say's while mounting from client:

Sep 24 18:33:19 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 named[2378]: client 2001:14b8:100:8363:d5bc:33c:1c2c:6bc2#23423 (_kerberos-master._udp.KURROLA.FI): query (cache) '_kerberos-master._udp.KURROLA.FI/SRV/IN' denied
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: can't open /run/rpc_pipefs/gssd/clntXX/info: No such file or directory
Sep 24 18:33:23 mpi1 rpc.gssd[1176]: ERROR: failed to read service info

This is OK whit server, so principal should work,...
sudo kinit -k -t /etc/krb5.keytab <email address hidden>

Client:
  MIT: kerberos client

joni@kaak:~$ sudo mount -a
mount.nfs4: access denied by server while mounting mpi1.kurrola.dy.fi:/

and log says at client:

Sep 24 18:37:53 kaak sudo: joni : problem with defaults entries ; TTY=pts/2 ; PWD=/home/joni ;
Sep 24 18:37:53 kaak sudo: joni : TTY=pts/2 ; PWD=/home/joni ; USER=root ; COMMAND=/bin/mount -a
Sep 24 18:37:53 kaak sudo: pam_unix(sudo:session): session opened for user root by joni(uid=0)
Sep 24 18:37:54 kaak sudo: pam_unix(sudo:session): session closed for user root

This at client work's do principals should be ok.
 sudo kinit -k -t /etc/krb5.keytab <email address hidden>

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Joni-Pekka Kurronen (joni-kurronen)]

Tried to use propose but kernel 13.63 was last no 64 !!!

[Joni-Pekka Kurronen (joni-kurronen)]

hi,

Real Bug is that NFS logging dose not give clear information wgat's happening and diagnose is hard to do!

Problem solved:
1) I finaly found upgrade kernel, after installing it gssapi error
message dissapeared and NO ERROR MESSAGES at log's.

2) by useing wireshark I found _kerberos._udp DNS requests and
due DNS had not answers I configured BIND9 to publish all kerberos
service addresses.

I have NFS + MIT kerberos client and apacheDS LDAP + KERBEROS server
there is no addministration address at kerberos.

joni

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.13.0-37.64

---------------
linux (3.13.0-37.64) trusty; urgency=low

  [ Joseph Salisbury ]

  * Release Tracking Bug
    - LP: #1372576

  [ dann frazier ]

  * [Config] CONFIG_HW_RANDOM_XGENE=m on arm64

  [ Edward Lin ]

  * SAUCE: Add use_native_backlight quirk for Dell Inspiron 5721/3521
    - LP: #1354253, #1354313

  [ Tim Gardner ]

  * SAUCE: Fix nfs oops stable regression
    - LP: #1348670
  * [Config] Add mpt3sas to d-i
    - LP: #1368907
  * [Config] CONFIG_X86_16BIT=y
    - LP: #1371601

  [ Timo Aaltonen ]

  * SAUCE: i915_bdw: Rebase to v3.15.8
    - LP: #1359213

  [ Upstream Kernel Changes ]

  * Revert "x86-64, modify_ldt: Make support for 16-bit segments a runtime
    option"
    - LP: #1371601
  * mmc: rtsx: add R1-no-CRC mmc command type handle
    - LP: #1365378
  * rpc_pipe: remove the clntXX dir if creating the pipe fails
    - LP: #1365869
  * sunrpc: add an "info" file for the dummy gssd pipe
    - LP: #1365869
  * rpc_pipe: fix cleanup of dummy gssd directory when notification fails
    - LP: #1365869
  * hwrng: xgene - add support for APM X-Gene SoC RNG support
    - LP: #1365593
  * Documentation: rng: Add X-Gene SoC RNG driver documentation
    - LP: #1365593
  * arm64: dts: add random number generator dts node to APM X-Gene
    platform.
    - LP: #1365593
  * xen/balloon: cancel ballooning if adding new memory failed
    - LP: #1304001
  * x86/xen: resume timer irqs early
    - LP: #1368724
  * xen/manage: Always freeze/thaw processes when suspend/resuming
    - LP: #1368724
  * scsi_transport_sas: move bsg destructor into sas_rphy_remove
    - LP: #1368991
  * drm/i915: Enable 5.4Ghz (HBR2) link rate for Displayport 1.2-capable
    devices
    - LP: #1369633
  * bnx2x: Fix link for KR with swapped polarity lane
    - LP: #1370716
  * drm: add DRM_CAPs for cursor size
    - LP: #1359213
  * drm/dp: Add AUX channel infrastructure
    - LP: #1359213
  * drm/dp: Add drm_dp_dpcd_read_link_status()
    - LP: #1359213
  * drm/dp: Add DisplayPort link helpers
    - LP: #1359213
  * drm/dp: Allow registering AUX channels as I2C busses
    - LP: #1359213
  * drm/dp: let drivers specify the name of the I2C-over-AUX adapter
    - LP: #1359213
  * drm/dp: make aux retries less chatty
    - LP: #1359213
  * Bluetooth: Enable Atheros 0cf3:311e for firmware upload
    - LP: #1371477
  * bnx2x: fix crash during TSO tunneling
    - LP: #1371601
  * inetpeer: get rid of ip_id_count
    - LP: #1371601
  * ip: make IP identifiers less predictable
    - LP: #1371601
  * tcp: Fix integer-overflows in TCP veno
    - LP: #1371601
  * tcp: Fix integer-overflow in TCP vegas
    - LP: #1371601
  * macvlan: Initialize vlan_features to turn on offload support.
    - LP: #1371601
  * net: Correctly set segment mac_len in skb_segment().
    - LP: #1371601
  * iovec: make sure the caller actually wants anything in
    memcpy_fromiovecend
    - LP: #1371601
  * batman-adv: Fix out-of-order fragmentation support
    - LP: #1371601
  * sctp: fix possible seqlock seadlock in sctp_packet_transmit()
    - LP: #1371601
  * sparc64: Fix argument sign extension for compat_sys_futex().
    - LP: #1371601
  ...