idmapd does not starts to work after system reboot

[Alexander Betaev]

[Impact]
Certain systems which make extensive use of NFS mounts have not been able to boot reliably since the migration to mountall in Ubuntu 9.10. This is because of race conditions in the handling of mounts vs. the startup of NFS client daemons. A proper fix of the startup of the NFS client daemons would in turn cause a deadlock of mountall, which processes mounts serially. We should fix mountall to parallelize its handling of mounts, so that the NFS daemons can be fixed properly.
This issue also impacts the cloud-init package, which relies on being able to do 'start on mounted MOUNTPOINT=/' and the like.

[Test case]
1. Have Scott Moser confirm that the package in -proposed resolves the 120-second boot-time delay in cloud images when using cloud-init.

[Regression potential]
Difficult to quantify. This code was included in the 12.10 release, however a regression was found in that code (bug #1059471) which is now being fixed there in SRU. There may be other latent regressions that have not yet been identified. Furthermore, introducing additional asynchronous handling here has the potential to expose race conditions (bugs) in other code that currently works by accident due to the serial handling of mounts.

Binary package hint: nfs-common

I have server which runs Kerberos+LDAP+NFSv4. After installing minimal Ununtu 10.10, I set up Kerberos client and NFSv4. But I've got a problem when restarted computer. After some retries I've noticed that idmapd does not work properly after system restarts but there is a workaround which makes it work:
1. Edit /etc/rc.local and place there following commands
sleep 5
service autofs stop
service idmapd restart
service autofs start
2. Add rc.local to system startup:
update-rc.d rc.local enable

Hope this workaround will help to find out the source of the problem. It is obvious that something launches in wrong way. But what is that?

Thanks!

[Steve Langasek (vorlon)]

Building on Clint Byrum's work on bug #525154, I'm much closer now to understanding a possible solution for this issue, but it's going to require some coordination. Details:

- the current idmapd job starts on 'local-filesystems or mounting TYPE=nfs4' because it needs to start whenever an nfs4 filesystem is mounted and it also needs to wait until /usr and /var/lib are available before starting up (/usr because idmapd is located in /usr/sbin; /var/lib because it uses /var/lib/nfs/rpc_pipefs). The only way to wait for /usr and /var/lib is by waiting for 'local-filesystems'; it's *possible* that one or both of these filesystems is not local, but that's a local configuration error anyway.
- the start condition used here is buggy. If local-filesystems is emitted first, idmapd will proceed to start up without blocking any further 'mounting' hooks. If 'mounting TYPE=nfs4' is emitted first, there is no way to make the job wait for the local-filesystems signal to be received, which can cause the job to try to start before the filesystem is usable and wind up in an inconsistent state when idmapd aborts.
- using jobs in the style of portmap-wait and statd-mounting, it is possible to construct a set of jobs that will only start idmapd on local-filesystems, and *also* block any nfs4 mounts until idmapd is started.
- unfortunately, it appears that mountall itself blocks on the result of the 'mounting' hook before doing any further processing of *any* mount points, with the result that, if 'local-filesystems' has not already been emitted at the time it tries to mount the first nfs4 filesystem, we end up in a deadlock: the 'mounting' hook is waiting for idmapd to start; idmapd is waiting for local-filesystems to be emitted; and mountall is waiting for the 'mounting' hook to return before going on to do any other mounts.

I see three possible solutions here.

1. Change mountall to be able to do other work while waiting for the 'mounting' hook to return. Conceptually I don't see any reason this isn't possible, so it should just be a matter of code reordering.
2. Change mountall to special case nfs4 mounts so that they are never handled until after local-filesystems is emitted. Yuck for the special-casing, though conceptually not actually different from what we're trying to achieve through the nfs-common upstart jobs.
3. Move idmapd and its dependencies (libevent; libnfsidmap, /usr/lib/libnfsidmap/) to the root filesystem (/sbin, /lib) and move /var/lib/nfs/rpc_pipefs to /var/run/nfs/rpc_pipefs. The latter may be correct in its own right (I'm pretty sure there's nothing on this in-kernel mount point that would count as 'persistent state'); the former doesn't even cover all cases unless we also move the kerberos+ldap stack to /lib, due to /usr/lib/libnfsidmap/umich_ldap.so.

I believe option 1 is the most straightforward to SRU and is correct per se, although parts of 3 are probably worth pursuing in their own right as part of an overall effort to improve FHS compliance.

[Alexander Achenbach (xela)]

I have invested some time into analysis of some of the problems the introduction of upstart/mountall created.

Building on Steve Langasek's and Clint Byrum's ideas of introducing fixes in the start-up procedure to make idmapd/gssd and even statd start reliably again, I initially failed due to the limiting sequential (globally blocking) handling of event emission in mountall (see Steve's previous post), which may leave the start-up process with an impossible condition of not having all local file systems mounted, but needing them to actually continue with mounting.

Attached is a fix for this problem (refer to Steve's idea #1). The patch should apply cleanly to mountall 2.20+nmu1, and with only slight context adaption should apply to earlier versions of mountall (such as of lucid).

Note, however, that this fix _alone_ will *not* make statd/idmapd/gssd start cleanly. This is due to the (IMHO weird) handling of multiple instances of the same event category in upstart. The first 'mounting TYPE=nfs4' would trigger the corresponding upstart jobs and block the respective mount in mountall, but subsequent 'mounting TYPE=nfs4' events issued by mountall would not block their mounts in mountall, leaving them to races with statd/idmapd/gssd.
In other words, the first mount in fstab would work ok, others would (probably) still fail.

To fix the latter, I will post adapted job configuration files for statd/idmapd/gssd/rpc_pipefs against nfs-common, and will provide cross reference information here for you to locate them. (The basic idea is to introduce multi-instance versions of the lately introduced statd-mounting job for all of statd/idmapd/gssd, and to take a portmap-wait like approach for rpc_pipefs.)

The fixes have been tested several dozen times on a few machines that represented the most problematic cases of NFS servers and NFS clients that would not even boot up cleanly using stock lucid.

[Alexander Achenbach (xela)]

This is a secondary patch meant to prevent miscounting of mounts in mountall.

The problem may actually already exist in mountall 2.20+nmu1 (the core of the patch should work there, too).

With asynchronous event handling, this problem may show up severely, which is why I post it here.

[Alexander Achenbach (xela)]

Suitable for the mountall async patch set just provided, here is my set of job configuration files for nfs-common:

gssd -- (triggered by portmap and local-filesystems)
gssd-mounting -- (triggered by mounting, triggers gssd)
idmapd -- (triggered by local-filesystems)
idmapd-mounting -- (triggered by mounting, triggers idmapd)
statd -- (triggered by portmap and local-filesystems)
statd-mounting -- (triggered by mounting, triggers statd)
rpc_pipefs -- (not triggered directly)
rpc_pipefs-wait -- (triggers rpc_pipefs; starting this allows to trigger/wait for rpc_pipefs)

You may find their job configuration files in the attached tarball.

The *-mounting jobs are meant to exclusively deal with individual emissions of mounting events by mountall. They are multi-instance jobs to make concurrent mounts independent.

The rpc_pipefs-wait job is similar (wrt rpc_pipefs) to portmap-wait (wrt portmap). It is needed since more than one daemon (idmapd, gssd) may have to wait for rpc_pipefs, and the previous multi-instance approach in rpc_pipefs (as of 2.20+nmu1) was unreliable (races between eg idmapd starting / rpc_pipefs mounting and [unneeded] gssd stopping / rpc_pipefs unmounting).

Since there is a large amount of script re-use in most of the above, I moved testers for gssd/idmapd/statd applicability into their own script (/usr/share/nfs-common/functions). This script is also included in the attached tarball.

I originally intended to have 'start on local-filesystems and mounting ...' in the corresponding *-mounting.conf files. Unfortunately, this does not work reliably with upstart. Generally, the whole state machine of upstart seems to be rather fragile or inconsistent with regard to handling of multiple events. So instead, I made an attempt to create a stateful form of the local-filesystems event. The following two jobs are meant for it and should probably go into the mountall package:

mountall-local -- (represents persistent state after local-filesystems, ie local file systems are up)
mountall-local-wait -- (starting this allows to wait for the state of local file systems being up)

The two job configuration files for these are also included in the tarball.

Most of these jobs are just kludges for missing or unreliable functionality in upstart, but they worked fine for our needs. I hope they may prove useful until upstart provides better ways to deal with things.

[Steve Langasek (vorlon)]

James, can you review Alexander's proposed patch to mountall here?

[James Hunt (jamesodhunt)]

@Steve - I've performed a code review of the 2 patches. Aside from a few minor formatting suggestions for mountall.diff (#2), they look good:

mountall.c:162: indent of 'handler' element should align with 'mnt'.
mountall.c:1559: Suggest comment changed to, "All mounts have been attempted, so wait for pending events."
mountall.c:1682: add space after nih_info and before '('.
mountall.c:1685: add space after nih_info and before '('.
mountall.c:1694: add space after nih_info and before '('.
mountall.c:1697: add space after nih_info and before '('.
mountall.c:2349: add space after NIH_MUST and before '('. Ditto for nih_alloc.

I've also done a very quick "sniff test" on a natty system patched to incorporate both patches. The test system was not configured for NFS, so clearly further testing is required on this front. However, the system boots and the async mounts appear to work correctly.

Full testing clearly requires the NFS scenario described. Additionally, consideration needs to be given to the following questions:

- what happens when an fsck fails?
- what happens if a swap image is corrupt?
- how will this impact mountall in initramfs?
  (https://blueprints.launchpad.net/ubuntu/+spec/packageselection-foundations-n-event-based-initramfs)

[Jean Christophe André (progfou)]

Hi guys!

First thanks for your analysis

I'm in the process to let deploy more than 400 computers using the current LTS with NFSv4 (and autofs here too) and I'm stuck at randomly facing exactly the same problem of idmapd failing to start, here because of the late rpc_pipefs start:

 Feb 28 22:20:42 test kernel: [ 6.305583] Installing knfsd (copyright (C) 1996 <email address hidden>).
 Feb 28 22:20:43 test rpc.statd[888]: Version 1.1.6 Starting
 Feb 28 22:20:43 test rpc.statd[888]: Flags:
 Feb 28 22:20:43 test rpc.idmapd[901]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory
 Feb 28 22:20:43 test init: idmapd main process (901) terminated with status 1
 Feb 28 22:20:43 test init: idmapd main process ended, respawning
 ... (more of them)
 Feb 28 22:20:43 test init: idmapd main process (954) terminated with status 1
 Feb 28 22:20:43 test init: idmapd respawning too fast, stopped

So what's next? Is there any package ready for testing yet?

Cheers, J.C.

[Alexander Achenbach (xela)]

Hi Jean,

I can only (temporarily) offer the patched mountall packages we use
on our cluster for just over 400 machines:

  http://fias.uni-frankfurt.de/~xela/mountall_2.20.1xela2_amd64.deb
  http://fias.uni-frankfurt.de/~xela/mountall_2.20.1xela2_i386.deb

They should work ok on lucid.

For nfs-common related stuff and the state jobs 'mountall-local'
and 'mountall-local-wait', we directly installed the upstart job
files in the corresponding locations (see my tarball on comment #4)
-- no package integration for these exists on my side yet.

Cheers,
Alex

[Jean Christophe André (progfou)]

Thanks Alexander!

I've investigated your patches and find them very interesting, clearly based on the same model of portmap/portmap-wait.

Still, I was looking for the less intrusive possible modification in the time being (while this bug is being investigated thoughtfully, I hope, by Canonical and/or the community) so I just added this line at the end of idmapd's pre-start script:

  if ! mountpoint -q /var/lib/nfs/rpc_pipefs ; then do_modprobe sunrpc ; mount -t rpc_pipefs rpc_pipefs /var/lib/nfs/rpc_pipefs || true ; fi

I know this is not the correct way to do it, but it's enough in my case to ensure idmapd will always start/restart/stop correctly, without needing to touch any other file than /etc/init/idmapd.conf.

I think your way is a really good one and I hope it will soon be officially integrated through a standard LTS update.

Cheers, J.C.

[Robert Crida (rcrida)]

We appear to have solved this problem by modifying /etc/init/idmapd.conf and changing:

start on (local-filesystems or mounting TYPE=nfs4)

to

start on (net-device-up or mounting TYPE=nfs4)

Comments?

[Leonardo Borda (lborda)]

Hi Steve,

I have an environment for testing purposes currently configured with Kerberos + NFSv4 running on Lucid 10.04. I still could not replicate the issue since it looks like it occurs randomly.
Please let me know if you have some ideas on how to create a testing scenario for this issue. We can use my own environment for this matter.

Leonardo

[Steve Atwell (satwell)]

In my configuration, /etc/fstab has only local filesystems. NFS filesystems are not mounted until later by autofs.

This means that idmapd should be starting on the local-filesystems event.

rpc_pipefs has "start on (starting gssd or starting idmapd)". My understanding is that rpc_pipefs should run to completion before idmapd is allowed to start.

For some reason this dependency insertion based on "starting" isn't working. I find that I can make idmapd fail reliably by making the rpc_pipefs job take longer. If I insert a sleep 10 into /etc/init/rpc_pipefs.conf just before the mount command, idmapd will always fail to start, complaining about /var/lib/nfs/rpc_pipefs/nfs just as in comment #7 above.

[Steve Atwell (satwell)]

Interestingly, I'm not able to reproduce my findings in the comment above with a stock Lucid install (choosing just openssh in tasksel, then installing nfs-common and turning on statd, gssd, and idmapd in /etc/default/nfs-common).

However, I did verify that, on the systems I can reproduce this on, /etc/init/{idmapd.conf,rpc_pipefs.conf} are unmodified from what nfs-common 1:1.2.0-4ubuntu4.1 provides. (Other than my added sleep for testing, of course.)

I'll have to do some more digging to see if I can come up with a repro case.

[Clint Byrum (clint-fewbar)]

Excerpts from Steve Atwell's message of Sat Jun 25 01:09:05 UTC 2011:
> In my configuration, /etc/fstab has only local filesystems. NFS
> filesystems are not mounted until later by autofs.
>
> This means that idmapd should be starting on the local-filesystems
> event.
>
> rpc_pipefs has "start on (starting gssd or starting idmapd)". My
> understanding is that rpc_pipefs should run to completion before idmapd
> is allowed to start.
>
> For some reason this dependency insertion based on "starting" isn't
> working. I find that I can make idmapd fail reliably by making the
> rpc_pipefs job take longer. If I insert a sleep 10 into
> /etc/init/rpc_pipefs.conf just before the mount command, idmapd will
> always fail to start, complaining about /var/lib/nfs/rpc_pipefs/nfs just
> as in comment #7 above.

Or's can be tricky for blocking. The first one that gets hit will block,
but not the second one.

Since you have 'start on starting gssd or starting idmapd', the first
one to trigger rpc_pipefs is the only one that gets blocked. gssd
is start on started portmap, which starts on virtual-filesystems and
net-device-up IFACE=lo.

So whats actually happening is rpc_pipefs is only blocking gssd.

What should probably be done is the new 'wait-for-state' job that I
introduced recently should be added to the pre-start of gssd and idmapd..

start wait-for-state WAIT_FOR=rpc_pipefs WAITER=gssd

I recall that ther eis a bug in upstart to track the OR condition problem,
but its bug # escapes me.

[Leonardo Borda (lborda)]

Hi Guys,

I reproduced the issue using Steve Atwell's modification in /etc/init/rpc_pipefs.conf and I am able to reproduce the same issue in a stock Lucid install.

It looks like /etc/init/idmapd.conf needs also another if statement that checks whether the client is using nfsv4 or not. I made a small change to the upstart script and now I don't see the issue anymore. Idmapd start successfully and I am able to mount using krb5.

Here is what I did:
I've changed the last statement (script/end script) in /etc/init/idmapd.conf to the following:

script
 if [ -f "$DEFAULTFILE" ]; then
  . "$DEFAULTFILE"
  #
         # We also need to check in /etc/default/nfs-common if the client wants to use nfs4
         #
         if grep -q "NEED_IDMAPD=yes" $DEFAULTFILE ; then
                 mount -t rpc_pipefs rpc_pipefs /var/lib/nfs/rpc_pipefs
         fi

 fi
  exec rpc.idmapd
end script

Can you guys check on your side as well?

Leonardo

[Steve Atwell (satwell)]

I don't see the point of checking whether NEED_IDMAPD=yes is in /etc/default/nfs-common. The "script" block is only going to run when the pre-script determines that idmapd should be started. (Even if you did need to check NEED_IDMAPD, you'd want to read it out of the environment rather than grepping the defaults file.)

It looks like your proposed solution boils down to copying what the rpc_pipefs upstart job does into the idmapd job. This seems like a possible workaround, but definitely not a fix.

[Steve Langasek (vorlon)]

I agree with Steve's analysis; the NEED_IDMAPD check here is redundant. I have actually considered moving the rpc_pipefs job's contents directly into the idmapd and gssd jobs since the current start condition of rpc_pipefs makes it significantly less useful to have it as a separate job, but haven't gotten to it yet. But that current start condition in natty and above should not have races that allow idmapd to start before rpc_pipefs is mounted.

It may be that backporting the current rpc_pipefs job to lucid, to ensure that we block *both* gssd and idmapd until rpc_pipefs is mounted instead of only blocking the first of the two which tries to start, would solve the problem Leonardo and Steve are having. I haven't done this yet because it's not a complete solution for the issues with idmapd (see comment #1 regarding the changes needed in mountall to go with this), but provided /usr is not a separate filesystem, it may be sufficient.

BTW, calling grep, mount, and rpc.idmapd from the script block is definitely wrong, as this job is 'expect fork' and upstart will wind up trying to supervise the first forked process instead of rpc.idmapd.

[Steve Langasek (vorlon)]

I think this bug is in a somewhat confused state at the moment. I have tried now to reproduce the problem using oneiric, which as of the latest SRU had the same upstart job handling as in lucid. (Apparently we had already SRUed for this issue, and I had subsequently forgotten!) The only problem I've been able to reproduce is that caused by trying to start idmapd before /usr is mounted, and the job failing to start as a result. I have a workaround (not a fix) for this, which I will SRU for shortly. But with the current revision of upstart jobs (which has rpc_pipefs as an 'instance' job that starts separately for, and blocks, each of gssd and idmapd), I definitely cannot reproduce this most recently described problem:

 Feb 28 22:20:43 test rpc.idmapd[901]: main: open(/var/lib/nfs/rpc_pipefs/nfs): No such file or directory

I can't reproduce it even when adding a 10 second delay to rpc_pipefs as suggested.

Now, the SRU that addressed this issue, nfs-utils 1:1.2.0-4ubuntu4.1, was published to lucid-updates on January 27. Is it possible that the people who were reporting this problem after that date had not yet applied the SRU fix?

Steve, you mention idmapd failing to start for you in a configuration where there are no nfs mounts in /etc/fstab. How does it fail to start? Does it give you a job in state 'start/running' with no PID (which would be the /usr race), or does it give you the "/var/lib/nfs/rpc_pipefs/nfs: No such file or directory" log error (which I can only reproduce with -4ubuntu4)? You mentioned you were going to try to find another reproducer case - did you succeed?

[Ivan Frederiks (idfred)]

Met same problem in natty.

@Steve Langasek
There is no way to reproduce open(/var/lib/nfs/rpc_pipefs/nfs) problem in oneric because mount-related actions from /etc/init/rpc_pipefs.conf were moved to /etc/init/idmapd.conf and /etc/init/gssd.conf.
I think, that best solution for this bug is to backport contents of /etc/init directory of nfs-common package.

[Steve Langasek (vorlon)]

iFred,

I could not reproduce this problem even before I made that change to /etc/init{idmapd,gssd}.conf in oneiric.

Something has just occurred to me, however, Could it be that the users seeing this bug all had /var as a separate partition? In thas case, the rpc_pipefs job would fail if /var isn't yet mounted, causing the idmapd job to *continue* in spite of the failure, and resulting in this error.

In that case, it seems that my recently proposed SRU to drop the rpc_pipefs job may have precisely the right effect, if entirely by accident.

[Steve Langasek (vorlon)]

Those who are experiencing problems with idmapd not starting reliably at boot, please see bug #811823 and provide feedback there for the SRU packages that have just been accepted into the -proposed pocket for lucid, maverick, and natty. I've opened this as a separate bug because there is still an architectural issue here with mountall+idmapd which we should solve permanently, but 811823 seems to address most of the symptoms.

Feedback from users seeing this problem is needed in order for the SRU to be validated and published where it can help all users.

[Alexander Achenbach (xela)]

Hi Steve,

I will have a look at #811823 later on, but for a start:

We have /var mounted as a separate partition on all installations
on which we see the idmapd start-up problems. The same goes for
/usr and /tmp.

Cheers,
Alex

On 19/07/11 18:59, Steve Langasek wrote:
> iFred,
>
> I could not reproduce this problem even before I made that change to
> /etc/init{idmapd,gssd}.conf in oneiric.
>
> Something has just occurred to me, however, Could it be that the users
> seeing this bug all had /var as a separate partition? In thas case, the
> rpc_pipefs job would fail if /var isn't yet mounted, causing the idmapd
> job to *continue* in spite of the failure, and resulting in this error.
>
> In that case, it seems that my recently proposed SRU to drop the
> rpc_pipefs job may have precisely the right effect, if entirely by
> accident.
>

[maxjos (magne-fortytwo)]

I have enabled -proposed (ref bug 811823) and idmapd still not starting.

The setup is natty with -proposed, Automount, NFSv4, Kerberos and NEED_IDMAPD=yes in /etc/default/nfs-common

After boot idmapd is (consistently) not running. Using 'service idmapd start' after boot does not work and I need to manually launch rpc.idmapd as root and then restart autofs for mounts to work.

[Clint Byrum (clint-fewbar)]

maxjos, this bug is not fixed in -proposed or even in a dev release of Ubuntu. Please comment on bug 811823, and try the explicit "TEST CASE" mentioned in the description of that bug, reporting your results. Thanks!

[Steve Langasek (vorlon)]

maxjos, you don't mention in your comment whether you have /usr on a separate partition or not, so I have no way to confirm whether yours is the same issue as the one being fixed in bug #811823. Can you confirm whether you have /usr mounted as a separate partition?

[maxjos (magne-fortytwo)]

Hi Steve, I do not have a separate /usr partition

[Steve Atwell (satwell)]

Steve, I tried out nfs-common 1:1.2.0-4ubuntu4.2 from lucid-proposed (as referenced in bug 811823), and it works fine for me.

My problem was caused by a race condition with the rpc_pipefs dependency (as verified by adding a sleep in rpc_pipefs.conf), so this will obviously fix this problem.

[maxjos (magne-fortytwo)]

Is there any update on this? 11.04 with Kerberos, NFSv4, automount 5, idmapd does not start automatically.

[Steve Langasek (vorlon)]

On Fri, Nov 04, 2011 at 11:35:04AM -0000, maxjos wrote:
> Is there any update on this? 11.04 with Kerberos, NFSv4, automount 5,
> idmapd does not start automatically.

The update is that bug #811823 already fixes the startup of idmapd to the
best of anyone's knowledge, and the remaining issue is only cosmetic. If
you're having problems with idmapd not starting in 11.04, I recommend filing
a new bug report describing your problem.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Torsten Bronger (bronger)]

I've never had /usr on a separate partition, only /home/bronger, yet I see this bug.

[Vang Le (lqvang79)]

I found this bug effect my systems today after installing LDAP. I hope there will be update soon for Lucid. Working on solving the bug is out of my reach. So big thanks great appreciation to all the developers.

Vang.

[Scott Moser (smoser)]

Steve, I've got a mountall build a ppa at https://launchpad.net/~smoser/+archive/ephermal-fixes and am testing it. Generally it seems good for me. Is there anything I can do to help get this tested to be uploaded?

[Launchpad Janitor (janitor)]

This bug was fixed in the package mountall - 2.41

---------------
mountall (2.41) unstable; urgency=low

  [ Alexander Achenbach ]
  * Don't block other, unrelated mounts from being processed while one
    mount is blocked on its mounting signal to process. LP: #643289.

  [ Steve Langasek ]
  * Likewise, allow 'mounted' signals to be processed in parallel with
    other unrelated mounts.

 -- Steve Langasek <email address hidden> Tue, 25 Sep 2012 23:21:59 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package nfs-utils - 1:1.2.6-3ubuntu2

---------------
nfs-utils (1:1.2.6-3ubuntu2) quantal; urgency=low

  [ Steve Langasek ]
  * Adjust upstart jobs to treat TYPE=nfs and TYPE=nfs4 mounts identically,
    since TYPE=nfs4 is considered deprecated.
  * Fix various boot-time race conditions between mountall and nfs-utils by
    moving handling of the 'mounting' events to separate gssd-mounting and
    idmapd-mounting jobs. Requires mountall 2.41 or better to avoid deadlock
    on boot. LP: #643289, LP: #611397.
  * Fix the stop conditions: never stop on 'runlevel [06]' since that gives
    the system no time to cleanly unmount nfs mounts; instead, stop only on
    the unmounted-remote-filesystems event.
  * Newer versions of gssd don't talk to portmap, so don't make the upstart
    job depend on it.
  * Add an instance to statd-mounting, and change it to just wait for statd
    instead of trying to trigger it potentially out of order. This also means
    we don't need to try to force portmap to start from statd.

  [ Matthew L. Dailey ]
  * Add "-e" (ticket expiry is error) option to rpc.gssd to prevent hangs due
    to EKEYEXPIRED error from kernel on ticket expiry. LP: #794112
 -- Stephane Graber <email address hidden> Fri, 28 Sep 2012 13:58:43 -0400

[Colin Watson (cjwatson)]

Hello Alexander, or anyone else affected,

Accepted mountall into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/mountall/2.36.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Steve Langasek (vorlon)]

marking verification-failed due to bug #1078926.

[Adam Conrad (adconrad)]

Hello Alexander, or anyone else affected,

Accepted mountall into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/mountall/2.36.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Sergey Pashinin (pashinin)]

I had this crazy problem with NFS and not started idmapd after reboot on my Ubuntu 12.10 x64.
I tested a lot of advices from the internet but found my crazy soltion. I don't know why it works :)

I used to have in my fstab:
10.254.239.1:/usr/data/disk_1 /usr/data/disk_1 nfs4 _netdev,rsize=8192,wsize=8192,timeo=1,soft,retry=0,auto,bg 0 0

The only thing I changed was ip address:
domain.com:/usr/data/disk_1 /usr/data/disk_1 nfs4 _netdev,rsize=8192,wsize=8192,timeo=1,soft,retry=0,auto,bg 0 0

Actually domain.com resolves to 10.254.239.1 (My DNS server is on 10.254.239.1 also)
But there are even errors "Can't be resolved" in log. So I don't know why it works.

But still it mounts(!) now and everything is working.
I hope that it will help somebody and hope will never see such bugs again

[Scott Moser (smoser)]

I *think* this is fix-released in precise, right?
precise now has 2.36.3 and changelog reports it fixed in 2.36.2.

I'll test bug bug 1031065 tomomorrow.

[Steve Langasek (vorlon)]

The mountall part of this is fixed. now we can update nfs-utils to match.

[Mathew Hodson (mhodson)]

The precise-proposed package was superseded, so removing verification-needed tag.

[Steve Langasek (vorlon)]

Since precise has reached end of life, this is 'wontfix' for that release.