[SRU] libnih upgrade does not respawn init, resulting in unclean shutdown

[James Hunt]

If required package libnih1 (or libnih-dbus1) are upgraded, Upstart needs to be restarted to avoid it holding open "stale" file descriptors to files in the old versions of the packages. This results in the following issues:

- the shutdown sequence displays error messages.
- the shutdown sequence results in an unclean unmount of the root partition.
- the subsequent boot experience is slowed down and may show potentially alarming messages to users.

All this amounts to a non-optimal user experience.

See original bug 740390.

= Fix =

The fix released for quantal on bug 740390 is to add an explicit debian/libnih1.postinst and debian/libnih-dbus1.postinst maintainer script that simply touches a 'magic file' that forces upstart to restart on shutdown. The magic file is:

    /var/run/init.upgraded

This magic file is used by /etc/init.d/umountroot as a signal to ask Upstart to re-exec itself and thus relinquish all "stale" file descriptors it may have open.

The magic file is already used by:

- libdbus (package dbus): recent change - see bug 740390
- libc6 (package eglibc): since 2011-01-12.

= Justification for Back-Porting =

The fix is simple and has already been released to quantal.

= Test Case =

1) Boot a Precise system without the 'quiet' and 'splash' boot options.

2) Check version of libnih1 is 1.0.3-4ubuntu9:

     $ dpkg -p libnih1 libnih-dbus1|grep ^Version:

3) force a re-install of libnih1:

    $ sudo apt-get install --reinstall libnih1 libnih-dbus1

4) Add a sleep to /etc/init.d/umountroot so you can observe any possible error message:

  --- /etc/init.d/umountroot.ORIG 2012-10-05 11:48:40.384382009 +0100
+++ /etc/init.d/umountroot 2012-10-05 11:48:55.024382058 +0100
@@ -86,6 +86,7 @@
        mount $MOUNT_FORCE_OPT -n -o remount,ro -t dummytype dummydev / 2>/dev/null \
        || mount $MOUNT_FORCE_OPT -n -o remount,ro dummydev / 2>/dev/null \
        || mount $MOUNT_FORCE_OPT -n -o remount,ro /
+ sleep 5
        ES=$?
        [ "$VERBOSE" = no ] || log_action_end_msg $ES
 }

5) Shutdown

    $ sudo shutdown -h now

6) Ensure you see the expected error message. The last 2 lines of output should be:

     umount: /run/lock: not mounted
     mount: / is busy

   (Note that the 'umount' message is not an error - just informational).

7) Reboot again without the 'quiet' and 'splash' boot options.

8) Install libnih1 and lib-dbus1 both at version '1.0.3-4ubuntu9.1'.

9) Check that install has created the expected magic file:

    ls -l /var/run/init.upgraded

10) Shutdown

    $ sudo shutdown -h now

11) Ensure you see the expected output. The last line of output should be:

     umount: /run/lock: not mounted

     There should be no 'mount: / is busy' line.

12) Reboot and revert changes to /etc/init.d/umountroot if desired.

= Regression Potential =

No regression potential.

[Marc Deslauriers (mdeslaur)]

FYI, for testing, you can also do a "dmesg | grep orphan" once the machine has rebooted to see if it wasn't properly unmounted.

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted libnih into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libnih/1.0.3-4ubuntu9.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

The fix for this bug has been awaiting testing feedback in the -proposed repository for precise for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

[Steve Langasek (vorlon)]

James, can you please run a verification of this test case to ensure that the libnih in precise-proposed fixes the issue? We don't want to lose this fix due to lack of SRU verification.

[Colin Watson (cjwatson)]

I verified this following James' and Marc's instructions.

[Colin Watson (cjwatson)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libnih - 1.0.3-4ubuntu9.1

---------------
libnih (1.0.3-4ubuntu9.1) precise-proposed; urgency=low

  * Addition of debian/libnih-dbus1.postinst and
    debian/libnih1.postinst to force Upstart re-exec on shutdown
    to avoid unmounting disks uncleanly (LP: #740390, LP: #1062202).
 -- James Hunt <email address hidden> Fri, 05 Oct 2012 14:14:10 +0100