runit doesn't stop and /var isn't umounted on shutdown

[Christian Holtje]

[Impact]
Runit does not properly terminate the services it supervises, it just leaves them running. This can lead to files being open during unmount and with that dirty filesystems on the next reboot.

A minor annoyance is that postinst prints errors during installation because it tries to grep an inexistent file.

[Test Case]
1. Install runit from a purged state (e.g. dpkg -P runit; apt-get install runit). Observe the errors from postinst: grep: /etc/inittab: No such file or directory
2. Add the directory test to /etc/service.
3. Add a script called 'run' and a script called 'finish' to this directory. Add a line 'echo run > /test; sleep 10000' and 'echo finish > /test' to run and finish respectively, then make them executable.
4. When run is marked executable, runsvdir will start it, check with pstree -a to see the runsvdir process tree and run somewhere beneath. Observe that /test contains 'run'.
5. Execute the command 'stop runsvdir', which will shutdown the runit service.
6. Observe that pstree -a will still show runsv to be running, only the parent runsvdir process has been terminated. Also observe that /test will not contain 'finish', which means that runsv has never been told that runsvdir is going to terminate.

runsvdir expects SIGHUP to correctly shutdown all supervised services. Upstart sends SIGTERM by default, which causes runsvdir to exit immediately without cleaning up supervised processes.

[Regression potential]
IMO small, unless someone came to rely on the broken behaviour.

[Christian Holtje (docwhat)]

OS: Ubuntu Hardy Heron.

[Daniel Néri (dne)]

Pre-stop and post-stop scripts can be added to /etc/event.d/runsvdir to fix this problem:

pre-stop script
        sv -w15 force-stop /etc/service/*
end script
post-stop script
        sv exit /etc/service/*
end script

[tested with runit 2.0.0 on Intrepid]

Regards,
Daniel

[Francis Russell (francis-russell)]

I've created a debdiff against the version of runit in maverick that should fix this issue. Building and testing the modified package would be appreciated.

[Francis Russell (francis-russell)]

Hi,

I've revised the patch slightly because it didn't work properly when the /etc/service folder used symlinks. Also, there appears to be no benefit to using both pre-stop and post-stop scripts compared to just telling the runsv processes to force-shutdown after the runsvdir process is killed.

[Daniel Néri (dne)]

Hello,

It's been a while, but the extra step was probably inspired by the sample Debian "runit-as-init" stage 3 script: http://smarden.org/runit/debian/3.

[eraserix (eraserix)]

More than two years have passed, and this bug is still there in the runit package :(.

The patch is ok except for two things in debian/runsvdir.upstart:
- There is no "shutdown" event, so this does not work.
- Upstart has also learned to send a user defined signal to terminate a process, using "kill signal HUP" or similar. [http://upstart.ubuntu.com/cookbook/#kill-signal]

I attached an updated patch. This will only work for upstart versions that support the "kill signal" stanza.

[Mathieu Trudel-Lapierre (cyphermox)]

eraserix, so this works (the kill signal) in Precise?

As a first step, this should land in the development release, I'll sponsor it there first. Then please follow the steps at http://wiki.ubuntu.com/StableReleaseUpdates#Procedure to update the bug description for what's required for SRU, and subscribing ubuntu-sru...

[Mathieu Trudel-Lapierre (cyphermox)]

For SRU, you'll want to fix the debdiff to be version ...ubuntu2.1 rather than ...ubuntu3.

Also, please consider sending your patches that are applicable to debian (postinst, etc.) there.

[Launchpad Janitor (janitor)]

This bug was fixed in the package runit - 2.1.1-6.2ubuntu3

---------------
runit (2.1.1-6.2ubuntu3) raring; urgency=low

  [ Christoph Mathys ]
  * Kill runsvdir with SIGHUP when entering runlevel [016]. Fixes LP: #245728,
    LP: #315541 & LP: #539567.

  [ Francis Russell ]
  * Remove installation of runsvdir.conf in obsolete location /etc/event.d.
  * Modify postinst script so that it doesn't to try to grep /etc/inittab on
    initial package install if it doesn't exist.
 -- Christoph Mathys <email address hidden> Fri, 07 Dec 2012 20:43:15 -0500

[Iain Lane (laney)]

I'm going to unsubscribe ubuntu-sponsors as the bug requires someone to prepare an SRU debdiff and also to update the description with the SRU information (test case, regression potential). The description is more important than the debdiff, as the latter should be the same as was uploaded to raring but for the version number.

Please re-subscribe ubuntu-sponsors once this is done.

Thank you.

[Brian Murray (brian-murray)]

I'll go ahead and upload this to the precise-proposed queue for the SRU team to review but bugs 315541 and 539567 will both also need test cases for the SRU to proceed.

[Brian Murray (brian-murray)]

Hello Christian, or anyone else affected,

Accepted runit into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/runit/2.1.1-6.2ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Christian, or anyone else affected,

Accepted runit into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/runit/2.1.1-6.2ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Christian Holtje (docwhat)]

Alas, I don't have any systems using runit at this time.

[Javier López (javier-lopez)]

I can confirm that after applying the -proposed update the issue gets fixed in Ubuntu Precise:

$ stop runsvdir
runsvdir stop/waiting
root@quantal-i386:/etc/service/test# cat /test
finish

Also the errors from postinst: grep: /etc/inittab: No such file or directory disappeared at the installation stage.

$ apt-cache policy runit
runit:
  Installed: 2.1.1-6.2ubuntu2.1
  Candidate: 2.1.1-6.2ubuntu2.1
  Version table:
 *** 2.1.1-6.2ubuntu2.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise-proposed/universe i386 Packages
        100 /var/lib/dpkg/status
     2.1.1-6.2ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/universe i386 Packages

[Javier López (javier-lopez)]

I can confirm that after applying the -proposed update the issue gets fixed in Ubuntu Quantal:

stop runsvdir
runsvdir stop/waiting
root@quantal-i386:/etc/service/test# cat /test
finish

Also the errors from postinst: grep: /etc/inittab: No such file or directory disappeared at the installation stage.

$ apt-cache policy runit
runit:
  Installed: 2.1.1-6.2ubuntu2.2
  Candidate: 2.1.1-6.2ubuntu2.2
  Version table:
 *** 2.1.1-6.2ubuntu2.2 0
        500 http://us.archive.ubuntu.com/ubuntu/ quantal-proposed/universe i386 Packages
        100 /var/lib/dpkg/status
     2.1.1-6.2ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ quantal/universe i386 Packages

As a side note to the testcase, I was unable to run those commands till I added "#!/usr/bin/env bash" at the beginning of both scripts in both, Ubuntu Precise and Ubuntu Quantal

[Launchpad Janitor (janitor)]

This bug was fixed in the package runit - 2.1.1-6.2ubuntu2.1

---------------
runit (2.1.1-6.2ubuntu2.1) precise; urgency=low

  [ Christoph Mathys ]
  * Kill runsvdir with SIGHUP when entering runlevel [016]. Fixes LP: #245728,
    LP: #315541 & LP: #539567.

  [ Francis Russell ]
  * Remove installation of runsvdir.conf in obsolete location /etc/event.d.
  * Modify postinst script so that it doesn't to try to grep /etc/inittab on
    initial package install if it doesn't exist.
 -- Christoph Mathys <email address hidden> Tue, 11 Sep 2012 13:14:37 +0200

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for runit has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package runit - 2.1.1-6.2ubuntu2.2

---------------
runit (2.1.1-6.2ubuntu2.2) quantal; urgency=low

  [ Christoph Mathys ]
  * Kill runsvdir with SIGHUP when entering runlevel [016]. Fixes LP: #245728,
    LP: #315541 & LP: #539567.

  [ Francis Russell ]
  * Remove installation of runsvdir.conf in obsolete location /etc/event.d.
  * Modify postinst script so that it doesn't to try to grep /etc/inittab on
    initial package install if it doesn't exist.
 -- Christoph Mathys <email address hidden> Tue, 11 Sep 2012 13:14:37 +0200