libdbus-1-3 upgrade does not respawn init, resulting in unclean shutdown

Bug #740390 reported by Alan on 2011-03-22
42
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ubuntu on EC2
Undecided
Unassigned
dbus (Ubuntu)
High
Marc Deslauriers
Lucid
Medium
Marc Deslauriers
Natty
Medium
Marc Deslauriers
Oneiric
Medium
Marc Deslauriers
Precise
Medium
Marc Deslauriers
Quantal
High
Marc Deslauriers
libnih (Ubuntu)
Undecided
James Hunt
Lucid
Undecided
Unassigned
Natty
Undecided
Unassigned
Oneiric
Undecided
Unassigned
Precise
Undecided
Unassigned
Quantal
Undecided
James Hunt
upstart (Ubuntu)
Lucid
Undecided
Unassigned
Natty
Undecided
Unassigned
Oneiric
Undecided
Unassigned
Precise
Undecided
Unassigned
Quantal
Wishlist
Unassigned

Bug Description

Binary package hint: dbus

Steps to reproduce:
1. Start 20110321 daily build on Amazon EC2.
ami-30669459 ubuntu-images-testing/ubuntu-natty-daily-amd64-server-20110321
2. SSH.
3. sudo apt-get update
4. To simulate an upgrade, sudo apt-get --reinstall install libdbus-1-3
libdbus-1-3_1.4.6-1ubuntu6 is reinstalled.
5. sudo lsof yields (snipped)
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
init 1 root DEL REG 202,1 134262 /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
upstart-u 201 root DEL REG 202,1 134262 /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
upstart-s 369 root DEL REG 202,1 134262 /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
6. sudo shutdown -h now
Error:
mount: / is busy

Possible solution: touch /var/run/init.upgraded in package postinst to force init to respawn/close open files at shutdown. This is checked in /etc/rc[06].d/S60umountroot. See bug 672177 for a similar situation with libc6.

Scott Moser (smoser) wrote :

Ubuntu on EC2 project is not the right place to file these any more. Instead, please use 'ubuntu-bug <package>' on an instance, or tag with 'ec2-images' and 'uec-images'

Changed in ubuntu-on-ec2:
status: New → Invalid
tags: added: ec2-images uec-images
Scott Moser (smoser) wrote :

I'm unable to reproduce this bug.

The ami given (ami-30669459 ubuntu-images-testing/ubuntu-natty-daily-amd64-server-20110321) is no longer public.
I've tried running the latest daily :
  us-east-1 ami-d8e21eb1 canonical ubuntu-natty-daily-i386-server-20110418

then doing:
 $ sudo apt-get update
 $ sudo apt-get install --reinstall libdbus-1-3
 $ sudo shutdown -h now

I can see the '/ is busy' message in the console out, but the instance *does* go to terminated.

I suspect that the fallout of this is that the filesystem is not unmounted and thus not clean, right?

>I suspect that the fallout of this is that the filesystem is not unmounted and thus not clean, right?

Correct. This results in orphaned inodes on reboot. Touching
/var/run/init.upggraded is a possible solution. This will cause init
to respawn on shutdown.

You have reproduced the bug. The problem isn't that the system doesn't
shut down. The problem is an unclean filesystem.

Thanks for starting to take a look.

On 4/18/11, Scott Moser <email address hidden> wrote:
> I'm unable to reproduce this bug.
>
> The ami given (ami-30669459
> ubuntu-images-testing/ubuntu-natty-daily-amd64-server-20110321) is no longer
> public.
> I've tried running the latest daily :
> us-east-1 ami-d8e21eb1 canonical ubuntu-natty-daily-i386-server-20110418
>
> then doing:
> $ sudo apt-get update
> $ sudo apt-get install --reinstall libdbus-1-3
> $ sudo shutdown -h now
>
> I can see the '/ is busy' message in the console out, but the instance
> *does* go to terminated.
>
> I suspect that the fallout of this is that the filesystem is not
> unmounted and thus not clean, right?
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/740390
>
> Title:
> libdbus-1-3 upgrade does not respawn init, resulting in unclean
> shutdown
>
> Status in Official Ubuntu Images on EC2:
> Invalid
> Status in “dbus” package in Ubuntu:
> New
>
> Bug description:
> Binary package hint: dbus
>
> Steps to reproduce:
> 1. Start 20110321 daily build on Amazon EC2.
> ami-30669459
> ubuntu-images-testing/ubuntu-natty-daily-amd64-server-20110321
> 2. SSH.
> 3. sudo apt-get update
> 4. To simulate an upgrade, sudo apt-get --reinstall install libdbus-1-3
> libdbus-1-3_1.4.6-1ubuntu6 is reinstalled.
> 5. sudo lsof yields (snipped)
> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE
> NAME
> init 1 root DEL REG 202,1 134262
> /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
> upstart-u 201 root DEL REG 202,1 134262
> /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
> upstart-s 369 root DEL REG 202,1 134262
> /lib/x86_64-linux-gnu/libdbus-1.so.3.5.4
> 6. sudo shutdown -h now
> Error:
> mount: / is busy
>
> Possible solution: touch /var/run/init.upgraded in package postinst to
> force init to respawn/close open files at shutdown. This is checked in
> /etc/rc[06].d/S60umountroot. See bug 672177 for a similar situation
> with libc6.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu-on-ec2/+bug/740390/+subscribe
>

Clint Byrum (clint-fewbar) wrote :

I just tested this on a clean natty VM with all updates applied, and I do indeed see an orphaned inode left over on /

We really need to trigger /var/run/init.upgraded on any of upstart's dependent libraries. Rather than making all the libraries do this, a trigger on /lib seems to make more sense where it can do an 'ldd' on /sbin/init and if the libraries changed fall into it, touch /var/run/init.upgraded. So, I'll open a task against upstart to do that. In the interim, we need to have libdbus-1-3 touch /var/run/init.upgraded as well.

Changed in dbus (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in upstart (Ubuntu):
importance: Undecided → Wishlist
status: New → Confirmed
Scott Moser (smoser) wrote :

Well, hindsight being 20-20, I should have seen this as more serious. Bug 1058517 is now open, it is really a dupe of this. we should collapse them one way or another.

Changed in upstart (Ubuntu):
status: Confirmed → Invalid
Changed in dbus (Ubuntu Lucid):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → Confirmed
Changed in dbus (Ubuntu Natty):
importance: Undecided → Medium
status: New → Confirmed
assignee: nobody → Marc Deslauriers (mdeslaur)
Changed in dbus (Ubuntu Oneiric):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → Confirmed
Changed in dbus (Ubuntu Precise):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → Confirmed
Changed in dbus (Ubuntu Quantal):
status: Triaged → Confirmed
assignee: nobody → Marc Deslauriers (mdeslaur)
Changed in upstart (Ubuntu Lucid):
status: New → Invalid
Changed in upstart (Ubuntu Natty):
status: New → Invalid
Changed in upstart (Ubuntu Oneiric):
status: New → Invalid
no longer affects: upstart (Ubuntu)
Changed in upstart (Ubuntu Precise):
status: New → Invalid
Colin Watson (cjwatson) wrote :

Clint: I don't think there's any point in doing a complex trigger arrangement when (a) only a tiny number of libraries are involved, and (b) stateful-reexec will solve this anyway.

FWIW, libnih1/libnih-dbus1 have this problem too.

Steve Langasek (vorlon) wrote :

Added tasks for libnih.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.6.4-1ubuntu3

---------------
dbus (1.6.4-1ubuntu3) quantal-proposed; urgency=low

  * REGRESSION FIX: some applications launched with the activation helper
    may need DBUS_STARTER_ADDRESS. (LP: #1058343)
    - debian/patches/CVE-2012-3524-regression-fix.patch: hardcode the
      starter address to the default system bus address.
  * Fix unclean shutdown after dbus upgrade (LP: #740390)
    - debian/libdbus-1-3.postinst: trigger an upstart re-exec before
      shutdown or reboot so that it can safely unmount the root
      filesystem.
 -- Marc Deslauriers <email address hidden> Wed, 03 Oct 2012 07:14:40 -0400

Changed in dbus (Ubuntu Quantal):
status: Confirmed → Fix Released
Clint Byrum (clint-fewbar) wrote :

If stateful re-exec does indeed handle this then agreed, no need for any triggers.

James Hunt (jamesodhunt) on 2012-10-04
Changed in libnih (Ubuntu Quantal):
assignee: nobody → James Hunt (jamesodhunt)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.4.18-1ubuntu1.3

---------------
dbus (1.4.18-1ubuntu1.3) precise-security; urgency=low

  * REGRESSION FIX: some applications launched with the activation helper
    may need DBUS_STARTER_ADDRESS. (LP: #1058343)
    - debian/patches/CVE-2012-3524-regression-fix.patch: hardcode the
      starter address to the default system bus address.
  * REGRESSION FIX: unclean shutdown after dbus upgrade (LP: #740390)
    - debian/libdbus-1-3.postinst: trigger an upstart re-exec before
      shutdown or reboot so that it can safely unmount the root
      filesystem.
 -- Marc Deslauriers <email address hidden> Wed, 03 Oct 2012 06:12:39 -0400

Changed in dbus (Ubuntu Precise):
status: Confirmed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.2.16-2ubuntu4.7

---------------
dbus (1.2.16-2ubuntu4.7) lucid-security; urgency=low

  * REGRESSION FIX: some applications launched with the activation helper
    may need DBUS_STARTER_ADDRESS. (LP: #1058343)
    - debian/patches/CVE-2012-3524-regression-fix.patch: hardcode the
      starter address to the default system bus address.
  * REGRESSION FIX: unclean shutdown after dbus upgrade (LP: #740390)
    - debian/libdbus-1-3.postinst: trigger an upstart re-exec before
      shutdown or reboot so that it can safely unmount the root
      filesystem.
 -- Marc Deslauriers <email address hidden> Wed, 03 Oct 2012 07:05:52 -0400

Changed in dbus (Ubuntu Lucid):
status: Confirmed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.4.6-1ubuntu6.4

---------------
dbus (1.4.6-1ubuntu6.4) natty-security; urgency=low

  * REGRESSION FIX: some applications launched with the activation helper
    may need DBUS_STARTER_ADDRESS. (LP: #1058343)
    - debian/patches/CVE-2012-3524-regression-fix.patch: hardcode the
      starter address to the default system bus address.
  * REGRESSION FIX: unclean shutdown after dbus upgrade (LP: #740390)
    - debian/libdbus-1-3.postinst: trigger an upstart re-exec before
      shutdown or reboot so that it can safely unmount the root
      filesystem.
 -- Marc Deslauriers <email address hidden> Wed, 03 Oct 2012 07:03:55 -0400

Changed in dbus (Ubuntu Natty):
status: Confirmed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dbus - 1.4.14-1ubuntu1.3

---------------
dbus (1.4.14-1ubuntu1.3) oneiric-security; urgency=low

  * REGRESSION FIX: some applications launched with the activation helper
    may need DBUS_STARTER_ADDRESS. (LP: #1058343)
    - debian/patches/CVE-2012-3524-regression-fix.patch: hardcode the
      starter address to the default system bus address.
  * REGRESSION FIX: unclean shutdown after dbus upgrade (LP: #740390)
    - debian/libdbus-1-3.postinst: trigger an upstart re-exec before
      shutdown or reboot so that it can safely unmount the root
      filesystem.
 -- Marc Deslauriers <email address hidden> Wed, 03 Oct 2012 07:02:41 -0400

Changed in dbus (Ubuntu Oneiric):
status: Confirmed → Fix Released
dino99 (9d9) wrote :

The dbus upgrade has been applied, and this basic unique desktop still show on shutdown: mount: / is busy (each time it is shutdown). This is on QQ i386 with pata/sata hdds and bios.

tags: added: i386 quantal
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libnih - 1.0.3-4ubuntu11

---------------
libnih (1.0.3-4ubuntu11) quantal; urgency=low

  * Addition of debian/libnih-dbus1.postinst and
    debian/libnih1.postinst to force Upstart re-exec on shutdown
    to avoid unmounting disks uncleanly (LP: #740390).
 -- James Hunt <email address hidden> Wed, 03 Oct 2012 16:49:40 +0100

Changed in libnih (Ubuntu Quantal):
status: New → Fix Released
James Hunt (jamesodhunt) wrote :

@dino99: Thanks for spotting this - I too saw this in testing. I've isolated the problem - see bug 1061639.

dino99 (9d9) wrote :

Thanks James, confirmation done.

James Hunt (jamesodhunt) wrote :

I've created bug 1062202 to handle the lbnih SRU for precise.

dino99 (9d9) wrote :

Still get uncleaned shutdown process:

with the last shutdown, i've seen:
- modemmanager reloading its dozen drivers
- mount / is busy
- and last with a [drm] nouveau complaint, switching to fbcon (or so)

with the next reboot:

oem@dub:~$ dmesg | grep orphan
[ 8.812542] EXT4-fs (sdb5): orphan cleanup on readonly fs
[ 8.819511] EXT4-fs (sdb5): ext4_orphan_cleanup: deleting unreferenced inode 400809
[ 8.819581] EXT4-fs (sdb5): 1 orphan inode deleted

Hello Alan, or anyone else affected,

Accepted libnih into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libnih/1.0.3-4ubuntu9.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in libnih (Ubuntu Precise):
status: New → Fix Committed
tags: added: verification-needed

The fix for this bug has been awaiting testing feedback in the -proposed repository for precise for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

tags: added: removal-candidate
Colin Watson (cjwatson) wrote :

I verified this following James' and Marc's instructions in bug 1062202.

tags: added: verification-done
removed: removal-candidate verification-needed

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libnih - 1.0.3-4ubuntu9.1

---------------
libnih (1.0.3-4ubuntu9.1) precise-proposed; urgency=low

  * Addition of debian/libnih-dbus1.postinst and
    debian/libnih1.postinst to force Upstart re-exec on shutdown
    to avoid unmounting disks uncleanly (LP: #740390, LP: #1062202).
 -- James Hunt <email address hidden> Fri, 05 Oct 2012 14:14:10 +0100

Changed in libnih (Ubuntu Precise):
status: Fix Committed → Fix Released
Adolfo Jayme (fitojb) on 2013-07-27
Changed in libnih (Ubuntu Lucid):
status: New → Won't Fix
Changed in libnih (Ubuntu Natty):
status: New → Won't Fix
Changed in libnih (Ubuntu Oneiric):
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers