When hardy's ubiquity is running from a live cd it sometimes hangs, its migration assistant gives an incomplete list of userid candidates, and nautilus pops up /home/ubuntu windows

[John S. Gruber]

Binary package hint: ubiquity

When hardy's ubiquity is running from live cd it sometimes hangs, its migration assistant gives an incomplete list of userid candidates, and nautilus pops up /home/ubuntu windows.

Description: Ubuntu hardy (development branch)
Release: 8.04

Applies to ubiquity and its component (dependency) os-prober

Ubiquity leaves behind a set of its mounts, and those mounts overlap on one mount point, with latter mounts necessarily obscuring earlier ones. The mounts left behind are not always the same.

Background:

I'm running on an older pc with quite a few partitions. When I run ubiquity from the daily test live cd from 4/29/2008 I get the above errors. When I investigated the leftover mounts I found I could get ubiquity to resume from the hang by un-mounting the leftover mounts.

Running ubiquity with most partitions mounted beforehand allows it to run properly. I believe this is why running ubiquity from my already installed hardy system doesn't hang or produce the other symptoms. (My installed system mounts all existing partitions through /etc/fstab.)

Some findings, diagnostic files, suggested patches, and suggested circumventions will follow below.

[John S. Gruber (jsjgruber)]

I determined that ubiquity, and particularly os-prober, was leaving partitions mounted and that subsequent mounts would be mounted atop older ones. Ubiquity's components necessarily try to access already mounted partitions through existing mount points. When it does so it makes no attempt to remount the partitions and usually proceeds fine. When it accesses a problem mount point the previous mount is hidden and ubiquity (and any other process on the system) will be fooled into thinking it is looking at the first partition while really looking at the last one (until the bad mounts are corrected).

Testing with just the os-prober component of ubiquity created the same bad mounts. The bad mounts weren't always the same. I believed that something, probably the new nautilus version, was accessing new mounts, sometimes preventing umounts from ubiquity from running successfully.

More information:

os-prober doesn't check the status of umount commands. I also discovered that setting the new gconf property

/apps/nautilus/preferences/media_autorun_never

to *true* allowed proper operation in every subsequent ubiquity test and got rid of the nautilus pop ups. This did not allow os-prober to complete properly every time but was a great help.

Logging off all gnome sessions was successful in getting mount/umount sequences to run on tty1. Logging on created problems with those sequences on tty1.

(I believe that something other than nautilus or gnome volume manager also is probably, very briefly, accessing new mounts).

Adding a test to os-prober's umounts to retry after a short sleep solves the hang and incomplete list problems but doesn't address the pop-ups and obviously slows ubiquity. Setting the above property solved all three problems on my system in my tests, but there still seems to be something to potentially interfere with ubiquity other than nautilus, though much, much, less frequently. I respectfully suggest that you consider applying patches similar to both of those below.

[John S. Gruber (jsjgruber)]

Reproducing:

Running os-prober several times on a live-cd system may produce error messages. This is less time consuming than running ubiquity to reproduce the errors, but I verified the success of the suggested patches below by running both os-prober and ubiquity (up to the verification screen).

Since this bug is prompted by asynchronous processes results can be haphazard.

Circumvention:

Mount all partitions to the live-cd system using

sudo mkdir /media/hdxx
sudo mount /dev/hdxx /media/hdxx

(using using Applications/Accessories/Terminal)

the target partitions should probably be unmounted with the sudo umount command before proceeding with formatting and installation. I haven't tried going on beyond the verification screen.

or more easily and perhaps more safely:

Bring up Home Folder from the Places menu
Select Edit->Preferences
Select the Media Tab
Check the second last selection at the bottom: "Never prompt or start programs on media insertion"

Please let me know if I can help with further information or otherwise. Logs and suggested patches follow.

[John S. Gruber (jsjgruber)]

ubiquity debug log

[John S. Gruber (jsjgruber)]

syslog, please note the errors from umounts

[John S. Gruber (jsjgruber)]

partman log (just in case, I don't think it is necessary)

[John S. Gruber (jsjgruber)]

Patch to ubiquity's gtk_ui.py

--- gtk_ui.py.old 2008-04-01 13:02:20.917370401 +0000
+++ gtk_ui.py 2008-04-01 13:14:14.753433264 +0000
@@ -271,6 +271,11 @@
             self.gconf_previous[gconf_key] = gconftool.get(gconf_key)
             if self.gconf_previous[gconf_key] != 'false':
                 gconftool.set(gconf_key, 'bool', 'false')
+ gconf_key = '/apps/nautilus/preferences/media_autorun_never'
+ self.gconf_previous[gconf_key] = gconftool.get(gconf_key)
+ if self.gconf_previous[gconf_key] != 'true':
+ gconftool.set(gconf_key, 'bool', 'true')
+

         self.thunar_previous = self.thunar_set_volmanrc(
             {'AutomountDrives': 'FALSE', 'AutomountMedia': 'FALSE'})
@@ -292,6 +297,12 @@
             elif self.gconf_previous[gconf_key] != 'false':
                 gconftool.set(gconf_key, 'bool',
                               self.gconf_previous[gconf_key])
+ gconf_key = '/apps/nautilus/preferences/media_autorun_never'
+ if self.gconf_previous[gconf_key] == '':
+ gconftool.unset(gconf_key)
+ elif self.gconf_previous[gconf_key] != 'true':
+ gconftool.set(gconf_key, 'bool',
+ self.gconf_previous[gconf_key])

         if self.thunar_previous:
             self.thunar_set_volmanrc(self.thunar_previous)

[John S. Gruber (jsjgruber)]

os-prober patch:

--- 50mounted-tests 2007-11-22 18:15:48.000000000 +0000
+++ /usr/lib/os-probes/50mounted-tests 2008-03-31 04:31:50.773366540 +0000
@@ -32,17 +32,24 @@
    if [ -f $test ] && [ -x $test ]; then
     if $test $partition $tmpmnt $type; then
      debug "os found by subtest $test"
- umount $tmpmnt
+ if ! umount $tmpmnt ; then # retry once
+ log "Bad umount after success"
+ sleep 5
+ umount $tmpmnt || true
+ fi
      rmdir $tmpmnt || true
      exit 0
     fi
    fi
   done
- umount $tmpmnt
+ if ! umount $tmpmnt ; then # retry failure to umount once more
+ log "Bad umount after failure finding partition type"
+ sleep 5
+ umount $tmpmnt || true
+ fi
   break
  fi
 done
-
 rmdir $tmpmnt || true

 # No tests found anything.

[John S. Gruber (jsjgruber)]

Output of mount command after running into problems running ubiquity. Notice the mounts to /var/lib/os-prober/mount

proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
tmpfs on /lib/modules/2.6.24-12-generic/volatile type tmpfs (rw,mode=0755)
tmpfs on /lib/modules/2.6.24-12-generic/volatile type tmpfs (rw,mode=0755)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)
gvfs-fuse-daemon on /home/ubuntu/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=ubuntu)
/dev/hdb7 on /var/lib/os-prober/mount type ext3 (ro)
/dev/hdb9 on /var/lib/os-prober/mount type ext3 (ro)
/dev/hda5 on /var/lib/os-prober/mount type ext3 (ro)
/dev/hdb1 on /media/hdb1 type fuseblk (rw,nosuid,nodev,noatime,allow_other,blksize=4096)

[John S. Gruber (jsjgruber)]

Output from os-prober:

/dev/hda1:Microsoft Windows XP Professional:Windows:chain
/dev/hda5:Ubuntu hardy (development branch) (8.04):Ubuntu:linux
/dev/hda6:Windows 95/98/Me:Windows9xMe:chain
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
/dev/hdb2:Debian GNU/Linux (4.0):Debian:linux
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
rmdir: failed to remove `/var/lib/os-prober/mount': Device or resource busy
rmdir: failed to remove `/var/lib/os-prober/mount': Device or resource busy
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
umount: /var/lib/os-prober/mount: device is busy
/dev/hdb7:Ubuntu 7.10 (7.10):Ubuntu1:linux
rmdir: failed to remove `/var/lib/os-prober/mount': Device or resource busy
rmdir: failed to remove `/var/lib/os-prober/mount': Device or resource busy
/dev/hdb9:Ubuntu 7.10 (7.10):Ubuntu2:linux
rmdir: failed to remove `/var/lib/os-prober/mount': Device or resource busy

[John S. Gruber (jsjgruber)]

Screen shot of popped up nautilus window. The same window pops up running just os-prober.

[John S. Gruber (jsjgruber)]

An earlier hardy live-cd had nautilus popping up windows reflecting the mount points, rather than /home/ubuntu

[Colin Watson (cjwatson)]

I'm surprised that the change in ubiquity 1.8.0 to run the whole of ubiquity under hal-lock (thereby preventing media autorun) didn't fix this.

[John S. Gruber (jsjgruber)]

I figured that was what the hal-lock was for. During my test of mount umount sequences the hal-lock used by ubiquity did me no good either.

I just ran dbus-monitor --system on the live-cd system during a failing install. It has many of the following messages. The whole output file is attached, starting with the reflection of the monitor program starting, and the lock done by ubiquity.

signal sender=:1.5 -> dest=(null destination) path=/org/freedesktop/Hal/devices/volume_uuid_CA009B90009B825D; interface=org.freedesktop.Hal.Device; member=PropertyModified
   int32 3
   array [
      struct {
         string "volume.mount_point"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted_read_only"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted"
         boolean false
         boolean false
      }
   ]
signal sender=:1.5 -> dest=(null destination) path=/org/freedesktop/Hal/devices/volume_uuid_CA009B90009B825D; interface=org.freedesktop.Hal.Device; member=PropertyModified
   int32 3
   array [
      struct {
         string "volume.mount_point"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted_read_only"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted"
         boolean false
         boolean false
      }
   ]
signal sender=:1.5 -> dest=(null destination) path=/org/freedesktop/Hal/devices/volume_uuid_781e325e_2ea7_40af_9b16_4a452fe1f5c2; interface=org.freedesktop.Hal.Device; member=PropertyModified
   int32 3
   array [
      struct {
         string "volume.mount_point"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted_read_only"
         boolean false
         boolean false
      }
      struct {
         string "volume.is_mounted"
         boolean false
         boolean false
      }
   ]

[Colin Watson (cjwatson)]

OK, I see that nautilus uses the GIO volume monitor, which seems to be roughly inotify on fstab and mtab, and so unaffected by the HAL lock. Sigh.

I've uploaded a variant of the idea in your os-prober patch (defining a repeat_umount function and calling it everywhere instead of umount), and committed something similar to your ubiquity patch. Thanks!

[Colin Watson (cjwatson)]

By the way, this was a fantastic piece of analysis and first attempt at a ubiquity bug-fix (at least I don't remember seeing your name in this context before). If you decide you want to get involved with development on an ongoing basis, I think you could do a good job and we'd be happy to help out; https://wiki.ubuntu.com/InstallerDevelopment has more details.

[Colin Watson (cjwatson)]

Rant on the general causes of this bug here: http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/ubuntu/2008-04-12-desktop-automount-pain.html

[Launchpad Janitor (janitor)]

This bug was fixed in the package os-prober - 1.23ubuntu1

---------------
os-prober (1.23ubuntu1) hardy; urgency=low

  * If any umount fails, sleep for five seconds and try again, up to three
    times; it seems that occasionally desktop automounting systems grab the
    device at just the wrong time (LP: #210620).

 -- Colin Watson <email address hidden> Sat, 12 Apr 2008 10:44:10 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 1.8.4

---------------
ubiquity (1.8.4) hardy; urgency=low

  [ Jonathan Riddell ]
  * kde_ui:
   - Fix crash in PartitionModel localisation (LP: #216031).
   - Fix translations on buttons
   - Replace the resize slider with the custom widget
   - Fix occasional bug where the app would still quit after the confirm
     exit dialogue had been dismissed
   - Fix exception in on_partition_edit_use_combo_changed when non-ASCII
     characters are used
   - Translate dialogues
   - Fix launching browser from release notes label
  * kde_ui, gtk_ui
   - In partition edit dialogue, enable format tickbox when a format type is
     selected

  [ Colin Watson ]
  * KDE frontend: Translate the partition create dialog too.
  * Fix crash if opening /cdrom/.disk/info fails (with Jonathan and Evan).
  * Show current filesystem type if it isn't currently being used for
    anything; now that partman no longer sets up automatic mount points for
    everything, the lack of filesystem types made it difficult to get any
    context on which partition was which.
  * Set /apps/nautilus/preferences/media_autorun_never to true during
    installation (thanks, John S. Gruber; LP: #210620).
  * Update translations from Launchpad.

 -- Colin Watson <email address hidden> Sat, 12 Apr 2008 13:23:13 +0100

[John S. Gruber (jsjgruber)]

The hang and migration assistant omission due to this root cause are fixed. Thanks for both the fix and the good words, Colin.

[Colin Watson (cjwatson)]

We think that this should help as well, thanks to David Zeuthen of Red Hat:

gvfs (0.2.3-0ubuntu3) hardy; urgency=low

  * debian/patches/92_from_upstream_only_show_allowed_mounts.patch:
    - change from upstream, don't list filesystems mounted out of the media or user directory either,
      should fix some issues on the desktop cd where system mounts should not be consider for example

 -- Sebastien Bacher <email address hidden> Wed, 16 Apr 2008 11:08:57 +0200