Open files on ejected media cause automount of new media to break

[Phillip Susi]

Open a file on a cdrom, perhaps as simply as changing the working directory of a shell to a location in /media/cdrom, then eject the disc. The filesystem is forcibly unmounted, so it no longer appears in df, mount, or /proc/mounts, though it appears to still actually be sort of mounted since you can run stat . from the shell. The kernel begins complaining filling /var/log/kern.log with messages like:

VFS: busy inodes on changed media or resized disk sr0

At this point the drive has vanished from view in nautilus. It no longer appears on the places menu or in the computer folder. If you insert a different disc it also will not show up in nautilus and you can not mount it through the gui until whichever processes that still have file descriptors open on files on the ejected media are closed.

[Phillip Susi (psusi)]

Note that I first noticed this because I had been playing music on a dvd+rw with Rhythmbox and long after I had stopped listening to musc, I hit the eject button on the drive and put the livecd in the drive to try and make a bootable usb flash stick with the boot disk creator, but apparently Rhythmbox still had open file descriptors on the dvd+rw so the livecd would not show up. Playing music on a cd/dvd then trying to switch to other media without forcing Rhythmbox to close seems a fairly likely scenario for the average user, especially since clicking the close window button on Rhythmbox normally only hides it, keeping it running but visible only in the system tray.

[Martin Pitt (pitti)]

Closing udisks task, since udisks can't do anything about open FDs in other processes.

Indeed forced unmounts are done with -l (lazy unmount), there is not much else that you can do -- you can't pull away open file descriptors from underneath bash and other processes -- what should they do about it? You'd need to tell them "please go drop this file or change directory, etc", which only the application itself can do. The alternative would be to force the application to crash, but in terms of not losing data this isn't the best solution either.

However, what does look fishy is that a newly added USB stick wouldn't mount and work normally -- if there's a stale mount on, say, /dev/sdb, then the next device should just get /dev/sdc assigned.

Can you please send the output of "mount" and /var/log/kern.log after this happened?

[Phillip Susi (psusi)]

On 4/9/2010 4:39 AM, Martin Pitt wrote:
> Closing udisks task, since udisks can't do anything about open FDs in
> other processes.

I wouldn't expect it to, but it ( or whichever component nautilus
depends on ) should not get confused and make the drive vanish and
refuse to recognize new media.

> Indeed forced unmounts are done with -l (lazy unmount), there is not
> much else that you can do -- you can't pull away open file descriptors
> from underneath bash and other processes -- what should they do about
> it? You'd need to tell them "please go drop this file or change
> directory, etc", which only the application itself can do. The
> alternative would be to force the application to crash, but in terms of
> not losing data this isn't the best solution either.

Media removal should be using a forced unmount, not a lazy one. This
should cause the open inodes to be discarded, and the file descriptors
that point to them to be invalidated. Any further requests using those
file descriptors should fail with -EBADF, which may or may not cause
applications to crash.

Windows actually has a nice trick where it basically does a lazy unmount
when the media is removed and suspends any IO requests to that fs, then
if the original media is put back, the mount point is reattached and the
still open fds continue working.

> However, what does look fishy is that a newly added USB stick wouldn't
> mount and work normally -- if there's a stale mount on, say, /dev/sdb,
> then the next device should just get /dev/sdc assigned.

No USB is involved; this is just inserting a new media in the cdrom.
You can still manually mount /dev/sr0, but for some reason, nautilus
refuses to even see that there is a drive there with no media in it.
Closing the stale file descriptors after removing the media causes the
drive to show up again in nautilus.

[Martin Pitt (pitti)]

> No USB is involved; this is just inserting a new media in the cdrom.

Ah, sorry, I misunderstood that. This works fine here, so it seems to be hardware specific, and I'm afraid I need some more logs.

Can you please run "gvfs-mount -oi" and "udisks --monitor-detail" and "udevadm monitor -e --udev", then reproduce the bug, and attach the three logs here? Please also attach the dmesg output afterwards.

Thanks!

[Phillip Susi (psusi)]

The problem seems to be that the kernel STILL can not properly force unmount. If I umount -f /dev/sr0 it fails with -EBUSY. A lazy umount leaves the filesystem still mounted. If I umount -l /dev/sr0 then try to open the drive in nautilus again I get:

Error mounting: mount exited with exit code 1: helper failed with:
mount: /dev/sr0 already mounted or /media/cdrom0 busy

As long as a process keeps an open fd, the filesystem remains mounted ( but hidden from the namespace ), which prevents you from mounting it again, even if new media has been inserted. You also can not use lsof to find what process still has files open since they have been removed from the namespace.

There also does not seem to be a way to undo a lazy umount and reattach the mount point to the namespace.

[Martin Pitt (pitti)]

Hm, the udisks etc. log files look fine (although udev fails to detect the drive capabilities in the last run, but that seems to be an unrelated bug). Still missing /var/log/kern.log after this happens, it might reveal something.

[Jeremy Foshee (jeremyfoshee)]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu release http://www.ubuntu.com/getubuntu/download . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Phillip Susi (psusi)]

Nothing appears in kern.log after the first disc is initially mounted. Eject, and insert second disc and no new messages appear.

For reference, all I've got is:

Sep 19 01:50:34 faldara kernel: [30007.432864] UDF-fs: No VRS found
Sep 19 01:50:34 faldara kernel: [30007.432869] UDF-fs: No partition found (1)
Sep 19 01:50:34 faldara kernel: [30007.471443] ISO 9660 Extensions: Microsoft Joliet Level 3
Sep 19 01:50:34 faldara kernel: [30007.500946] ISOFS: changing to secondary root

[Phillip Susi (psusi)]

Hrm... actually it seems to behave a bit differently in maverick. It used to do a lazy unmount of the fs, which removed it from the namespace so no new processes could access the old mount. Now it appears it never unmounts the old disc. I can hit the eject button while a terminal has an open file on the disc and as far as anything can see, the old disc is still there. I can open the drive in my computer in nautilus and still see the files on the disc even though nothing is in the drive. Inserting a new disc has no effect. Nautilus does inform me that the terminal window has files open on the disc when I click the eject button, but when I click unmount anyway, the window just pops right back up again.

[penalvch (penalvch)]

Phillip Susi, this bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? If so, could you please test for this with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ .

If it remains an issue, could you please run the following command in the development release from a Terminal (Applications->Accessories->Terminal), as it will automatically gather and attach updated debug information to this report:

apport-collect -p linux <replace-with-bug-number>

Also, could you please test the latest upstream kernel available following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue. Please do not test the daily kernel folder, but the one all the way at the bottom. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this bug is fixed in the mainline kernel, please add the following tags:
kernel-fixed-upstream
kernel-fixed-upstream-VERSION-NUMBER

where VERSION-NUMBER is the version number of the kernel you tested. For example:
kernel-fixed-upstream-v3.11.1

This can be done by clicking on the yellow circle with a black pencil icon next to the word Tags located at the bottom of the bug description. As well, please remove the tag:
needs-upstream-testing

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-VERSION-NUMBER

As well, please remove the tag:
needs-upstream-testing

Once testing of the upstream kernel is complete, please mark this bug's Status as Confirmed. Please let us know your results. Thank you for your understanding.

[Phillip Susi (psusi)]

It seems to have been fixed.