unmounting of removable drives should show a progress dialog

[John Dong]

USB Removable disks are currently mounted asynchroniously, which can lead to users removing the media under the false impression that their data is safely written.

In GNOME, choosing the GUI "remove"/"unmount" option immediately causes a visual cue that the device is no longer mounted (i.e. icon disappears from Computer and Desktop). Users usually interpret this as it's safe to remove the disk.

However, a sync is still running in the background, and this is visible for drives with an access light (not all do!). Usually, this lasts for about 10-15 seconds post unmount, but in worst-case scenarios it could take a minute or more to sync everything. If a user removes the disk at this stage, there will be data loss or even filesystem corruption.

KDE, on the other hand, shows an "Unmounting..." dialog that disappears when the underlying umount command exits (after all the data is written and the volume is actually unmounted).

So, GNOME needs to get a similar "Wait a second" dialog, or the underlying media should be mounted sync by hal.

[John Dong (jdong)]

I've had friends lose data this way, and I've corrupted a thumb drive also. I consider this problem to be pretty major.

[Sebastien Bacher (seb128)]

Thanks for the bug report. This particular bug has already been reported into our bug tracking system, but please feel free to report any further bugs you find.

[Sebastien Bacher (seb128)]

Duplicate of https://bugzilla.ubuntu.com/show_bug.cgi?id=13169 which seems to have not been imported during the migration...

[John Dong (jdong)]

Are there any plans for a fix? The remount r/o, then unmount idea sounds
like it works; definitely this one should be fixed before Dapper, IMO.

On 2/24/06, Sebastien Bacher <email address hidden> wrote:
>
> Public bug report changed:
> https://launchpad.net/malone/bugs/32643
>
> Comment:
> Duplicate of https://bugzilla.ubuntu.com/show_bug.cgi?id=13169 which
> seems to have not been imported during the migration...
>

[Sitsofe Wheeler (sitsofe)]

This might be a dupe of Bug #20717 .

[Chris Weiss (cweiss)]

hm. so "wontfix" because of performance and flash wear issues? then why the 1Gig threashold for disabling sync? see https://launchpad.net/distros/ubuntu/+source/hal/+bug/33608

so the new bug is: Nautilus does not show sync progress when unmounting a usb disk thus causing data loss because the user thinks the disks has been unmounted and unplugs the drive.

eitherway, you can edit the hal fdi and disable, or just modify this undesirable behaviour on your system.

[John Dong (jdong)]

Alright; mounting the drives async is perfectly fine to me; everyone likes performance :)

However, how about getting that gumount.c dialog a part of Nautilus? If unmounting a heavily used volume is going to take 30+ seconds (USB 1.1 and 100MB of buffers to flush, for example), users need to know not to yank out a drive.

[Martin Pitt (pitti)]

Yesterday I uploaded a new nautilus which now shows a progress dialog while data is flushed to a to-be-removed device. It doesn't look very beautiful yet and will get cleaned up a bit, but it should generally work.

I would appreciate feedback.

[Maximilian Federle (max-federle)]

I don't get such a dialog.

Version:
max@max:/media$ nautilus --version
Gnome nautilus 2.13.92

Package: nautilus_2.13.92-0ubuntu4_i386.deb

[Maximilian Federle (max-federle)]

I'm sorry. It works perfectly.
But a little animation would be nice in this dialog.

[Jonathan Kempa (jonikempa)]

"Writing to storage device" also appears when unmounting a cd rom.

[Lukas Sabota (punkrockguy318)]

This seems to work nicely for me as well. Good work devs :)

[John Dong (jdong)]

cool. Can we also get this dialog for umounting from the panel applet?

[Martin Pitt (pitti)]

> "Writing to storage device" also appears when unmounting a cd rom.

Yes, known bug and filed several times. I will fix this soon.

> cool. Can we also get this dialog for umounting from the panel applet?

Yes, as soon as everybody is fine with the nautilus one.

[Martin Pitt (pitti)]

With this upload, drivemount applet has the same dialog:

 gnome-applets (2.14.0-0ubuntu4) dapper; urgency=low
 .
   * Add debian/patches/16_driveapplet_unmountprogress.patch:
     - Add an unmount progress dialog (the same as in nautilus).
     - Malone #32643

[Martin Pitt (pitti)]

I consider this fixed now; both the applet and nautilus now have progress dialogs, which should keep users from prematurely ripping out their drives.

Feedback is appreciated! :)

[John Dong (jdong)]

Thanks, Martin! I consider this fixed, also.
On 3/16/06, Martin Pitt <email address hidden> wrote:
>
> Public bug report changed:
> https://launchpad.net/malone/bugs/32643
>
> Bug watches changed:
> + Bug 334788 [GNOME Bug Tracker]
>

[Gabriel Bauman (gabrielbauman)]

Don't mean to be a picky bastard, but the phrasing in the dialog box is pretty inconsistent.

 o Window title is "Removing device". I got the dialog by selecting "Unmount" or "Eject", why is something being "Removed"?

 o The bold text, "Writing data to storage device <device label>" is much more indicative of what is actually happening (if a bit wordy).

Might I suggest something like the following:

 o Window title: "Saving changes"
 o Bold caption: "Saving changes to files on <device label>"
 o Warning: "Please do not remove the device until this operation is complete."

I think this might be a bit more clear.

[John Dong (jdong)]

On 3/16/06, Gabriel Bauman <email address hidden> wrote:
>
> Public bug report changed:
> https://launchpad.net/malone/bugs/32643
>
> Comment:
> Don't mean to be a picky bastard, but the phrasing in the dialog box is
> pretty inconsistent.
>
> o Window title is "Removing device". I got the dialog by selecting
> "Unmount" or "Eject", why is something being "Removed"?

o The bold text, "Writing data to storage device <device label>" is
> much more indicative of what is actually happening (if a bit wordy).

Good points.

> Might I suggest something like the following:
>
> o Window title: "Saving changes"
> o Bold caption: "Saving changes to files on <device label>"

Then again, what "changes" are being saved? "Writing unwritten data" sounds
better, IMO.

o Warning: "Please do not remove the device until this operation is
> complete."
>
> I think this might be a bit more clear.
>

[Martin Pitt (pitti)]

Hi Gabriel!

Gabriel Bauman [2006-03-17 2:32 -0000]:
> Don't mean to be a picky bastard,

Oh, be, I'd be glad to further improve the window.

> o Window title is "Removing device". I got the dialog by selecting
> "Unmount" or "Eject", why is something being "Removed"?

(I'm not a native English speaker) I thought it would convey the idea
that the user wants to unplug/remove the device soon. However, I don't
mind changing it to something else.

> o The bold text, "Writing data to storage device <device label>" is
> much more indicative of what is actually happening (if a bit wordy).
>
> Might I suggest something like the following:
>

> o Window title: "Saving changes"
> o Bold caption: "Saving changes to files on <device label>"

Saving changes seems to be pretty unclear to me. Maybe we can stick to
'Writing pending data'?

> o Warning: "Please do not remove the device until this operation is complete."

That didn't change, right?

Thanks,

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntu.com
Debian Developer http://www.debian.org

In a world without walls and fences, who needs Windows and Gates?

[John Dong (jdong)]

On 3/17/06, Martin Pitt <email address hidden> wrote:
>
> Public bug report changed:
> https://launchpad.net/malone/bugs/32643
>
> Comment:
> Hi Gabriel!
>
> Gabriel Bauman [2006-03-17 2:32 -0000]:
> > Don't mean to be a picky bastard,
>
> Oh, be, I'd be glad to further improve the window.
>
> > o Window title is "Removing device". I got the dialog by selecting
> > "Unmount" or "Eject", why is something being "Removed"?
>
> (I'm not a native English speaker) I thought it would convey the idea
> that the user wants to unplug/remove the device soon. However, I don't
> mind changing it to something else.
>
> > o The bold text, "Writing data to storage device <device label>" is
> > much more indicative of what is actually happening (if a bit wordy).
> >
> > Might I suggest something like the following:
> >
>
> > o Window title: "Saving changes"
> > o Bold caption: "Saving changes to files on <device label>"
>
> Saving changes seems to be pretty unclear to me. Maybe we can stick to
> 'Writing pending data'?
>
> > o Warning: "Please do not remove the device until this operation is
> complete."
>
> That didn't change, right?

I think the warning should be more threatening. "Removing the device before
this operation completes may cause data loss!"

Thanks,
>
> Martin
>
> --
> Martin Pitt http://www.piware.de
> Ubuntu Developer http://www.ubuntu.com
> Debian Developer http://www.debian.org
>
> In a world without walls and fences, who needs Windows and Gates?
>

[magilus (magilus)]

Feature request:
In the GNOME Cd/Dvd creator there is a blue bar which has a button to burn the cd.

Something similar would be great when opening a storage device with nautilus: A bar which says that the data has to be written on the device and that you have to click unmount for that. An unmount button should be on the bar, too.

[Daniel Holbach (dholbach)]

Martin Jürgens: please file this as a separate bug report, preferrably in the upstream bug tracker, since it requires bigger changes and has nothing to do with the "progress dialog" this bug report is about.

[Cosmin Saveanu (csaveanu)]

Excellent work Martin. I'm new to Linux and it is the first time that I see the enormous benefits of 'open' software development. Tested the fix with Dapper Drake Flight 5. Very nice, thank you,

 Cheers,

Cosmin

[Dietmar Hofer (didi156)]

I appreciate all your effort.
Nevertheless I would like to add my thoughts about this issue.
After my update to Breezy (from Warty) and after my first USB stick data losses I couldn't understand why this step backward (regarding usability) was done.
From the bugreports about this issue I understand that there was and are reasons to do that.

But in my opinion everything else than complete transparency to the user isn't a real solution to this bug for a distro which claims to be user friendly.
The point is: I put a USB stick, I copy some files, I see a progress dialog which goes to 100% and the disappears.
At this point I assume that the files are copied and that they are there if I remove the stick.
No matter how it's done, every additional task to do in order to really get the data on the stick is not acceptable in my eyes, in long term it will fail like all non-tranparent mount/unmount concept failed when it came to end user friendlyness.

This was about the user level.
On the technical level it's just the question about sync and async, as far as I understand.
Personally, I would prefer the sync drawbacks, even if it's factor 10 slower (and I remember that in Warty I wondered about the slow speed) and reduces my sticks life time.
I don't know what other end users think about this (I think we also shouldn't forget that maybe the users discussing here are technically at a higher level than the avg user), but since there exists a sort of "default PC operating system", we maybe should look at how it works there.
Somebody wrote about the "remove safely icon" that appears in the Windows taskbar when you put in an USB stick. That's true. But I can assure you that it's not necessary to use it, and I think most users aren't using it. I never had data loss with Windows when just putting off the stick after the copy progress GUI dialog disappeared.

My suggestion would be to make sync the default again and add the async-option as option in the graphical config tool.
Linux desktop systems have suffered so much for so many years about non transparent mount/unmount operation, please don't reintroduce it, if it's not absolutely necessary (which in my opinion is not the case here).
My time has more value than my USB sticks life time.

[Dietmar Hofer (didi156)]

sorry for the double posting, it's always dangerous to hit the "reload"-button...

[Martin Pitt (pitti)]

@Dietmar: I agree that time is important for people. That's why most of them will certainly appreciate the factor 10 to 100 speedup when using async, and will appreciate not destroying their Flash memory, too.

I agree that the current solution is still suboptimal, but it's the best the current kernel offers. There is progress in that area: http://www.ussg.iu.edu/hypermail/linux/kernel/0512.3/0050.html If that goes into the kernel, then we can offer a much better sync behaviour, but for dapper the current solution has to suffice.

Thanks for your thoughts!

[Dietmar Hofer (didi156)]

I'm very happy to read that there's news on the kernel side.
I understand that you distro guys can just try to make the best out the situation. However I'm still convinced that for the average user the sync-variant would have been the better one, regardless of it's drawbacks.
Nevertheless, thanks for your effort.

[Chris Weiss (cweiss)]

why do we need this perceieved speed up? if sync causing it to be slower than Windows or MacOS? I think it isn't, and async really is only faster if you make multiple changes to the same files in a "session". how often does async actualy commit to disk anyway? if I make an edit, asave, then wait 30 minutes before saving again, is async still going to increase my flash lifetime? at wait interval won't it?

And what's the answer to this senario?
- user is typing a document, saving work as she goes
- power goes out (most desktops don't have a UPS)
- power comes on and finds all those saves she was doing weren't actualy writting to disk

how is this faster? she now has to re-write the whole thing. how is this simpler and more user friendly?

[Ben Collins (ben-collins)]

> And what's the answer to this senario?
> - user is typing a document, saving work as she goes
> - power goes out (most desktops don't have a UPS)
> - power comes on and finds all those saves she was doing weren't actualy writting to disk
>
> how is this faster? she now has to re-write the whole thing. how is
> this simpler and more user friendly?

Why should we have a different way to handle large removable devices
than we do internal static drives? The scenario you show would also
happen to a normal disk drive, because they are async by default
(however, things like journaling help this a lot).

I don't think this scenario should dictate the resolution of this
problem.

[John Dong (jdong)]

Windows recognizes the difference between a USB mass storage device that's a
smaller removable device (i.e. flash drive) vs a larger device (i.e.
external hard drive). It sets sync on smaller devices and async on larger
devices. I feel we should do the same, also.

I would rather lose some of my life on a thumb drive rather than having to
remember to sync it periodically to avoid losing data in case of a system
lockup/crash/poweroff.

On 5/14/06, Chris Weiss <email address hidden> wrote:
>
> why do we need this perceieved speed up? if sync causing it to be
> slower than Windows or MacOS? I think it isn't, and async really is
> only faster if you make multiple changes to the same files in a
> "session". how often does async actualy commit to disk anyway? if I
> make an edit, asave, then wait 30 minutes before saving again, is async
> still going to increase my flash lifetime? at wait interval won't it?
>
> And what's the answer to this senario?
> - user is typing a document, saving work as she goes
> - power goes out (most desktops don't have a UPS)
> - power comes on and finds all those saves she was doing weren't actualy
> writting to disk
>
> how is this faster? she now has to re-write the whole thing. how is
> this simpler and more user friendly?
>
> --
> unmounting of removable drives should show a progress dialog
> https://launchpad.net/bugs/32643
>

[Dietmar Hofer (didi156)]

I must confess: I wasn't aware about how messed up sync works in the actual kernel.
Some days before I was copying something to my USB-stick on SuSE 10, which still mounts sync. I was used to get a data rate of avg. 15 MB/s with this stick on SuSE 9.3. On 10 it's about 40 kB/s. That's not 10, not 100 times slower, but even more! I was not aware of the fact that this went worse with actual kernels while it worked well with olders.

About the scenarios discussed:
I think most users come here because the following happened: They copied something onto their USB-stick, removed this Stick and wanted to access this data on another Computer (I think this is what we mainly use USB-sticks for...) And then they had to recognize that no copied data was there. Now, a user which have previously used Linux or which has some knowledge about how it works under the hood, may be aware of the mount/unmount mechanism. But I think everybody else will or give up or sit there for some extra minutes without knowing what he's doing wrong.
Maybe you should introduce a big info box when a USB-storage-device is mounted the first time, explaining that.

[John Dong (jdong)]

Because users are far more likely to prematurely/improperly remove USB thumb
drives than internal static drives or even USB external hard drives.

On 5/14/06, Ben Collins <email address hidden> wrote:
>
>
> > And what's the answer to this senario?
> > - user is typing a document, saving work as she goes
> > - power goes out (most desktops don't have a UPS)
> > - power comes on and finds all those saves she was doing weren't actualy
> writting to disk
> >
> > how is this faster? she now has to re-write the whole thing. how is
> > this simpler and more user friendly?
>
> Why should we have a different way to handle large removable devices
> than we do internal static drives? The scenario you show would also
> happen to a normal disk drive, because they are async by default
> (however, things like journaling help this a lot).
>
> I don't think this scenario should dictate the resolution of this
> problem.
>
> --
> unmounting of removable drives should show a progress dialog
> https://launchpad.net/bugs/32643
>

[magilus (magilus)]

 > Maybe you should introduce a big info box
I already reported that into the Gnome Bugtracker: http://bugzilla.gnome.org/show_bug.cgi?id=336392 but I do not think that this is going to be integrated into Dapper.

[gizbot (charles-merriam)]

I read the comments here and the crosslinked bugs.

While many people are being helpful. I think you fail to 'get it'.

Right now, USB drives are catastrophically broken. A user saying "you can't use USB devices on Ubuntu" are correct.

[gizbot (charles-merriam)]

Sorry, I was unclear.

On every non-unix device, USB keys and drives are updated as soon as possible. There is never a several minute delay before data is written. No unmount, no file browser, it just works.

If I loose data in my checkbook program (KMyMoney), then Ubuntu and not the user is broken.

[John Dong (jdong)]

Charles,

The trade-off for mounting USB devices sync would be faster wear and tear on
the drive, along with applications hanging for longer periods of time while
trying to write data.

Even operating systems that mount sync (Linux OR Windows or whatever) do not
make it safe to yank out USB devices. That still leaves the possible of
corrupted filesystem metadata structures -- I've interned at the IT desk for
a large company before and I've personally seen cases (multiple) on windows
XP where FAT32 thumb drives were unrecoverably corrupted due to improper
removal.

Mounting 'sync' does exactly what XP tells you: Optimize for quick removal.
In other words, the "unmounting" (safely remove hardware) option executes
faster, but you still have to do it. Never ever is it safe to yank thumb
drives out of systems -- windows or linux. Just because you got away with it
before doesn't make it safe!