USB flash drive (NTFS) keeps writing for quite a while even after "Eject" completes

[sumedh]

Release: Ubuntu 14.04

Nautilus version: 1:3.10.1-0ubuntu9.1

Expected: After pressing "Eject" on a flash drive (formatted with NTFS -- haven't tried with FAT yet) I expected it to return only after flushing and unmounting the drive so the drive is completely safe to remove at that point

What happened: "Eject" returned after a short while. The drive still kept writing for a long time. Mount shows the drive still mounted for the period while the flash drive keeps blinking though "df" does not show the drive anymore. Removing the drive at this point results in partial file write and NTFS filesystem damage (fixable with ntfsfix but dangerous).

Steps to reproduce:

1) Insert a flash drive (tried with NTFS mounted drive though I believe FAT will show similar behaviour though latter uses "flush" mount option so may not be that prominent)

2) Try copying a large file. For testing use a size that will also mostly fit into filesystem cache so that real wait for writing is during unmount: dd if=/dev/zero of=/media/<user>/<drive>/test.out bs=1M count=1000
<user> is the user name and <drive> the name where drive was mounted by nautilus as per the label.

3) "Eject" the drive from nautilus context menu after dd command above returns.

4) Eject returns shortly. However drive is still writing as seen by the blinking, or disk throughput in gkrellm, iostat. Also note that "mount" still shows the drive as mounted though "df" and other tools don't.

I have confirmed the same issue exists on all similar file managers namely nemo and caja. Haven't filed bugs against those yet.

For comparison, the gnome-disks system utility correctly unmounts, ejects and stops the drive (last one even powers down the drive so the LED on drive is switched off which one would expect nautilus "Eject" to also do) so it doesn't seem to be a bug in udisks2.

[sumedh (sumwale)]

Workaround: apart from workarounds involving "sync" etc from command-line or always using command-line tools, the only way I have found to make all the GUI tools behave properly with "Eject" is to add the "sync" mount option.

While an explicit fstab entry can do it, the more user friendly way is to open the gnome-disks utility, click on gears "More action" icon after the drive has been inserted, then "Edit Mount Options", then unselect automatic mount option in the new window, then add "sync" separated by a comma after the list of options that are shown in the fourth text box.

One can select "/dev/sdb1" or whatever be the case in "Identy As" select box instead of the specific USB so that this works for all USB drives, but then it will not work for drives that are on other locations like /dev/sdc1. Perhaps one can repeat the same for all such possibiities, but one cannot be sure to cover all cases or cases where /dev/sdb1 is not a USB flash drive. This could potentially use the more complex matching criteria which I was a bit familiar with in the udev rules in older releases, but I don't know if that still works with the new infrastructure (perhaps need to look through udisks docs).

Either way adding "sync" is just a workaround and not the solution to this problem since adding that potentially loses on a lot of overall performance benefits of kernel filesystem cache. Proper solution will be to correctly unmount the drive (e.g. like what "udisks --unmount /dev/sdb1" does) and then eject. Not sure how the software manages to eject without even unmounting in the first place.

[sumedh (sumwale)]

FYI, using FAT32 as filesystem does not cause the issue. As I reported, mounting FAT32 add the "flush" mount option which probably avoids the above issue. I have updated the bug title accordingly. I suspect that adding the same manually instead of "sync" as suggested in previous comment as workaround will work better since "sync" will write to disk too often compared to flush. With mentioned "dd" I got average of 1.6MB/s with "sync" while >4MB/s with"flush" or otherwise without anything on my old USB flash drive. Haven't tried "flush" with NTFS yet but from mount man page still does not seem like that it will guarantee anything. Proper fix remains issuing "unmount" and waiting for it in nautilus, nemo, caja.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michele Giacomoli (michele-giacomoli)]

Hi, this afternoon I realized I have the same problem. I never noticed before because i often work with FAT32 or EXT filesystems, that aren't affected from this problem

[Michele Giacomoli (michele-giacomoli)]

@sumedh

Where did you see that mounting a FAT32 device adds the "flush" mount option?

In this moment I have both an ext4 and a ntfs external hdds mounted and running. reading from a "cat /etc/mtab" output none of them has the flush option, but I'm pretty sure that with the ext4 hdd I don't have this problem. I noticed that the NTFS device has "fuseblk" as mount type. Maybe the "problem" is inside fuse. Here is my cat /etc/mtab portion:

 /dev/sdd1 /media/michele/47D7D336684BC661 fuseblk rw,nosuid,nodev,allow_other,default_permissions,blksize=4096 0 0
/dev/sdc /media/michele/Backup_Server ext4 rw,nosuid,nodev,uhelper=udisks2 0 0

[sumedh (sumwale)]

When I plug in a FAT32 formatted pen drive, mount shows:

/dev/sdc1 /media/sumedh/PenDrive vfat rw,nosuid,nodev,uid=1000,gid=100,shortname=mixed,dmask=0077,utf8=1,showexec,flush,uhelper=udisks2 0 0

The man page for mount says this about "flush":

       flush If set, the filesystem will try to flush to disk more early than normal. Not set by default.

This tells me that the problem will be much less visible on FAT but there is likely to be a window of time where it can still happen.

[Seb Bonnard (sebma)]

Hi,

I have the same pb. in Ubuntu 17.10.

Where is the rule (udev ?) that automatically mounts my USB pendrive so that I can add "flush" or "sync" to it ?

[Seb Bonnard (sebma)]

I have the same pb. on Ubuntu 17.10 with Xfce + thunar file manager, so I believe this bug is maybe not linked to nautilus but to fuseblk

[Seb Bonnard (sebma)]

I see this process :

$ ps -fp $(\pgrep ntfs)
UID PID PPID C STIME TTY TIME CMD
root 25693 1 0 16:04 ? 00:00:00 /sbin/mount.ntfs /dev/sdb1 /media/mansfeld/Win7_USB_Installer -o rw,nodev,nosuid,uid=1000,gid=1000,uhelper=udisks2
$

[Gleb (glebsa)]

I encountered several times with this bug, with data loss on the flash drive

[MasterCATZ (mastercatz)]

Ubuntu 22.04 Kernel 5.15's and this issue still exists
I really do wish someone would do something about this

I have been waiting hours for a ~100meg file to finish writing to a NTFS partition
and yep its almost midnight
its a USB3 drive that I can read 400mbs from ...

Every 2.0s: grep -e Dirty: -e Writeback: /proc... aio: Sun Dec 12 23:25:38 2021

Dirty: 4820 kB
Writeback: 4 kB

what gets me is I can copy a 3 gig file , and the slow down will start after 1 Gb when dirtybit limit is reached then it just trickles along at kbs

this does not happen with fat32 , but I need NTFS for mounting Bootable ISO's
I really should have just hoped on a windows PC and copied the files ..