can't easily use unison-gtk to sync a local folder to my VFAT USB drive

[Subharo Bhikkhu]

My simple way of backing up my important personal data is to have a folder in my home directory called "sync". I file all data under there that ought to be periodically backed up. I also have a USB drive, formatted with VFAT. I have a "sync" folder on that USB drive as well. I want to be able to use unison-gtk to syncronize these two folders called "sync". But when I plug in the USB drive, it gets automounted with the mount option "umask=077". So all files on the USB drive get seen and created with permissions looking like "rwx------". But my umask (that all users have by default) is "0022", thus all their newly created files will have permissions like "rw-r--r--".

When I try to do a sync in unison-gtk, it can sync in only one direction: from the USB drive to my home folder. And the files it syncs to my home folder all annoyingly have permissions like "rwx------". This leads to trouble like text files that are wrongly made "executable", and directories that you intend other users to be able to read cannot be, because they are not "executed" by any other user (even if one remembers to add group permission allows their reading of that folder).

Furthermore, one cannot sync files from the local folder to the USB drive, because the desired permissions unison-gtk would like to use (ie. "rw-r--r--" that most local files would have by default) can't be used on the USB drive, as the mount options insist that all newly created files on the USB drive have the permissions "rwx------" only. So you see errors in unison-gtk for each and every file it tries to copy onto the USB drive.

I have two ideas to address this:

1) In Gnome's System menu -> Preferences -> Removable Drives and Media -> "Storage" tab, please add an "Advanced" button under "Removable Storage", and allow the user to set a different umask to be used when USB drives get automounted, and in my case I would probably want to use something like "133", making all the files on the USB drive appropriately appear as "rw-r--r--" (and hopefully the directories on the USB drive can somehow still be navigatable, ie. "executable").

2) At the time of automounting the USB drive, perhaps the user's current umask (ie. as defined in .bash_profile) needs to be looked at, and a umask mount option for automounting needs to be dynamically figured out to nicely match this, such that unison-gtk will be happy.

[Andrea Garbarini (garba)]

you can't store unix file permission on a vfat filesystem

[Subharo Bhikkhu (subharo)]

The "mask" that sets the permissions when making vfat files and
folders appear like they are posix-compliant should be adjustable
somewhere, that's the heart of the issue. Right now the mask used
causes problems, because it blindly makes all vfat files appear as
having executable bit set.

On 8/15/07, Andrea Garbarini <email address hidden> wrote:
> you can't store unix file permission on a vfat filesystem
>
> --
> can't easily use unison-gtk to sync a local folder to my VFAT USB drive
> https://bugs.launchpad.net/bugs/118283
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Dustin Harriman

My Blog: http://tinyurl.com/3xsa9x
RSS Feed: http://tinyurl.com/2jq6fe
GnuPG Public Key: http://tinyurl.com/yvcdgh

"The serene and peaceful mind is the true
epitome of human achievement." -Ajahn Chah

[Frans Mäyrä (frans-mayra)]

I seem to suffer from this same bug -- it does not make much sense to use Unison, if it will only copy files to one direction while doing sync, what do you think?

[faif (faifcn)]

I had seam problem, when I try to backup my usb disk into my home linux machine and also synchronise with company windows machine. You can work around by following method. Find your profile name for example : in ~/.unison/default.prf
add the following line (parameters):

owner = false
perms = 0

It now works fine.

[Subharo Bhikkhu (subharo)]

Thanks for the tip, faif, but it didn't work for me. Even with your suggested options in defaults.prf, for each folder it tries to sync onto my FAT32 USB flash drive, I get an error message in unison-gtk's status area like this:

backgrounds
Failed to set permissions of file /media/disk/.#backgrounds.7d108fa1a22221d4a88fe3a9b0ce0dfc.unison.tmp/00610_peace_1280x1024.jpg to rw-r--r--: the permissions was set to rw------- instead

See my 3 screenshots. The first screenshot is what unison-gtk proposes to sync, which looks OK. The second screenshot shows the failure that occurs if I just press "Go", accepting those proper-looking proposed actions. The third screenshot shows the same failure if I press "Restart" to get to the step before, but then also opt to have the "props" also be synced from right to left, ie. onto the USB flash drive.

[faif (faifcn)]

Hi Dustin,
I have same problem now again, The
owner = false
perms = 0
doesn't work any more.

[Olivier Mehani (shtrom)]

Hi,

Same setup here: syncing an ext3 fs to a vfat USB disk on Gutsy.

Changing the PRF file actually solves the problem, but a more generic problem remains. Gnome-mount should mount the VFAT volumes with the "quiet" option by default, to avoid errors when trying to set Unix-like permissions.

[Olivier Mehani (shtrom)]

Filed a bug about `gnome-mount`: https://bugs.launchpad.net/ubuntu/+source/gnome-mount/+bug/178154.

[Maia Everett (linneris)]

Removed the "sync" tag so it isn't mistaken for a package sync request.

[muhalifsirin (alperense)]

The same problem here

These permission problems about writing to a partition, sharing a file in some discs...etc; they are all killing me, and this is what I hate most in ubuntu. It has to be much more simpler if they are claiming it is linux for human beings!

[Subharo Bhikkhu (subharo)]

I've come up with a simple, decent workaround to this problem that I'd like to share. It won't help everyone in all cases, but it's a lot better than nothing ;)

For months now I have been pondering how to easily transport, say, important configuration files (in ~) from one Ubuntu computer to another, using a VFAT/FAT32 USB flash drive. It's quite tricky. Let me illustrate:

I want to use VFAT on the flash drive so that all files are also easily accessible in Windows. But VFAT does not store the meticulous Unix-style permissions required for many configuration files. So for example, you get in trouble if you want to copy your ~/.gnupg files from one Ubuntu machine to the other (using a VFAT USB flash drive to transport the files), because the file permissions get mangled from being -rw------- to -rwx------. Similarly, any file having -rw-r--r-- gets mangled to being -rwx------, and any directory having drwxr-xr-x gets mangled to drwx------.

If you use ext2, ext3, or reiserfs as the filesystem on your USB flash drive (to save these permissions), then you can still get into "permission trouble" because you may not have the same UID on both Ubuntu machines. And accessing ext2, ext3, or reiserfs from Windows is very non-trivial at best, dodgy at worst.

The best solution for propagating any "permission-sensitive" files, like the ~/.gnupg config files in this example (to another Ubuntu machine) seems to be to store them in a tar file on the VFAT USB drive. The tar file does store the file permissions properly! Gnome's "File Roller" is very smooth and is well-integrated into Gnome/Nautilus (eg. from context menus): it's all point-and-click. Whee! And, when files get untarred, they take on whatever UID you have at the time, so the UID problem goes away too (i.e. UIDs possibly not being the same on two Ubuntu machines). Plus, a tar file is easy to work with from Windows using a free tool like 7zip.

This workaround still leaves the annoying problem that when you copy "regular" files (ie. not "permission-sensitive" files) from the VFAT Flash drive (ie. that weren't stored in a tar files), they will have the mangled -rwx------ permissions.

[ceg (ceg)]

You can make your fat filesystem permissions look like matching to your default umask.

Mount options:

for umask 002: dmask=002,fmask=113
for umask 022: dmask=022,fmask=133

(do not use the unspecific umask option)

As a workaround adjust your personal hal vfat mount options in the gconf-editor, but the defaults need to dealt with by ubuntu.

[ceg (ceg)]

More information on a corresponging debian bug, that has been closed by maintainers without resolving.
http://bugs.debian.org/344278

[Subharo Bhikkhu (subharo)]

ceg, your suggestions sounds perfect. I've verified your fix. Here is the procedure, for anyone else who wants to do this.

Within gconf-editor, I used the "Find" tool to search for "vfat" (searching also in all key names and key values).

This returned the key "/system/storage/default_options/vfat"

I deleted the umask key, then added the 2 keys you suggested as follows:
dmask=022
fmask=133

Now all directories/folders on my vfat USB flash drives and USB external hard drives are presented with the permissions:

drwxr-xr-x

...and all files on my vfat USB flash drives and USB external hard drives are presented with the permissions:

-rw-r--r--

Perfect! This matches the default umask that Ubuntu sets up by default for all users in /etc/profile

ceg, you da man!!

[Subharo Bhikkhu (subharo)]

I've also added this suggested fix to bug 178154 (because it's gnome-mount which is closer to the root of the problem than unison).

[Subharo Bhikkhu (subharo)]

With the dmask/fmask fix now applied in gdonf-editor, I've verified that unison-gtk will properly sync files and folders back and forth to a vfat drive. But the fix only works if all files and folders being synced have permissions like "-rw-r--r--" and "drwxr-xr-x", respectively. Plus, if there are any symlinks or other such funky unix-only file types, then the sync will fail ("Operation not permitted").

But this isn't so bad. This fix should cover the needs of 98% of all Ubuntu users, who just want to make backups of their "user files" (let's call them) such as documents, pictures, mp3's, etc, which all will have the default permissions of "-rw-r--r--". That is to say, most users do not use any special permissions on their "work" or personal files. They almost always just go along with the default umask set in /etc/profile. This is because they are almost always the only user on their computer, having no need to worry about group permissions, or sharing files (at the filesystem level) with other Ubuntu users. Also, most Ubuntu users probably do not know what a symlink is, or if they did, they would not have reason to create any. (What's the point when Tracker finds these "user files" so easily now?)

This fix will NOT cover that last 2% of Ubuntu users who might want to manually back up "system files" (let's call them) such as dotfiles, symlinks, and anything else not under /home. In these far-less-common circumstances, you might want to use my suggested workaround above (ie. by storing any files and folders with funky permissions and other "UNIX-ness" inside .tar files, using "File Roller"/"Archive Manager"). Or even better yet, by using a more proper system-wide backup solution, like say BackupPC, when it comes to backing up these "system files."

So I'm not sure if this bug should be closed, since this dmask/fmask fix doesn't "just work" in all circumstances. The end user still needs to be aware if there are any special permissions or "unix-only" file types (like symlinks) that they will try to include amongst the files they want to sync, and if so, then don't use unison-gtk.

Subharo Samanera

My Blog: http://tinyurl.com/3xsa9x
RSS Feed: http://tinyurl.com/2jq6fe
GnuPG Public Key: http://tinyurl.com/3cc84m

"The serene and peaceful mind is the true
epitome of human achievement." -Ajahn Chah

[Olivier Mehani (shtrom)]

“But the fix only works if all files and folders being synced have permissions like "-rw-r--r--" and "drwxr-xr-x", respectively.”

Have you tried adding the {{{quiet}}} option as well? That may be even better (don't have my test Unison test setup to try it here, though).

[Alex Mauer (hawke)]

The "/system/storage/default_options/vfat" doesn’t seem to exist in Ubuntu 9.10, and presumably wouldn’t work for NTFS. Is there a newer place where this can be corrected?

[Subharo Bhikkhu (subharo)]

I've opened Bug 592748 to try to get this fixed.

[Clint Byrum (clint-fewbar)]

Affects multiple people, marking Confirmed. Workarounds exist and this really isn't the perfect use case for unison, so marking Medium priority.