"Operation not permitted" from chmod when trying to create branch on ntfs-3g or fat filesystem

Bug #190725 reported by David Jaša
44
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
Medium
Unassigned
Breezy
Fix Released
Medium
Unassigned

Bug Description

Bazaar can't init repository on ntfs-3g (fuse) filesystem:

david@cihla:/mnt/data/skola$ bzr init
bzr: ERROR: Transport error: [Errno 1] Operation not permitted: '/mnt/data/skola/.bzr' [Errno 1] Operation not permitted: '/mnt/data/skola/.bzr'
david@cihla:/mnt/data/skola$ bzr init
bzr: ERROR: File exists: u'/mnt/data/skola/.bzr': [Errno 17] File exists: '/mnt/data/skola/.bzr'
david@cihla:/mnt/data/skola$ bzr version
Bazaar (bzr) 1.0.0
  Python interpreter: /usr/bin/python 2.5.2.alpha.0
  Python standard library: /usr/lib/python2.5
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib
  Bazaar configuration: /home/david/.bazaar
  Bazaar log file: /home/david/.bzr.log

Revision history for this message
David Jaša (dejv) wrote :
Revision history for this message
Aaron Bentley (abentley) wrote : Re: [Bug 190725] [NEW] Bzr can't init branch on ntfs-3g filesystem

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Jaša wrote:
> Public bug reported:
>
> Bazaar can't init repository on ntfs-3g (fuse) filesystem:
>
> david@cihla:/mnt/data/skola$ bzr init
> bzr: ERROR: Transport error: [Errno 1] Operation not permitted: '/mnt/data/skola/.bzr' [Errno 1] Operation not permitted: '/mnt/data/skola/.bzr'
> david@cihla:/mnt/data/skola$ bzr init

The root cause is that ntfs-3g gives "Operation not permitted" for chmod.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHrzdL0F+nu1YWqI0RAhLEAJ9HW3EIQkib+JgpHZtspdwqCJDAHQCfd1FC
PC3JfJvnoPLtpeF2tB/i30U=
=7T15
-----END PGP SIGNATURE-----

Revision history for this message
David Jaša (dejv) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Aaron Bentley napsal(a):
|
| The root cause is that ntfs-3g gives "Operation not permitted" for
| chmod.
|
| Aaron

Does bzr's design allow to recover from not being able to chmod
files/directories?

David

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHr1sI5ckLLQrmzAYRAsUYAJ49xC2ZmvBTsnB2wMcgTYil016h+wCfVnIB
YRBvfUx4eRxt3QyGZ/CFg0Y=
=XDFA
-----END PGP SIGNATURE-----

Revision history for this message
Robert Collins (lifeless) wrote :

On Sun, 2008-02-10 at 20:14 +0000, David Jaša wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Aaron Bentley napsal(a):
> |
> | The root cause is that ntfs-3g gives "Operation not permitted" for
> | chmod.
> |
> | Aaron
>
> Does bzr's design allow to recover from not being able to chmod
> files/directories?

Yes, of course. But generally a chmod failures means 'no permissions
here' so we have been treating it as a fatal error.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Revision history for this message
David Jaša (dejv) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins napsal(a):
| Yes, of course. But generally a chmod failures means 'no permissions
| here' so we have been treating it as a fatal error.
|
| -Rob

It would be nice if bzr had an option to ignore chmod failure on some
branch then. Managing documents with bzr on shared filesystem on
dual-boot machine is sensible scenario, isn't it?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHr5Lg5ckLLQrmzAYRAvU1AJ4tSkEAeLhjTuMFzLW8w9e29090KwCePC51
fdgfEzsyWOiyXF2bILJMN1k=
=ARKp
-----END PGP SIGNATURE-----

Revision history for this message
Mander (arwintcher) wrote : Re: Bzr can't init branch on ntfs-3g filesystem

I just wanted to chime in and say that's exactly what my setup is. For complicated reasons I used the Wubi installer on my Vista machine, so at the moment everything is ntfs. I'm new to both Linux and version control but I want to use it to manage my PhD work. Is there a workaround for this?

Revision history for this message
Misha Dorman (mishad-work) wrote :

I too would like to be able use bzr to manage repos and branches and access them from both "sides" of my dual-boot setup.

[Hg/Mercurial has the same problem, btw.]

What is the cleanest way to fix this? A simple flag (ideally remembered in the branch) to ignore such errors and continue?

A variant of transports/local.py that handles the exceptions differently, which can be specified via a URI scheme?

Why is failure-to-chmod treated as a fatal error? Could it be made non-fatal (warn-and-continue) in all cases, or would that introduce data loss/security problem risks?

Thinking more laterally:

Is there an alternative way of accessing NTFS shares from Linux (rather than ntfs-3g) which allows writing and supports chmod?

Do any of the ext2/3 IFS solutions for Windows provide read-write support such that they could be used (mounting an "alien" ext2 fs via an IFS in Windows, and using bzr.exe against the IFS)?

Revision history for this message
John A Meinel (jameinel) wrote :

So... I'm a bit curious if you couldn't mount with the right options and current umask so that the chmod isn't necessary, or is, at least, a no-op.
That would mean that the file mask should be exactly the directory mask, only without the executable bit (0777, 0666, umask 0000 for example, or 0775 0664 and a umask of 0002).

I certainly see this code in "local.py":
st = os.fstat(fd)
if mode is not None and mode != S_IMODE(st.st_mode):
    # Because of umask, we may still need to chmod the file.
    # But in the general case, we won't have to
    os.chmod(abspath, mode)

Alternatively, bzrdir._dir_mode and _file_mode set the modes that we want to achieve, and they can be set to "None" to indicate we don't want to do any chmodding. At the moment, this is triggered when "transport.stat('directory')" raises TransportNotPossible.

So this simple plugin could do the trick:

from bzrlib import transport, errors
from bzrlib.transport import local

class AltLocal(local.LocalTransport):

  def stat(self, relpath):
    raise errors.TransportNotPossible()

transport.register_transport('file:///', AltLocal)

Put the above code into a file like ~/.bazaar/plugins/no_local_stat.py

And see if ntfs-3g starts working for you. There are certainly other things you could do, like test "self.base" for being inside the mount point, etc.

Changed in bzr:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Misha Dorman (mishad-work) wrote :

John,

Thanks for looking at this.

I tried the plugin idea, but it didn't work. (Very nice idea though -- its for this sort of thing that I like bzr!) The plugin was found -- if I insert a syntax error I get a corresponding error -- but it didn't seem to have any effect.

Based on suggestions elsewhere, I tried mounting with the ntfs-3g -o silent option. This works (it causes chmod/chown to suppress their own errors) BUT only if _none_ of the uid, gid or umask options are specified. [Apparently the problem is that ntfs-3g is OK but FUSE raises the error; see http://forum.ntfs-3g.org/viewtopic.php?p=896&sid=c20dc858ef417d7c223e95603e216a0a]. Unfortunately, I'm using gid to allow non-root users to have access to the mounted drive.

I also tried the "make the fmask/dmask/umask match" idea: setting dmask=770, umask=660, umask=007 (I want no access except to root and members of the disk-access group). Unfortunately that also didn't work, though I'm not sure exactly why.

I feel so close, but not quite there .... :-(

Revision history for this message
Savvas Radevic (medigeek) wrote :

Note: this bug might be connected to bug #240429
https://bugs.launchpad.net/bzr/+bug/240429

(As Misha commented: "[Hg/Mercurial has the same problem, btw.]")

Revision history for this message
Savvas Radevic (medigeek) wrote :
Revision history for this message
Savvas Radevic (medigeek) wrote :

Misha have you tried the last comment?
http://forum.ntfs-3g.org/viewtopic.php?p=3341#p3341

Quote:
A little late, but nonetheless...
I was able to overcome problems with preserving times:
Code:
cp: preserving times for 'foo': Operation not permitted

by adding 'uid=' option in /etc/fstab, like this:

Code:
# /dev/sda3
UUID=E486D32886D2FA2A /media/files ntfs-3g defaults,umask=007,gid=46,uid=1000 0 1

Martin Pool (mbp)
Changed in bzr:
status: Triaged → Confirmed
Revision history for this message
Vernon Cole (vernondcole) wrote :

This is also related to bug 248333 where the X bit gets fouled up on an NTSF volume.
Is there a way that bzr can detect that it is running on an NTSF volume and take special steps?

Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 190725] Re: Bzr can't init branch on ntfs-3g filesystem

On 22 April 2010 02:42, Vernon Cole <email address hidden> wrote:
> This is also related to bug 248333 where the X bit gets fouled up on an NTSF volume.
> Is there a way that bzr can detect that it is running on an NTSF volume and take special steps?

Yes, it should be possible to detect this, by any of

1- calling fstatfs
2- a per-location user configuration option
3- testing whether the x-bit can be set and read back

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
nairbv (nairbv) wrote : Re: Bzr can't init branch on ntfs-3g filesystem

This might be related to #199654.

Jelmer Vernooij (jelmer)
tags: added: ntfs
Jelmer Vernooij (jelmer)
tags: added: filesystem
Martin Pool (mbp)
summary: - Bzr can't init branch on ntfs-3g filesystem
+ "Operation not permitted" from chmod when trying to create branch on
+ ntfs-3g filesystem
Martin Pool (mbp)
summary: "Operation not permitted" from chmod when trying to create branch on
- ntfs-3g filesystem
+ ntfs-3g or fat filesystem
Changed in bzr:
assignee: nobody → Martin Pool (mbp)
status: Confirmed → In Progress
Revision history for this message
Martin Pool (mbp) wrote :

I've just been testing this on a linux vfat mount, and these operations work:
 - checkout --lightweight
 - branch
 - init
 - st on the resulting branch (though it does demonstrate bug 248333)

Revision history for this message
Martin Pool (mbp) wrote :

I can't reproduce this on ntfs either, but misha's comments in #7 are suggestive - perhaps it's something about gvfs, perhaps something about having the device not owned by the user in question.

Revision history for this message
Martin Pool (mbp) wrote :

I also can't reproduce this on a gvfs-mounted ntfs filesystem, on Ubuntu Oneiric: branching onto there from bzr just works. So, perhaps something has changed in the filesystem implementation so that chmod no longer fails?

If anyone still can reproduce this, please let us know:
 * the traceback from ~/.bzr.log
 * how this filesystem is mounted
 * the output of 'mount'
 * what OS and version you're using

Changed in bzr:
assignee: Martin Pool (mbp) → nobody
status: In Progress → Confirmed
Revision history for this message
Sebastian Korczyk (sebastian-korczyk) wrote :
Download full text (10.4 KiB)

I think I can reproduce similiar thing, I've tried to import project from launchpad, using eclipse plugin, hovewer it was failing with unexpected error, I've checked and same happens when using bzr from command line. When running command below on ntfs system it is failing, when run it on ext4 filesystem, everything was fine:

When running: bzr branch lp:civ4stratoverlay civ.white

after entering passwd for RSA key I'm receiving such error:
bzr: ERROR: [Errno 1] Operacja niedozwolona: '/media/win/workspace/civ3/civ.white/.bzr/checkout/limbo/new-2/Config/init.xml'

Where "Operacja niedozwolona" means - illegal operation.

Below bzr.log (first when command was succesfull on Linux filesystem, and later when it has failed on)

System Debian Wheezy: 3.0.0-1-686-pae

fstab: /dev/sda4 /media/win ntfs defaults,umask=000,gid=1000,noatime 0 0

Mount output for this partition:
/dev/sda4 on /media/win type fuseblk (rw,nosuid,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096)

nie 2012-08-19 00:13:19 +0200
0.048 bazaar version: 2.6.0dev2
0.048 bzr arguments: [u'branch', u'lp:civ4stratoverlay', u'civ.white']
0.052 looking for plugins in /home/logan/.bazaar/plugins
0.052 looking for plugins in /usr/lib/python2.7/dist-packages/bzrlib/plugins
0.065 encoding stdout as sys.stdout encoding 'UTF-8'
0.124 ssh implementation is OpenSSH
10.176 creating repository in file:///home/logan/test/civ2/civ.white/.bzr/.
10.346 Using fetch logic to copy between RemoteRepository(bzr+ssh://bazaar.launchpad.net/%2Bbranch/civ4stratoverlay/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar pack repository format 1 (needs bzr 0.92)\n')) and KnitPackRepository('file:///home/logan/test/civ2/civ.white/.bzr/repository/')(RepositoryFormatKnitPack1())
10.354 fetching: <PendingAncestryResult heads:frozenset(['<email address hidden>']) repo:RemoteRepository(bzr+ssh://bazaar.launchpad.net/%2Bbranch/civ4stratoverlay/.bzr/)>
16.536 creating branch <bzrlib.branch.BzrBranchFormat6 object at 0x89857ec> in file:///home/logan/test/civ2/civ.white/
16.836 created new branch BzrBranch6(file:///home/logan/test/civ2/civ.white/)
16.842 trying to create missing lock '/home/logan/test/civ2/civ.white/.bzr/checkout/dirstate'
16.877 opening working tree '/home/logan/test/civ2/civ.white'
[26197] 2012-08-19 00:13:36.417 INFO: Branched 36 revisions.
17.002 Transferred: 353kB (20.9kB/s r:351kB w:1kB)
17.002 return code 0

nie 2012-08-19 00:15:36 +0200
0.048 bazaar version: 2.6.0dev2
0.048 bzr arguments: [u'branch', u'lp:civ4stratoverlay', u'civ.white']
0.052 looking for plugins in /home/logan/.bazaar/plugins
0.052 looking for plugins in /usr/lib/python2.7/dist-packages/bzrlib/plugins
0.065 encoding stdout as sys.stdout encoding 'UTF-8'
0.124 ssh implementation is OpenSSH
9.957 ignore error on chmod of u'/media/win/workspace/civ3/civ.white/.bzr': OSError(1, 'Operacja niedozwolona')
9.957 ignore error on chmod of u'/media/win/workspace/civ3/civ.white/.bzr/branch-lock': OSError(1, 'Operacja niedozwolona')
9.962 ignore error on chmod of u'/media/win/workspace/civ3/civ.white/.bzr/README.26213.azorek.sb...

Revision history for this message
Martin Packman (gz) wrote :

Sebastian, so your eventual error there is from utime rather than chmod. While setting the correct mtime on files isn't strictly required, bazaar does expect to be able to do that when creating the working tree.

So, I expect the following would not fail:

    bzr branch --no-tree lp:civ4stratoverlay civ.white

See bug 124795 on ntfs-3g for some discussion about this and pointers on how to fix your mount options.

Revision history for this message
Sebastian Korczyk (sebastian-korczyk) wrote :

Martin, I can confirm that when running command you had provided it has passed on ntfs filesystem. So I will go to discusion on bug 124795 that maybe message in such case should be more clear.

Revision history for this message
Peter (fpeter) wrote :

bzr init-repo sftp://xxxxxx/bzr/foo
bzr: ERROR: [Errno 13] Permission denied

the target was fat32 formatted.
a better error message needed.

Jelmer Vernooij (jelmer)
tags: added: check-for-breezy
Jelmer Vernooij (jelmer)
Changed in brz:
status: New → Triaged
importance: Undecided → Medium
Jelmer Vernooij (jelmer)
tags: removed: check-for-breezy
Changed in brz:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers