rfc: don't treat missing files as deleted

[Martin Pool]

Robert points out that it's inconsistent that bzr must be explicitly
informed of adds and renames, but it will heuristically detect deletion.
In particular if you rename a file outside of bzr then it will be
detected as deletion.

The alternatives are:

 - simply treat missing-but-not-removed files as an error, as in cvs

 - detect all adds/renames/deletes, using implicit ids (but unless implicit
   names are used for everything this does not really solve the problem)

Treating missing files as deleted is also a bit of a source of bugs, and
may continue to be so. It means that the working inventory does not
really describe the shape of the tree. (Perhaps it could be fixed
systematically by better factoring of WorkingTree.)

This is a cute and sometimes useful feature but perhaps the
inconsistency is a problem.

We could change the behaviour to this:

 * status reports missing files as missing

 * all other commands that try to access that file raise an error

We can have a command that marks all missing files as deleted, so the
current behaviour can still be available, but just not the default, and
implemented in one place, not several.

(What is the svn behaviour?)

[Martin Pool (mbp)]

Needs discussion, really.

[Martin Pool (mbp)]

I think we should change to be more like the svn behaviour: if a file is missing, operations on it give an error and the user must explicitly remove it.

[Elliott Hughes (enh)]

i agree. i'm the author of a GUI front-end for various revision control systems, and the current bzr(1) behavior makes my life more difficult. my plan (which i haven't yet got round to) is to explicitly ask the user how to interpret a missing file. i think that front ends are the right place to address this, rather than in the core tools.

if you're concerned about changing existing behavior, a flag that would let front end authors like me explicitly ask for the Subversion-like behavior would be fine, i think. anyone using the underlying tools too needs to understand potential interactions (like people using stdio and C++'s iostreams in the same program), even if in an ideal world there would be such conflicts.

anyway, i know this isn't a democracy, but having Subversion-like behavior would get my vote if it were ;-)

i don't think i have an opinion on whether "bzr remove" should delete the working copy or not. i don't think it makes any real difference either way. i reserve the right to change my mind, though, just in case!

[luna (luna)]

I want to be able to do this :

$ bzr add myfile
$ bzr ci -m"myfile added"
$ cat myfile > mycopy
$ rm myfile
$ bzr mv myfile mycopy

or something similar
Now when I try to do it it says something not very intelligible about the file "myfile" not existing in the wotking tree.

[Martin Pool (mbp)]

I agree.

The core change is to rename only in the inventory, not in the working
tree.

I propose two complimentary ways to turn this on: firstly by explicitly
saying "bzr mv --inventory-only" (or something similar), or otherwise
automatically.

The automatic behaviour should be triggered for 'mv A B' if the working
directroy contains B but not A, and the inventory contains A but not B.
This will require some care when B is a directory.

--
Martin

[Nicholas Allen (nick-allen)]

That would be great! The automatic recognition of OS mv commands would be the coolest feature in a VCS. The reason why this is so useful is because you would not need integrated support in IDEs. eg In Eclipse if I rename a Java class it renames the file too and this would then work without Eclipse having to know anything about bzr. Also it would make bzr a great backup system for non-technically inclined people. They could just rename their files from Explorer without having to jump to the command line and "bzr mv A B".

[Marius Kruger (amanica)]

+100
it would be VERY usefull to be able to inform bzr that
'removed file_a' and 'added file_b' is actually a
'rename file_a to file_b'
plus possibly 'updated file_b' (when you renamed and changed it in one go)

[Aaron Bentley (abentley)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marius Kruger wrote:
> +100
> it would be VERY usefull to be able to inform bzr that
> 'removed file_a' and 'added file_b' is actually a
> 'rename file_a to file_b'
> plus possibly 'updated file_b' (when you renamed and changed it in one go)

OS-level renames are not a reliable indicator of user intent. Many
applications rename files when modifying or deleting them.

In my opinion, the best place to capture intent is in the UI, e.g. the
shell (see 'bzr shell' in bzrtools) the IDE, or the file manager.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEvlvl0F+nu1YWqI0RApD+AJoCMjP7t+CTUKbgxesXnyPd+Le+7wCfaIlu
sDvfmYMQzjUPFGWA8LyIe28=
=phhs
-----END PGP SIGNATURE-----

[Jelmer Vernooij (jelmer)]

How about storing file ids in extended attributes? That way you can always find out what the original file was. It won't work on all systems, but it's better than not being able to find out at all.

[John A Meinel (jameinel)]

It doesn't help the problem Aaron is commenting on.

I can setup vim so that when I save the file, it renames the current file to a backup file, and saves a new file in its place.

(I used to use this, coupled with deleting the backup file, so that vim would break hardlinks when using hardlinked Arch working trees).

Anyway, if you don't delete the file, suddenly your filename has changed from 'foo.py' to 'foo.py~', and you have a new file named 'foo.py'.

Lots of applications use rename as a way to atomic updates to files (create a temp file, rename it over top the existing file).

I don't know many other ways that rename puts things out of the way, but backup files are an obvious use case.

So Aaron's basic comment is that the function call 'rename("a", "b")' at an os level does not mean you intend to move a to b. Running the command line 'mv' probably does, as does renaming the file in your IDE, etc. But not at the OS level.

I think bzr should support running 'bzr mv a b' after you already ran 'mv a b'. That is completely fixable (rather than having to do the opposite 'mv b a; bzr mv a b'). Auto detecting renames is not.

[Aaron Bentley (abentley)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jelmer Vernooij wrote:
> How about storing file ids in extended attributes? That way you can
> always find out what the original file was. It won't work on all
> systems, but it's better than not being able to find out at all.

We could certainly try that. I have the impression that extended
attributes are rather fragile, though. If even one of the programs you
regularly use doesn't respect extended attributes, then the whole thing
doesn't work.

Aaron

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEvmmG0F+nu1YWqI0RAnJAAKCBwfUaGa7ArS0h0UNNQLp+KW3qXgCfT4mi
rz67pouOBB5sX3j3hksnudY=
=wYjo
-----END PGP SIGNATURE-----

[Jelmer Vernooij (jelmer)]

On Wed, Jul 19, 2006 at 05:19:02PM -0000, Aaron Bentley wrote:
> Jelmer Vernooij wrote:
> > How about storing file ids in extended attributes? That way you can
> > always find out what the original file was. It won't work on all
> > systems, but it's better than not being able to find out at all.
> We could certainly try that. I have the impression that extended
> attributes are rather fragile, though. If even one of the programs you
> regularly use doesn't respect extended attributes, then the whole thing
> doesn't work.
It'll only break if one of those programs does weird things moving
files around.

Extended attributes are getting more important, at least in the
Windows world so people will be complaining about applications that
break ea's.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden> - http://jelmer.vernstok.nl/

[Aaron Bentley (abentley)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jelmer Vernooij wrote:
> On Wed, Jul 19, 2006 at 05:19:02PM -0000, Aaron Bentley wrote:

>>We could certainly try that. I have the impression that extended
>>attributes are rather fragile, though. If even one of the programs you
>>regularly use doesn't respect extended attributes, then the whole thing
>>doesn't work.
>
> It'll only break if one of those programs does weird things moving
> files around.

But everyone does the create/rename update dance: Microsoft Word, Vim,
bzr, you name it. Are you saying all of these apps take care to
preserve extended attributes?

> Extended attributes are getting more important, at least in the
> Windows world so people will be complaining about applications that
> break ea's.

Well, bzr breaks ea's and so far no one's complained...

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEvn9C0F+nu1YWqI0RAnSaAJ4lT61T7UWom0Fqqn4xiP4oYaLMbwCghPKU
Aa2wfgRTSGKi69ytVjW9GVE=
=YlDL
-----END PGP SIGNATURE-----

[Wouter van Heyst (larstiq)]

On Wed, Jul 19, 2006 at 05:06:13PM -0000, John A Meinel wrote:

...

> I think bzr should support running 'bzr mv a b' after you already ran
> 'mv a b'. That is completely fixable (rather than having to do the
> opposite 'mv b a; bzr mv a b'). Auto detecting renames is not.

I think this is what Marius was asking for, are there problems doing
that?

Wouter van Heyst

[Michael Ellerman (michael-ellerman)]

It should be an easy patch to change bzr mv to cope with the source and destination already being moved. It just probably requires a little thought as to whether it should spit out a warning, or require a special flag etc. It'd be a good first patch for some keen bzr newbie.

I've got the beginings of an auto-rename plugin at:

http://michael.ellerman.id.au/bzr/plugins/auto-rename

It tries to match files based on their SHA1, and if that fails does a fuzzy match based on the number of lines changed between the two versions. It needs a bit of polish, but it's 50% there.

[Marius Kruger (amanica)]

That is exactly what I'm asking for.
I also use eclipse, and when I change a class name, the filename is also
changed,
and sometimes I do other updates in it for the same commit.
Thus the file is already moved.

I want to be able to do something like the following:
bzr rename --already-renamed file_a file_b
bzr mv --already-moved file_a file_b

On 7/19/06, Wouter van Heyst <email address hidden> wrote:
>
> On Wed, Jul 19, 2006 at 05:06:13PM -0000, John A Meinel wrote:
>
> ...
>
> > I think bzr should support running 'bzr mv a b' after you already ran
> > 'mv a b'. That is completely fixable (rather than having to do the
> > opposite 'mv b a; bzr mv a b'). Auto detecting renames is not.
>
> I think this is what Marius was asking for, are there problems doing
> that?
>
> Wouter van Heyst
>
> --
> rfc: don't treat missing files as deleted
> https://launchpad.net/bugs/5158
>

--
--

May the source be with you.

[John A Meinel (jameinel)]

bumping to 0.11
Will probably happen with the dirstate changes as Robert refines that.

[John A Meinel (jameinel)]

still pending for 0.12

[John A Meinel (jameinel)]

still waiting on dirstate

[Nicholas Allen (nick-allen)]

I just tried out the auto-rename branch. This is really cool and works exactly how I would like. This should be a standard bzr command I think and in the trunk. Any chance of that?

[Robert Collins (lifeless)]

If someone wanted to update the plugin and ensure it has good test coverage we could at a minimum start shipping it in default, or merge it to core.

[Robert Collins (lifeless)]

This bug has grown out of all recognition; so I'm just fixing the original and we can discuss os volume journal parsing later :)

[Martin Pool (mbp)]

bug 131792 is one particular case where automatically considering missing files deleted causes confusion with rename.

[Robert Collins (lifeless)]

I'm not moving this forward anymore; my branch did address it and probably will still merge, but friction >> reward.