git: unusual_modes drops when a file having unusual mode stays continuing revisions.

[methane]

bzr-git stores unusual_mode into bzr's revision when inventory delta contains unusual modes.
But when rev1 have a file with unusual mode and rev2 have same file, the file is not in inventory delta.
So recreated git tree of rev2 missing unusual mode.

[methane (songofacandy)]

I don't know well about how bzr stores revisions.
So I tried to fix by store unusual mode even if rev2 doesn't have delta.

[Jelmer Vernooij (jelmer)]

I'm not convinced this is actually an issue. Can you please provide a
test case for this?

[methane (songofacandy)]

I've dpush with bzr-git in some months ago and such a strange revision created.
In http://git.sourceforge.jp/view?p=msgpack/msgpack.git, a 'msgpack' link has
lrwxrwxrwx mode:

http://git.sourceforge.jp/view?p=msgpack/msgpack.git;a=tree;f=c;h=1adac033442fa762cc4b5014c9197ef7326cd311;hb=4b2c15fde6f6f66299005dac4d423d452379ceb1

git fsck --all doesn't report an error.
bzr-git reports sha1 mismatch when dpushing to the repository.
My patch fixes sha1.

[methane (songofacandy)]

I've researched about git, symlink and bzr-git those days.
Something that I found is:

1) git doesn't create executable symlink normally.
"git update-index --chmod=+x" to symlink reports error.
To create executable symlink, I have to do like:
$git ls-tree treeid-without-executable-symlink > tree
$vim tree # change mode from 120000 to 120777
$git mktree <tree
new-tree-id
$git commit-tree new-tree-id -p parent-commit-id

2) bzr-git trunk makes executable symlink.
Maybe after http://bazaar.launchpad.net/~bzr/bzr-git/trunk/revision/703.
I hate my patch creates dirty symlink on many projects.

$ git init
$ mkdir foo
$ cd foo
$ ln -s . foo
$ cd ..
$ bzr add foo
$ bzr commit
$ git ls-tree last-tree-id
shows 120111 mode for symlink.

3) git perfectly works with symlink that have 120777 or 120111 mode.

4) git repository of messagepack project contains 120777 symlink.
git url: git://git.sourceforge.jp/gitroot/msgpack/msgpack.git
browse url: http://git.sourceforge.jp/view?p=msgpack/msgpack.git;a=summary

I've introduced 120777 symlink to messagepack repository on "16a208243b8ef6eadd626876d5224c81e0bda74e" commit.
I've used old bzr-git to make the commit, IIRC.

5) bzr-git fails to dpush to messagepack repo.

$ git clone git://git.sourceforge.jp/gitroot/msgpack/msgpack.git
$ git daemon --export-all --enable=upload-pack --enable=receive-pack --baes-path=.
$ bzr branch git://localhost/msgpack/ mp
$ cd mp
$ rm .bzr/repository/git.db
$ bzr commit --unchanged -m "foo"
$ bzr dpush git://localhost/msgpack/
bzr: ERROR: exceptions.AssertionError: recreated git commit had different sha1: expected c8765c4a8ee44afd0fe8bf05f2a9cbb0638d9ecc, got 15be17f58c0ad2c1f55ea2b357476609d81705e0

"c8765c4a8ee44afd0fe8bf05f2a9cbb0638d9ecc" is the commit with 120777 symlink.
bzr-git creates wrong sha1 because the revision in bzr doesn't contain "file-mode" property.
unusualmode-for-sameinv.patch fixes the problem.

[methane (songofacandy)]

I've created a minimum testcase.

To reproduce the bug is:
1) create test git repository (attached gzip contains this)

inada-n@koala:~/test$ mkdir foo
inada-n@koala:~/test$ cd foo/
/home/inada-n/test/foo
inada-n@koala:~/test/foo$ git init
Initialized empty Git repository in /home/inada-n/test/foo/.git/
inada-n@koala:~/test/foo$ ln -s . foo
inada-n@koala:~/test/foo$ git add foo
inada-n@koala:~/test/foo$ git commit -m "add foo"
[master (root-commit) ca3de8b] add foo
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 120000 foo
inada-n@koala:~/test/foo$ cat .git/refs/heads/master
ca3de8bc2a701da35d7ff26d2a4b2dfeeec9e048
inada-n@koala:~/test/foo$ git ls-tree ca3de8bc2a701da35d7ff26d2a4b2dfeeec9e048
120000 blob 945c9b46d684f08ec84cb316e1dc0061e361f794 foo
inada-n@koala:~/test/foo$ git ls-tree ca3de8bc2a701da35d7ff26d2a4b2dfeeec9e048 > tree
inada-n@koala:~/test/foo$ vim tree #edit tree file
inada-n@koala:~/test/foo$ git mktree < tree
c7b0af5a3a158d8eeb88d3d333ad076895dd54d0
inada-n@koala:~/test/foo$ git ls-tree c7b0af5a3a158d8eeb88d3d333ad076895dd54d0
120777 blob 945c9b46d684f08ec84cb316e1dc0061e361f794 foo
inada-n@koala:~/test/foo$ echo "change mode" | git commit-tree c7b0af5a3a158d8eeb88d3d333ad076895dd54d0 -p ca3de8bc2a701da35d7ff26d2a4b2dfeeec9e048
912fbbb93aa51215867875fcff126885718ffcdf
inada-n@koala:~/test/foo$ echo "change mode take2" | git commit-tree c7b0af5a3a158d8eeb88d3d333ad076895dd54d0 -p 912fbbb93aa51215867875fcff126885718ffcdf
7b413dc8152fd1a35ce1552456af2ea884268c8f
inada-n@koala:~/test/foo$ echo 7b413dc8152fd1a35ce1552456af2ea884268c8f > .git/refs/heads/master
inada-n@koala:~/test/foo$ git log
commit 7b413dc8152fd1a35ce1552456af2ea884268c8f
Author: Naoki INADA <email address hidden>
Date: Tue Feb 2 09:28:42 2010 +0900

    change mode take2

commit 912fbbb93aa51215867875fcff126885718ffcdf
Author: Naoki INADA <email address hidden>
Date: Tue Feb 2 09:28:15 2010 +0900

    change mode

commit ca3de8bc2a701da35d7ff26d2a4b2dfeeec9e048
Author: Naoki INADA <email address hidden>
Date: Tue Feb 2 09:26:05 2010 +0900

    add foo

2)Make empty git repository and run git daemon.
$ cd ..
$ mkdir baz
$ cd baz
$ git init
$ cd ..
$ $ git daemon --export-all --enable=upload-pack --enable=receive-pack --base-path=$HOME/test

3) branch with bzr-git
inada-n@koala:~/test$ bzr branch git://localhost/foo/ bar
/ Counting objects: 6, done.
Branched 3 revision(s).
inada-n@koala:~/test$ cd bar
/home/inada-n/test/bar
inada-n@koala:~/test/bar$ rm .bzr/repository/git.db
inada-n@koala:~/test/bar$ bzr dpush git://localhost/baz/
bzr: ERROR: exceptions.AssertionError: recreated git commit had different sha1: expected 7b413dc8152fd1a35ce1552456af2ea884268c8f, got 368b0bb7ff8782ced333c386d874594614cb5913

[Jelmer Vernooij (jelmer)]

Can you format this as a test that can go into tests/ ?

If not, that's fine too - it will mean some more delay before your
change can land though, as I need the time to do it.

[methane (songofacandy)]

I failed to make an unusual mode with GitBranchBuilder, because git fast-import rejects unusual mode...
Could you make it with dulwich? Or can I use external git repository for the test?

BTW, I'm less sure than before about bzr-git should support unusual modes.
I can fix msgpack repository with git fast-export & fast-import.

[Jelmer Vernooij (jelmer)]

On Fri, 2010-02-05 at 05:47 +0000, INADA Naoki wrote:
> I failed to make an unusual mode with GitBranchBuilder, because git fast-import rejects unusual mode...
> Could you make it with dulwich? Or can I use external git repository for the test?
You're welcome to use an existing git repository for the test. Please
  keep it as small as possible so that we don't grow the Dulwich source
 unnecessarily and so that we can easily debug issues.

> BTW, I'm less sure than before about bzr-git should support unusual modes.
> I can fix msgpack repository with git fast-export & fast-import.
As long as there are git repositories out there that contain unusual
modes we have to support them.

Cheers,

Jelmer

[methane (songofacandy)]

I've pushed test repo to github and confirmed test_unusualmode works.
I've made a merge request.