Doesn't preserve non-lhs parents for file texts

Bug #250480 reported by John Carr
22
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
Undecided
Unassigned
Bazaar Subversion Plugin
Fix Released
Critical
Jelmer Vernooij

Bug Description

As per irc conversation with lifeless, nzjrs and jelmer.

11:13 < lifeless> data looks completely fine
11:13 < lifeless> whats the branch you want to merge this with?
11:13 <@Jc2k> jstowers/conduit/devel on playground
11:13 <@Jc2k> which, aiui, is really just trunk right now
11:13 <@Jc2k> (which doesnt work either)
11:13 < nzjrs> I want to merge it with trunk. It hasnt been mreged in a week or so. The reason I started all this pain is because I want to push it too
               gnome bzr now, for people to browse
11:14 < nzjrs> (what Jc2k said)
11:14 < lifeless> ok, lets see what the difference is; bzr missing http://bzr-playground.gnome.org/~jstowers/conduit/devel ...
11:15 < lifeless> I'm just doing a check first
11:16 < lifeless> hmm, inconsistent parents; I wonder if bzr-svn needs a tweak
11:16 < lifeless> and four ghosts
11:16 < lifeless> jelmer: ^
11:17 < lifeless> ok, different error
11:18 < lifeless> running bzr check in /bzr/jstoers/conduit/devel
11:19 < lifeless> I have to go. Jc2k - can you turn this into a bug (there is a root cause here somewhere; might be bzr-svn, might be bzr), mark it critical
                  (user behaviour should never be able to cause this sort of confusion)
11:20 < lifeless> 6 ghosts and 10 inconsistent parents
11:20 <@Jc2k> sure thing
11:20 < lifeless> hopefully jelmer or odd_bloke or someone in a eu or possibly us tz can look at this
11:21 < nzjrs> lifeless, can I continue working on these branches, just starting with fresh branches of each?
11:21 < lifeless> I'll be away for ~0 hours minimum
11:21 < jelmer> lifeless: bzr-svn doesn't push right hand side parent information for individual files
11:21 < lifeless> jelmer: that explains the ghosts perhaps; but not the inconsistent parents
11:21 < lifeless> jelmer: more concerning is the missing text errors
11:22 < nzjrs> lifeless, it might be a coincidence, but the last successful push of that branch to svn resulted in every file in svn getting touched
11:22 < nzjrs> which I thought was odd
11:22 <@Jc2k> yes, i've noticed that
11:22 <@Jc2k> its like it copied trunk on top of itself
11:22 <@Jc2k> (which confuses the bejesus out of the stock launchpad svn import iirc)
11:23 < lifeless> what is really strange i
11:23 < lifeless> *is*
11:23 < lifeless> that both branches appears to have the damaged revisions
11:23 < lifeless> but to have different bits of them

Related branches

Revision history for this message
Robert Collins (lifeless) wrote :

I strongly suspect bzr-svn at this point. comparing the actual inventory for one of the problem revisions between both trees is likely to show differences (which is a violation of the identity-per-key rule in bzr).

Changed in bzr-svn:
importance: Undecided → Critical
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

What's the exact command that is failing? running "bzr missing" in a copy of http://bzr-playground.gnome.org/%7Ejstowers/conduit/devel/ against the trunk of conduit works fine here.

Revision history for this message
John Stowers (nzjrs) wrote :

The original problem was, I have been working on a branch for a few weeks at

http://www.johnstowers.co.nz/bzr/conduit-gio/

This was originally branched from trunk

http://bzr-mirror.gnome.org/conduit/trunk/

About a week ago it became impossible to merge in trunks changes into that branch.

bzr: ERROR: Revision {<email address hidden>} not present in "951@1811c9d2-c306-0410-a128-ae57aa55c946:trunk:conduit%2Fmodules%2FFileModule".

I dont understand the relevance of using http://bzr-playground.gnome.org/~jstowers/conduit/devel, as excluding delays in bzr-plyground updating trunk from svn, it should usually be the same as trunk anyway.

Unfortunately, due to above errors, the conduit-gio branch at johnstowers.co.nz became unbranchable. Lifeless instructed me to branch my local copy to /tmp, and then push this again (to deep copy the history, or something, IIRC). This is now available at

bzr branch http://www.johnstowers.co.nz/bzr/test-bzr-conduit

I want to be able to merge trunk changes into

http://www.johnstowers.co.nz/bzr/test-bzr-conduit or
http://www.johnstowers.co.nz/bzr/conduit-gio/

And then push that to http://bzr-playground.gnome.org/jstowers/... for people to actually test, and because It will go into trunk in a few weeks

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 250480] Re: Missing texts, ghosts and inconsistent parents

On Wed, 2008-07-23 at 01:12 +0000, Jelmer Vernooij wrote:
>
>
>
>
> What's the exact command that is failing? running "bzr missing" in a
> copy of http://bzr-playground.gnome.org/%7Ejstowers/conduit/devel/
> against the trunk of conduit works fine here.

bzr branch http://bzr-playground.gnome.org/~jstowers/conduit/devel 1
bzr branch http://www.johnstowers.co.nz/bzr/test-bzr-conduit 2
cd 1
bzr merge ../2
*boom*
cd ../2
bzr merge ../1
*boom*

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Am Mittwoch, den 23.07.2008, 01:59 +0000 schrieb Robert Collins:
> On Wed, 2008-07-23 at 01:12 +0000, Jelmer Vernooij wrote:
> >
> > What's the exact command that is failing? running "bzr missing" in a
> > copy of http://bzr-playground.gnome.org/%7Ejstowers/conduit/devel/
> > against the trunk of conduit works fine here.
>
> bzr branch http://bzr-playground.gnome.org/~jstowers/conduit/devel 1
> bzr branch http://www.johnstowers.co.nz/bzr/test-bzr-conduit 2
> cd 1
> bzr merge ../2
> *boom*
> cd ../2
> bzr merge ../1
> *boom*
The particular revision it is complaining about being missing is a ghost
revision (right-hand side so not actually fetchable from svn) so it
seems alright that is is not actually able to find that text.

The first merge command above seems to work for me as long as I fetch
ghosts first:

cd 1
bzr fetch-ghosts ../2
bzr merge ../2

I'm surprised that is necessary though.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden> - http://samba.org/~jelmer/
Jabber: <email address hidden>

Revision history for this message
Jelmer Vernooij (jelmer) wrote : Re: Missing texts, ghosts and inconsistent parents

this all appears to be caused by bzr-svn returning only the lhs parent revid for files in merges. this is fixable but requires the use of another svn file property.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

downgrade importance, since there is a workaround

Changed in bzr-svn:
importance: Critical → High
Jelmer Vernooij (jelmer)
Changed in bzr-svn:
status: New → Triaged
Jelmer Vernooij (jelmer)
Changed in bzr-svn:
milestone: none → 0.5.0
Revision history for this message
John Stowers (nzjrs) wrote :

It should be noted that

cd 1
bzr fetch-ghosts ../2
bzr merge ../2

does not work in the reverse direction.

cd 2
bzr fetch-ghosts ../1
bzr merge ../

It gives a KnitCorrupt error

Revision history for this message
John Stowers (nzjrs) wrote :

Can we please upgrade this to critical again, the workaround does not work...

after doing
cd 1
bzr fetch-ghosts ../2
bzr merge ../2

bzr ci -m "Merge foo branch"

The following happens
bzr push /tmp/foobzr: ERROR: Revision {<email address hidden>} not present in "56@1811c9d2-c306-0410-a128-ae57aa55c946:trunk:conduit%2FSynchronization.py".

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Trying to merge fails on bzr trying to access file id

951@1811c9d2-c306-0410-128--e57aa55c946:trunk:conduit%2Fmodules%2FFileModule%2FFileModule.py

in revision

 john@nzjrs-uni-20080604054725-xb4vevcba6m9c8u6

this revision id doesn't appear anywhere in branch "1"'s log and nowhere in the inventory xmls.

It does exist in "2" though where it is the parent of a right hand side parent of a revision that is also in "1", <email address hidden>

it seems to me like this is a case where merge doesn't fetch all the ghosts into the target repository before doing the actual merge.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

I'm not sure what's causing the Knit Corrupt Error. It would be a strange coincidence if this is unrelated.

The sha1sum for the inventory of svn-v3-trunk0:1811c9d2-c306-0410-a128-ae57aa55c946:trunk:1533 doesn't match the actual contents in "2". I'm not sure how bzr-svn is breaking this though - it only ever installs inventories by calling Repository.add_revision(). It never even specifies the sha1 and leaves all the marshalling to that function. It also doesn't insert inventory deltas, only full inventories.

The full error is:

bzrlib.errors.KnitCorrupt: Knit inventory corrupt:
  sha-1 01d855e337787d2e171eeb10b52ca40e4cf080a7
  of reconstructed text does not match
  expected 669584ff8344d2f403846bbb6bcc43813c4fb77f
  for version svn-v3-trunk0:1811c9d2-c306-0410-a128-ae57aa55c946:trunk:1533

fwiw, the expected sha1 matches the actual sha1 of the inventory found in "1".

Robert, any thoughts?

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 250480] Re: Doesn't preserve non-lhs parents for file texts

On Tue, 2008-07-29 at 23:48 +0000, Jelmer Vernooij wrote:
> Trying to merge fails on bzr trying to access file id
>
> 951@1811c9d2-c306-0410-128--
> e57aa55c946:trunk:conduit%2Fmodules%2FFileModule%2FFileModule.py
>
> in revision
>
> john@nzjrs-uni-20080604054725-xb4vevcba6m9c8u6
>
> this revision id doesn't appear anywhere in branch "1"'s log and nowhere
> in the inventory xmls.
>
> It does exist in "2" though where it is the parent of a right hand side
> parent of a revision that is also in "1",
> <email address hidden>
>
> it seems to me like this is a case where merge doesn't fetch all the
> ghosts into the target repository before doing the actual merge.

Well, merge does a fetch(), not a fetch-ghosts, because ghost checking
is O(history) and thats a problem.

One possible fix, if it truely is just a ghost issue is to have a
error-handler do a JIT fetch for this sort of thing.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

Revision history for this message
Robert Collins (lifeless) wrote :

So, first steps for me retackling this post-travel :)

robertc@lifeless-64:/tmp$ cd conduit
robertc@lifeless-64:/tmp/conduit$ ls
robertc@lifeless-64:/tmp/conduit$ bzr branch http://bzr-mirror.gnome.org/conduit/trunk/
Branched 1515 revision(s).
robertc@lifeless-64:/tmp/conduit$ bzr branch http://www.johnstowers.co.nz/bzr/test-bzr-conduit
Branched 1457 revision(s).
robertc@lifeless-64:/tmp/conduit$ cp -a test-bzr-conduit/ test
robertc@lifeless-64:/tmp/conduit$ cd test
robertc@lifeless-64:/tmp/conduit/test$ bzr merge ../trunk/
bzr: ERROR: Revision {<email address hidden>} not present in "809@1811c9d2-c306-0410-a128-ae57aa55c946:trunk:help%2FC".
robertc@lifeless-64:/tmp/conduit/test$ vim ~/.bzr.log

  File "/home/robertc/source/baz/bzr-test-fixes/bzrlib/repository.py", line 2765, in fetch
    revision_ids).pack()
  File "/home/robertc/source/baz/bzr-test-fixes/bzrlib/repofmt/pack_repo.py", line 594, in pack
    return self._create_pack_from_packs()
  File "/home/robertc/source/baz/bzr-test-fixes/bzrlib/repofmt/pack_repo.py", line 743, in _create_pack_from_packs
    self._check_references()
  File "/home/robertc/source/baz/bzr-test-fixes/bzrlib/repofmt/pack_repo.py", line 705, in _check_references
    missing_file_id)
RevisionNotPresent: Revision {<email address hidden>} not present in "809@1811c9d2-c306-0410-a128-ae57aa55c946:trunk:help%2FC".

So - fetch is fetching the new content. Which is delta'd against a text that is a ghost in the target.

This doesn't talk about inventories or anything at this point.

Its possible for a text referenced in an inventory to be deltad against a non-LHS revision (from that inventories perspective) if the text-change is one that was made in a non-LHS revision and no concurrent changes were made before it was merged; that is how you can get a revision referenced in an inventory in svn that was not part of the svn lhs history.

Jelmer Vernooij (jelmer)
Changed in bzr-svn:
assignee: nobody → jelmer
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

so, bzr-svn will have to start storing the revision id for each file text in a file property just like it does for other bzr-specific metadata.

Unfortunately this will badly hurt stacking performance since lookup up a file text may take O(rn) time (n as size of ancestry, r as roundtrip time to the svn server).

This will likely improve somewhat when revision properties-based mappings land, it'll make looking up such texts O(n) and probably even significantly better than that in the general case when the svn server is running svn >= 1.5.

Jelmer Vernooij (jelmer)
Changed in bzr-svn:
importance: High → Critical
milestone: 0.5.0 → 0.4.11
Revision history for this message
Robert Collins (lifeless) wrote :
Download full text (3.5 KiB)

re the knitcorrupt:
the full backtrace:
File "/data/jelmer/bzr/bzr.dev/bzrlib/commands.py", line 857, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/data/jelmer/bzr/bzr.dev/bzrlib/commands.py", line 797, in run_bzr
    ret = run(*run_argv)
  File "/home/jelmer/.bazaar/dev-plugins/bzrtools/command.py", line 13, in run_argv_aliases
    commands.Command.run_argv_aliases(self, argv, alias_argv)
  File "/data/jelmer/bzr/bzr.dev/bzrlib/commands.py", line 499, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/jelmer/.bazaar/dev-plugins/bzrtools/__init__.py", line 157, in run
    fetch_ghosts(branch, no_fix)
  File "/home/jelmer/.bazaar/dev-plugins/bzrtools/fetch_ghosts.py", line 67, in fetch_ghosts
    cmd_reconcile().run(".")
  File "/home/jelmer/bzr/bzr.dev/bzrlib/builtins.py", line 1167, in run
    reconcile(dir)
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 53, in reconcile
    reconciler.reconcile()
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 77, in reconcile
    self._reconcile()
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 84, in _reconcile
    self._reconcile_repository()
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 103, in _reconcile_repository
    repo_reconciler = self.repo.reconcile(thorough=True)
  File "/data/jelmer/bzr/bzr.dev/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repofmt/pack_repo.py", line 1861, in reconcile
    reconciler.reconcile()
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 195, in reconcile
    self._reconcile_steps()
  File "/data/jelmer/bzr/bzr.dev/bzrlib/reconcile.py", line 510, in _reconcile_steps
    new_pack = self._packer.pack(pb=self.pb)
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repofmt/pack_repo.py", line 594, in pack
    return self._create_pack_from_packs()
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repofmt/pack_repo.py", line 727, in _create_pack_from_packs
    self._copy_text_texts()
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repofmt/pack_repo.py", line 1001, in _copy_text_texts
    ideal_index = repo._generate_text_key_index(self._text_refs, ancestors)
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repository.py", line 1382, in _generate_text_key_index
    text_key_references, pb)
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repository.py", line 1422, in _do_generate_text_key_index
    for rev_tree in self.revision_trees(to_query):
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repository.py", line 1671, in revision_trees
    for inv in inventories:
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repository.py", line 1538, in _iter_inventories
    for text, revision_id in self._iter_inventory_xmls(revision_ids):
  File "/home/jelmer/bzr/bzr.dev/bzrlib/repository.py", line 1545, in _iter_inventory_xmls
    for record in stream:
  File "/data/jelmer/bzr/bzr.dev/bzrlib/knit.py", line 1208, in get_record_stream
    needed_from_fallback - absent_keys)
  File "/data/jelmer/bzr/bzr.dev/bzrlib/knit.py", line 1047, in _get_content_maps
    (actual_sha, digest, key))
KnitCorrupt: Knit <bzrlib.knit.KnitVersionedFiles object at 0x91ea94c> corrupt:
  sha-1 01d85...

Read more...

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

bzr-svn neither changes existing packs nor does it insert inventory deltas (only ever full inventories), so this must be caused by something else.

Perhaps fetching the ghosts did this?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Sorry, fix for this is taking a bit longer than I hoped. Should land tomorrow.

Jelmer Vernooij (jelmer)
Changed in bzr-svn:
status: Triaged → Fix Committed
Jelmer Vernooij (jelmer)
Changed in bzr:
status: New → Fix Released
Jelmer Vernooij (jelmer)
Changed in bzr-svn:
status: Fix Committed → Fix Released
Revision history for this message
Peter Bienstman (peter-bienstman) wrote :

Coming from the duplicate bug [Bug 262762] Re: KnitCorrupt: Knit inventory corrupt, I noticed that this bug here is marked 'fix released'. I upgrade to bzr-svn 0.4.11, but 'bzr branch lp:mnemosyne-proj' fails in exactly the same way as I reported earlier.

How can I bring my repository back into a sane state?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Create a new Bazaar branch from the branch in Subversion and then pull in your existing bzr branch from there.

If your existing bzr branch is named foo:

$ bzr svn://svn.example.com/bar/trunk bar
$ cd bar && bzr pull ../foo
$ rm -rf ../foo

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.