bzrlib.errors.RevisionNotPresent: Revision {('hg:.hgignore', 'hg-v1:48dfd5c9dc5d1591c6c64b69777d596dda227baf')} not present in "CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-for-branch-310101/bzr_branch/.bzr/repository/')".

Bug #519709 reported by Ali Sabil on 2010-02-10
68
This bug affects 8 people
Affects Status Importance Assigned to Milestone
Bazaar Hg Plugin
High
Unassigned
Launchpad itself
High
Unassigned

Bug Description

The following mercurial import fails: https://code.launchpad.net/~asabil/prosody-im/trunk

2011-05-02 06:03:04 INFO Starting job.
2011-05-02 06:03:04 INFO Getting exising bzr branch from central store.
2011-05-02 06:03:04 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-05-02 06:03:04 INFO 35 bytes transferred
2011-05-02 06:03:05 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-05-02 06:03:07 INFO Importing foreign branch.
2011-05-02 06:03:12 INFO fetching changesets 0
2011-05-02 06:03:12 INFO
2011-05-02 06:03:13 INFO updating cache 0/871
2011-05-02 06:03:17 INFO
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/scripts/code-import-worker.py", line 82, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/scripts/code-import-worker.py", line 77, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/lib/lp/codehosting/codeimport/worker.py", line 452, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/lib/lp/codehosting/codeimport/worker.py", line 590, in _doImport
    overwrite=True, **self.getExtraPullArgs())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/branch.py", line 279, in pull
    inter.fetch(revision_id=stop_revision)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 651, in fetch
    self.addchangegroup(cg, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 517, in addchangegroup
    self._target_overlay.remember_manifest_texts(self._unpack_manifests(manifestchunks, mapping, filetext_map, todo, pb))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 125, in remember_manifest_texts
    ((FulltextContentFactory((revid,), [(p,) for p in parent_revids] , None, text) for (revid, parent_revids, text) in entries)))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/knit.py", line 1600, in insert_record_stream
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 125, in <genexpr>
    ((FulltextContentFactory((revid,), [(p,) for p in parent_revids] , None, text) for (revid, parent_revids, text) in entries)))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 455, in _unpack_manifests
    unpack_chunk_iter(chunkiter, self._target_overlay.get_manifest_text)):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/parsers.py", line 153, in unpack_chunk_iter
    textbase = lookup_base(base)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 200, in get_manifest_text
    return self.get_manifest_text_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 215, in get_manifest_text_by_revid
    (manifest, flags) = self.get_manifest_and_flags_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 246, in get_manifest_and_flags_by_revid
.....
    return self._reconstruct_manifest_and_flags_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 251, in _reconstruct_manifest_and_flags_by_revid
    rev = self.repo.get_revision(revid)
  File "<string>", line 4, in get_revision_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repofmt/knitrepo.py", line 209, in get_revision
    return self.get_revision_reconcile(revision_id)
  File "<string>", line 4, in get_revision_reconcile_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1892, in get_revision_reconcile
    return self._get_revisions([revision_id])[0]
  File "<string>", line 4, in _get_revisions_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1907, in _get_revisions
    for revid, rev in self._iter_revisions(revision_ids):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1931, in _iter_revisions
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1413, in get_record_stream
    orig_keys, ordering, include_delta_closure):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1518, in _get_remaining_record_stream
    locations = self._index.get_build_details(keys)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 2006, in get_build_details
    for entry in entries:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1941, in _get_entries
    for node in self._graph_index.iter_entries(keys):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/index.py", line 1358, in iter_entries
    for node in index.iter_entries(keys):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 1167, in iter_entries
    nodes, nodes_and_keys = self._walk_through_internal_nodes(needed_keys)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 1121, in _walk_through_internal_nodes
    nodes = self._get_leaf_nodes(node_indexes)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 962, in _get_leaf_nodes
    found = self._get_nodes(self._leaf_node_cache, node_indexes)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 933, in _get_nodes
    found[idx] = cache[idx]
RuntimeError: maximum recursion depth exceeded
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

Related branches

  importance high
  status triaged
  affects bzr-hg
  tag code-import
  tag hg

Changed in launchpad-code:
importance: Undecided → High
status: New → Triaged
Max Bowsher (maxb) on 2010-06-16
summary: - Mercurial import for prosody fails
+ Import fails with infinite recursion through
+ _reconstruct_manifest_and_flags_by_revid
Jelmer Vernooij (jelmer) on 2010-06-16
Changed in bzr-hg:
status: New → Triaged
importance: Undecided → Medium
Timmie (timmie) wrote :

Why does this bug occur?

Martin Pool (mbp) wrote :

These are still failing, but not with infinite recursion:

Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/scripts/code-import-worker.py", line 82, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/scripts/code-import-worker.py", line 77, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/lib/lp/codehosting/codeimport/worker.py", line 452, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/lib/lp/codehosting/codeimport/worker.py", line 590, in _doImport
    overwrite=True, **self.getExtraPullArgs())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/branch.py", line 279, in pull
    inter.fetch(revision_id=stop_revision)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 651, in fetch
    self.addchangegroup(cg, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 530, in addchangegroup
    self._add_inventories(todo, mapping, pb)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 385, in _add_inventories
    manifest, flags, files, rev, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 337, in _import_manifest_delta
    self._symlink_targets.__getitem__))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 207, in manifest_to_inventory_delta
    (fileid, ie.revision))
KeyError: ('hg:jeash_sswf_sEditText.hx', 'hg-v1:dad204a930128dc913c27020691e7be7894bc81f')

Perhaps this is a dupe now?

Martin Pool (mbp) wrote :

This looks a lot like bug 692901; please un-dupe if it's not.

Martin Pool (mbp) wrote :

i think some of these branches failed due to bug 692901, but this is actually a distinct bug. it's occurring now in for instance <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror> <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-spyder_devel_mirror.log>.

description: updated
Changed in bzr-hg:
importance: Medium → High

On Mon, 2011-05-09 at 15:29 +0000, Martin Pool wrote:
> i think some of these branches failed due to bug 692901, but this is
> actually a distinct bug. it's occurring now in for instance
> <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror>
> <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-
> spyder_devel_mirror.log>.
I think this is actually fixed in bzr-hg trunk, trying it manually at
the moment to confirm.

Cheers,

Jelmer

Importing manually works in Ubuntu 11.04.
But not in Launchpad.

On Mon, 2011-05-09 at 17:52 +0200, Jelmer Vernooij wrote:
> On Mon, 2011-05-09 at 15:29 +0000, Martin Pool wrote:
> > i think some of these branches failed due to bug 692901, but this is
> > actually a distinct bug. it's occurring now in for instance
> > <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror>
> > <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-
> > spyder_devel_mirror.log>.
> I think this is actually fixed in bzr-hg trunk, trying it manually at
> the moment to confirm.
Confirmed that this works with bzr-hg trunk - we should get a new bzr-hg
deployed on Launchpad.

Cheers,

Jelmer

@Jelmer should the bzr-hg task of this bug be closed?

I believe we were blocked on the update because of the change in handling of incremental fetches and that is now fixed? So all that's needed to close the lp side of this is to just actually push a bzr-hg update into lp's dependencies, and probably also a new bzr?

Martin Pool (mbp) wrote :

Going off jelmer's comment i'm going to mark this fixed; please reopen if it's not.

Changed in bzr-hg:
assignee: nobody → Jelmer Vernooij (jelmer)
status: Triaged → Fix Released
Changed in launchpad:
assignee: nobody → Diogo Matsubara (matsubara)
status: Triaged → In Progress

On Fri, 27 May 2011 19:47:01 -0000, Martin Pool <email address hidden> wrote:
> @Jelmer should the bzr-hg task of this bug be closed?
>
> I believe we were blocked on the update because of the change in
> handling of incremental fetches and that is now fixed? So all that's
> needed to close the lp side of this is to just actually push a bzr-hg
> update into lp's dependencies, and probably also a new bzr?

I don't know the details, but I'm fairly sure the incremental fetch
stuff has never worked with bzr-hg, so updating just bzr-hg cannot
possibly make this aspect worse.

Cheers,
mwh

Jelmer Vernooij (jelmer) wrote :

On Sun, 2011-05-29 at 23:39 +0000, Michael Hudson-Doyle wrote:
> On Fri, 27 May 2011 19:47:01 -0000, Martin Pool <email address hidden> wrote:
> > @Jelmer should the bzr-hg task of this bug be closed?
> >
> > I believe we were blocked on the update because of the change in
> > handling of incremental fetches and that is now fixed? So all that's
> > needed to close the lp side of this is to just actually push a bzr-hg
> > update into lp's dependencies, and probably also a new bzr?
>
> I don't know the details, but I'm fairly sure the incremental fetch
> stuff has never worked with bzr-hg, so updating just bzr-hg cannot
> possibly make this aspect worse.
The problem is that updating bzr-hg will require a newer bzr on
Launchpad too (current version is 2.2-something), and updating bzr will
require updating bzr-git/bzr-svn too - and they are affected by that
issue.

All of this is in progress, I've landed newer versions of bzr-svn,
subvertpy, dulwich, bzr and bzr-git on the Launchpad branches and I
think Diogo has landed a newer bzr-hg. There is some fallout in the
Launchpad tree that I hope to work on this week.

Cheers,

Jelmer

I ec2 tested the branch with bzr-hg updated to 284 and bzr updated to 2.3.3. I got the test failures in update-bzr-hg-r13135.subunit.gz. I then ec2 tested the branch only with the bzr-hg update and got less test failure (update-bzr-hg-r13137.subunit.gz), but it failed nonetheless.

Diogo Matsubara (matsubara) wrote :
Robert Collins (lifeless) wrote :

We're now running r292 of bzr-hg

Changed in launchpad:
status: In Progress → Fix Committed
status: Fix Committed → Fix Released
status: Fix Released → Incomplete
Robert Collins (lifeless) wrote :

I've kicked the import off to see if its really fixed

Robert Collins (lifeless) wrote :
Download full text (3.2 KiB)

So the referenced branch fails with
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/scripts/code-import-worker.py", line 99, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/scripts/code-import-worker.py", line 94, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/lib/lp/codehosting/codeimport/worker.py", line 591, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/lib/lp/codehosting/codeimport/worker.py", line 745, in _doImport
    inter_branch.fetch(limit=revision_limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/branch.py", line 483, in fetch
    inter.fetch(revision_id=stop_revision, limit=limit)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 766, in fetch
    self.addchangegroup(cg, mapping, limit=limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 631, in addchangegroup
    self.target.texts.insert_record_stream(texts)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1662, in insert_record_stream
    for _ in self._insert_record_stream(stream, random_id=False):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1752, in _insert_record_stream
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 417, in _unpack_texts
    itertextchunks, get_text):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/parsers.py", line 166, in unpack_chunk_iter
    textbase = lookup_base(base)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 415, in get_text
    return self._get_target_fulltext(key)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 375, in _get_target_fulltext
    return self._target_overlay.get_file_fulltext(key)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/overlay.py", line 229, in get_file_fulltext
    ret = "".join(self.repo.iter_files_bytes([key + (None,)]).next()[1])
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/vf_repository.py", line 1516, in iter_files_bytes
    raise errors.RevisionNotPresent(record.key, self)
bzrlib.errors.RevisionNotPresent: Revision {('hg:.hgignore', 'hg-v1:48dfd5c9dc5d1591c6c64b69777d596dda227baf')} not present in "CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-for-branch-310101/bzr_branch/.bzr/repository/')".
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

which is at least a ...

Read more...

Changed in bzr-hg:
status: Fix Released → Confirmed
Changed in launchpad:
status: Incomplete → Triaged
summary: - Import fails with infinite recursion through
- _reconstruct_manifest_and_flags_by_revid
+ https://code.launchpad.net/~asabil/prosody-im/trunk fails to import
Jelmer Vernooij (jelmer) on 2012-02-03
Changed in bzr-hg:
assignee: Jelmer Vernooij (jelmer) → nobody
Jelmer Vernooij (jelmer) on 2012-05-17
tags: added: bzr-hg
Jelmer Vernooij (jelmer) on 2012-06-28
Changed in launchpad:
assignee: Diogo Matsubara (matsubara) → nobody
Jelmer Vernooij (jelmer) on 2012-08-08
summary: - https://code.launchpad.net/~asabil/prosody-im/trunk fails to import
+ bzrlib.errors.RevisionNotPresent: Revision {('hg:.hgignore',
+ 'hg-v1:48dfd5c9dc5d1591c6c64b69777d596dda227baf')} not present in
+ "CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-
+ for-branch-310101/bzr_branch/.bzr/repository/')".
Curtis Hovey (sinzui) on 2013-01-03
Changed in launchpad:
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers