NoSuchRevision error during git import

[Jelmer Vernooij]

I want to import a git repository from Google Code. But it fails with http://launchpadlibrarian.net/82551568/chimaira-development-chimaira-packages.log :

2011-10-11 16:25:12 INFO Starting job.
2011-10-11 16:25:12 INFO Getting exising bzr branch from central store.
2011-10-11 16:25:13 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-10-11 16:25:13 INFO 35 bytes transferred
2011-10-11 16:25:13 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-10-11 16:25:14 INFO Importing branch.
2011-10-11 16:25:21 INFO finding revisions to fetch 0
2011-10-11 16:25:22 INFO finding revisions to fetch:generating index 0/1918
2011-10-11 16:25:23 INFO finding revisions to fetch 1
2011-10-11 16:25:23 INFO
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/scripts/code-import-worker.py", line 98, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/scripts/code-import-worker.py", line 93, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/lib/lp/codehosting/codeimport/worker.py", line 550, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/lib/lp/codehosting/codeimport/worker.py", line 740, in _doImport
    inter_branch.fetch(limit=revision_limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/branch.py", line 666, in fetch
    self.fetch_objects(stop_revision, fetch_tags=fetch_tags, limit=limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/branch.py", line 689, in fetch_objects
    determine_wants, self.source.mapping, limit=limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/fetch.py", line 676, in fetch_objects
    limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/fetch.py", line 491, in import_git_objects
    target_git_object_retriever, trees_cache)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/fetch.py", line 370, in import_git_commit
    parent_trees = trees_cache.revision_trees(rev.parent_ids)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/object_store.py", line 112, in revision_trees
    return list(self.iter_revision_trees(revids))
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/bzrplugins/optional/git/object_store.py", line 106, in iter_revision_trees
    for tree in self.repository.revision_trees(todo):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/vf_repository.py", line 1794, in revision_trees
    for inv in inventories:
  File "/srv/importd.launchpad.net/production/launchpad-rev-14124/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/repofmt/groupcompress_repo.py", line 981, in _iter_inventories
    raise errors.NoSuchRevision(self, record.key)
bzrlib.errors.NoSuchRevision: CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-for-branch-517295/bzr_branch/.bzr/repository/') has no revision ('git-v1:a416d1fb4662544f6bac0bbc528b5dc8d1f39f87',)
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

See also https://code.launchpad.net/~chimaira-development/chimaira/packages for branch site. Actually launchpad fetches from "https://code.google.com/p/chimaira.git" but "https://code.google.com/p/chimaira" results same error.

Perhaps it is important that this git repo was a svn repo originally.

How can I prevent this error and achieve an successful import? Thanks in advance.

[Bert Voegele (bertvoegele-deactivatedaccount)]

The same error message comes up for importing from [https://code.google.com/p/iksemel ] .

In the question linked to this bug, a soon to be deployed version of bzr-git is mentioned. Since this was a couple of weeks ago, I wonder if this new version is already deployed?

[Martin Pool (mbp)]

On 7 November 2011 18:52, Bert Voegele <email address hidden> wrote:
> The same error message comes up for importing from
> [https://code.google.com/p/iksemel ] .
>
> In the question linked to this bug, a soon to be deployed version of
> bzr-git is mentioned. Since this was a couple of weeks ago, I wonder if
> this new version is already deployed?

not yet, but any day now. it is in the queue to be deployed, through
https://bugs.launchpad.net/launchpad/+bug/485932 and
http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/14253

[Bert Voegele (bertvoegele-deactivatedaccount)]

After seeing the status of the above mentioned bug (https://bugs.launchpad.net/launchpad/+bug/485932 [ Fix Commited on Nov. 8 afternoon, UTC], I just requested a new import. The import log shows exactly the same error message like before.

[Martin Pool (mbp)]

fixcommitted does not mean the fix is deployed. It should _now_ be
deployed so you might like to try again.

[Dominique Lasserre (lasall)]

Unfortunately, import ends with same error (on clean branch). See log here: http://launchpadlibrarian.net/84880201/lasall-chimaira-trunk.log

Is this version already deployed on all servers? How can I check the version?

[Jelmer Vernooij (jelmer)]

Are you sure this discussion is on the right bug ? This bug isn't marked fix in either bzr-git or launchpad.

[Alberto Mardegan (mardy)]

Also happens here:
https://code.launchpad.net/~mardy/webcredentials-libaccounts-glib/trunk

It's a git repository in google project hosting as well; but it wasn't a SVN repository originally.

[Bert Voegele (bertvoegele-deactivatedaccount)]

@jelmer : no offense, but wasn't it you in comment #3 to the linked question (see related question above), who wrote : "... Since I can't reproduce it locally, I hope that the newer version of bzr-git that is due to be deployed on Launchpad will fix the issue." ?

Hence the comments (3,5 and 7) - just to give feedback.

[Jelmer Vernooij (jelmer)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/11/11 20:14, Bert Voegele wrote:
> @jelmer : no offense, but wasn't it you in comment #3 to the
> linked question (see related question above), who wrote : "...
> Since I can't reproduce it locally, I hope that the newer version
> of bzr-git that is due to be deployed on Launchpad will fix the
> issue." ?
>
> Hence the comments (3,5 and 7) - just to give feedback
Ah, sorry - you're right.

It does indeed seem like this bug hasn't actually been fixed yet. FWIW I
can also reproduce the issue now trying to clone chimaira locally.

Cheers,

Jelmer

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOvrvQAAoJEACAbyvXKaRXQXEP/jREynQEAix8HQu1DMI+vE+8
E1mu1sA0hnH+DdHmIzVfiZ6j1mGPM0SpUg60l1QkdATmUviYeY2SEGOzlskFb5Xk
3jikt/3olz3TyXr+Tup+Eqea5skvi4oLniK1BOkk8H2SWj6mgZJEMBVw7K6vHHL3
nkoRIgiVuXP+Z9x57rFajJQ+97tE346DJnPphNb4LZEmYcpUKiSeQRIacTsaZKI3
6eEfNPrnXaG68fZWB63nJXJ1WJD2IYGMpWmdhLfpL8YrBkMzFv78efyqSHtWIhCU
/V2TJ+FWSYr9CsdW+IQzvNc4EhY4kihdrz+jESbVtVzFW6gXO3U0+U3mrdWTgZPW
kmlhISkbWoH0hlmIrQVR2ZD+oJMF+kfVHj4HyqB+TbPvASNh7qS9h3tMPMnq5+vI
bySPz6Oi4+T0uLcI9TS8fRgPuwSVzrsuFLiLSVnIzG5DQ4sr2J6lb6J1EuWOQgjm
Nx7bGmR3yFHXLzLjPT98SPYzcXrbZPW2B2M1TwTlBLyUT1kskVcHxFTn7YoY6C3o
jtxRj+Rf0PhMgFwbCR8owQM/tqbHuaQkN8L6wSXD5HzuznV2rGbm9M1BZ3MIuWoG
bJFNrI96IRr+wcgE+S3UsJIFAqbeG0CmJzB5tYhtAxYXtlLILPQGceoiajievE5i
35f9VpgXon3gJ2CVy6jl
=ffM6
-----END PGP SIGNATURE-----

[Alberto Mardegan (mardy)]

We renamed the project, so here are the logs from our failed imports:
https://code.launchpad.net/~mardy/opensesame-libaccounts-glib/trunk

We got around this problem by manually importing the git repository into bzr in my local machine, and then pushing that to LP; but in this way there is no indication in LP that the repository is an import of a remote one.
Such indication would be very helpful to direct contributors to propose their merge requests upstream, and not on the LP repository (though we can indeed handle them as well manually, having the contributors work upstream could save us some time).

[Liviu Andronic (landronimirc)]

I too have been hit with this issue and here are my findings.

I am trying to mirror SuperTux from GIT on Google Code [1]. As others have noticed, Launchpad fails to do this automatically. [2][3]
[1] https://code.google.com/p/supertux/
[2] https://code.launchpad.net/~landronimirc/supertux/master
[3] http://launchpadlibrarian.net/125511238/landronimirc-supertux-master.log

What is strange though is that 'bzr-git' works fine if it is invoked on a local clone of the project. So this is what I did:
$ mkdir /tmp/supertux
$ mkdir /tmp/supertux-bzr
$ cd /tmp/supertux
$ git clone https://code.google.com/p/supertux/
$ cd /tmp/supertux-bzr
$ bzr branch /tmp/supertux
$ cd master
$ bzr push lp:~landronimirc/supertux/trunk

And voila: I have imported SuperTux into Launchpad. [4]
[4] https://code.launchpad.net/~landronimirc/supertux/trunk

Now to update it should suffice, I think, to:
$ cd /tmp/supertux
$ git pull
$ cd /tmp/supertux-bzr/master
$ bzr update /tmp/supertux
$ bzr push lp:~landronimirc/supertux/trunk

I guess I could wrap this up into a cron job to automate the process.

But here's a question to the 'bzr-git' devels: Why would 'bzr branch ..' fail while importing directly from Google Code, but work when the GIT repo has first been cloned locally? Could this be implemented on Launchpad to allow automatic code imports from Google GIT?

[Jelmer Vernooij (jelmer)]

> But here's a question to the 'bzr-git' devels: Why would 'bzr branch ..' fail while importing directly from Google Code, but work
> when the GIT repo has first been cloned locally?
There are different code paths in bzr-git that handle local and remote repositories; presumably the remote repository path doesn't work for some reason.

> Could this be implemented on Launchpad to allow automatic code imports from Google GIT?
Possibly, but it would be a fairly complicated fix considering how the imports work; it will be easier to just fix the underlying bug in dulwich or bzr-git.

[Liviu Andronic (landronimirc)]

A quick fix:
$ cd /tmp/supertux
$ git pull
$ cd /tmp/supertux-bzr/master
$ bzr pull /tmp/supertux
$ bzr push lp:~landronimirc/supertux/trunk

In the above it should be 'pull' instead of 'update'. This can be put into a script to easily sync the Launchpad repo.

[Jelmer Vernooij (jelmer)]

On Sat, 2012-12-15 at 10:36 +0000, Liviu Andronic wrote:
> A quick fix:
> $ cd /tmp/supertux
> $ git pull
> $ cd /tmp/supertux-bzr/master
> $ bzr pull /tmp/supertux
> $ bzr push lp:~landronimirc/supertux/trunk
>
> In the above it should be 'pull' instead of 'update'. This can be put
> into a script to easily sync the Launchpad repo.
It's not quite as simple as that. The imports happen on 3 or 4 different
machines that don't have enough space to keep git clones of all imports
around.

At the moment the importer machines just clone the existing bzr branch
from the main server, do a pull from git (which usually doesn't actually
pull in any changes) and then push the newly pulled in changes back to
the master copy.

Cheers,

Jelmer

[pataquets (pataquets)]

I tried several times to do imports from Google Code, failing every time with "Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.". Looks like GC import is broken.
This one is badly needed for LP, since I haven't managed to make any GC code import to work, to date.

[Jelmer Vernooij (jelmer)]

On Sun, May 19, 2013 at 11:02:17AM -0000, pataquets wrote:
> I tried several times to do imports from Google Code, failing every time with "Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.". Looks like GC import is broken.
> This one is badly needed for LP, since I haven't managed to make any GC code import to work, to date.
That message is just the code importer giving up. It doesn't say anything about the actual reason the import is failing.

Cheers,

Jelmer

[pataquets (pataquets)]

Full log from the code at https://code.launchpad.net/~pataquets/scriptfs/trunk :
http://launchpadlibrarian.net/140258850/pataquets-scriptfs-trunk.log

Other GC imports failed in previous attempts when importing, too.

[Jelmer Vernooij (jelmer)]

On Sun, May 19, 2013 at 10:22:10PM -0000, pataquets wrote:
> Full log from the code at https://code.launchpad.net/~pataquets/scriptfs/trunk :
> http://launchpadlibrarian.net/140258850/pataquets-scriptfs-trunk.log
>
> Other GC imports failed in previous attempts when importing, too.
Right, that's likely an issue similar to this bug report.

[William Grant (wgrant)]

This is a one-liner: TemporaryPackIterator.data needs to set self._data.pack = self. It's only triggered when resolve_object is given a REF_DELTA, which is rare because most servers nowadays support the ofs-delta capability.

[Jelmer Vernooij (jelmer)]

On Wed, Aug 14, 2013 at 12:58:41AM -0000, William Grant wrote:
> This is a one-liner: TemporaryPackIterator.data needs to set
> self._data.pack = self. It's only triggered when resolve_object is given
> a REF_DELTA, which is rare because most servers nowadays support the
> ofs-delta capability.

Merge proposals welcome :-)

Jelmer

[Launchpad QA Bot (lpqabot)]

Fixed in stable r16730 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/16730>.

[Dominique Lasserre (lasall)]

Great job, thank you!

[Launchpad QA Bot (lpqabot)]

Fixed in stable r16731 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/16731>.

[Liviu Andronic (landronimirc)]

Today I tried to import [1] https://code.google.com/p/supertux/ from Google Code and it seems to have worked as expected.

Thanks for fixing this.