git: AssertionError: Invalid sha for <Commit ...>: ...

[Michael Lustfield]

I have an import from Debian git that is now failing. https://code.launchpad.net/~nginx/nginx/debian
The log: http://launchpadlibrarian.net/174083895/nginx-nginx-debian.log

I've done what I know how to do to verify that this repository is accurate. Maybe there's something wrong in the git repo, but I can't find anything. I'm able to clone the repo fine and it had been working fine.

[anatoly techtonik (techtonik)]

Same here - sync for https://code.launchpad.net/%7Etechtonik/spyderlib/debian-packaging/+index stopped month ago.

http://launchpadlibrarian.net/340648241/techtonik-spyderlib-debian-packaging.log

[anatoly techtonik (techtonik)]

I switched my imports from git->bzr to git->git.

[Jelmer Vernooij (jelmer)]

This is fixed with brz-git.

[Pavel (spvkgn)]

Same here https://code.launchpad.net/~mpv-team/mpv/master/
Last successful import was on 2018-03-16.

[Petr Kubánek (petr-kubanek)]

Same here: https://code.launchpad.net/%7Erts2/rts2/trunk/+index#

[Jelmer Vernooij (jelmer)]

This is fixed in brz-git, Launchpad runs bzr-git at the moment.

[Colin Watson (cjwatson)]

We have internal agreement on moving our backend over to Breezy, but I was kind of hoping for a non-alpha release first, ideally. Is that planned soon?

[Jelmer Vernooij (jelmer)]

On Tue, Apr 10, 2018 at 09:49:39AM -0000, Colin Watson wrote:
> We have internal agreement on moving our backend over to Breezy, but I
> was kind of hoping for a non-alpha release first, ideally. Is that
> planned soon?
Do you have a bug tracking the migration that I can update on our
status?

I wouldn't recommend a migration just yet, since we're still making
largeish changes to the core APIs - i.e. to support git, nested trees and
colocated branches.

Hopefully we can do some sort of alpha release when the core APIs have
stabilized.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden>
PGP Key: https://www.jelmer.uk/D729A457.asc

[Seafile Ltd (seafile)]

The same thing happened in https://code.launchpad.net/~seafile-team/seafile/lpad, e.g. see the logs in http://launchpadlibrarian.net/370644661/seafile-team-seafile-lpad.log. Anyone could help?

[Rob Norris (rw-norris)]

Also happening to me:

https://code.launchpad.net/~rw-norris/viking/trunk

http://launchpadlibrarian.net/370842251/rw-norris-viking-trunk.log

Both claimed failing commits in my log[1] & for seafile[1] are of a "Merge pull request XXX ..." type nature performed in Github.com.

[1] https://github.com/viking-gps/viking/commit/206165de5fe6812da5d509121a6f399a79a78021
[2] https://github.com/haiwen/seafile/commit/b128c7b574cbfe3c1adcc2f5ccf012f5caa07ae8

Hope that helps any investigations.

[Jelmer Vernooij (jelmer)]

FWIW it's clear what the problem is here, and there is a fix available for this issue in Breezy.
(I've pushed a manual import of the viking branch to lp:~jelmer/viking/brz)

In order for this issue to be fixed in Launchpad, it either needs to:

* migrate to Breezy; this is currently blocked on Breezy not having a stable API - yet (see #7)
* somebody needs to fix the issue in bzr-git and then update Launchpad to use a new bzr-git

[Seafile Ltd (seafile)]

Hello guys, is there any update on this? It's blocking us from releasing new versions of seafile client on all ubuntu versions since the sync from github can't work due to this bug.

We tried to push to the bazaar branch manually, but it seems one can't push to an imported branch[1], and we don't want to setup a new branch and recipes unless there is really no alternatives.

[1] http://linux.derkeiler.com/Mailing-Lists/Ubuntu/2014-09/msg00446.html

[Colin Watson (cjwatson)]

Jelmer, could you point to what commits fixed this in brz-git? We could backport it to Launchpad's bzr-git branch, which might be the most expedient approach for now.

[Jelmer Vernooij (jelmer)]

On Tue, Aug 21, 2018 at 11:01:19AM -0000, Colin Watson wrote:
> Jelmer, could you point to what commits fixed this in brz-git? We could
> backport it to Launchpad's bzr-git branch, which might be the most
> expedient approach for now.

Note that the title of this bug is a symptom that can be caused by any
kind of data corruption bug. bzr-git is trying to create a Git object
from some Bazaar data and comes up with content that doesn't match
the SHA1 checksum that it expected.

It's possible that the various repositories mentioned in comments
for this bug are because different issues; I've only looked at the
repository in the initial description. That said..

It looks like for seafile, the repository is corrupted, since bzr-git
can't recreate the original git objects from it.

This can be seen by running "bzr git-objects".

$ bzr branch lp:~seafile-team/seafile/lpad
$ cd lpad
$ bzr git-objects

This means that that branch will need to be nuked and then reimported
once the bug is fixed.

The bug that introduces the corruption is still present in lp:bzr-git
- a fresh clone with lp:bzr and lp:bzr-git still fails the "bzr git-objects" test
mentioned above. For Breezy this is not the case.

Unfortunately brz-git has changed significantly since lp:bzr-git, and
we can't just isolate a smaller number of commits to backport. :(

blodeuwedd:~/src/breezy% bzr diff -rbranch:lp:bzr-git.. breezy/git | diffstat -s
 63 files changed, 8706 insertions(+), 4458 deletions(-)

--
Jelmer Vernooij <email address hidden>
PGP Key: https://www.jelmer.uk/D729A457.asc

[Nicholas Zatkovich (nickz-roshub)]

Any update on this? Any timeline for this bug to be fixed in the version of bzr-git used by launchpad?

[Watteel (piwi3910)]

any updates?

[Jelmer Vernooij (jelmer)]

There are two paths towards fixing this:

1) fixing bzr-git or backporting the fix from breezy to bzr-git. I don't believe there is anybody working on this.

2) Switching Launchpad over to using Breezy rather than Bazaar, since Breezy has already fixed this bug. I believe this is the eventual plan, but it's blocked on several other things (including lack of svn support in Breezy).

[i30817 (i30817)]

I have encountered this problem on lp:munt while auto-importing it from its github page

http://launchpadlibrarian.net/462662122/munt-munt-trunk.log

but it occurs (with a different object, thus sha1) even with the bzr branch/bzr git-objects test with a local branch of only the commits already there.

This is pretty annoying since to change to bzr git i have to recreate all of the recipes and the repositories they use *and* the local repositories equivalents, because i'd have to change '# bzr-builder format ...' to '# git-builder format ...'

Please fix this!

[Cameron White (cameronwhite91)]

I've also been encountering this. https://code.launchpad.net/~pinta-maintainers/pinta/trunk hasn't been updating for several months, preventing new translation strings from being imported.

Are there any updates on this?

[FloSoft (flosoft)]

We have the same problem here:

https://code.launchpad.net/~s25rttrteam/s25rttr/s25rttr-languages

blocking us from importing language changes.

Can't you simply add support for having git repositories for language/translation support?

[Colin Watson (cjwatson)]

@flosoft I started on that a while ago: the launchpad-buildd side has the necessary support as of August, but we still need to do the corresponding things to Launchpad itself. This is a bit harder than you might think due to some data model differences between bzr and git hosting in Launchpad (basically, series aren't really involved in git hosting, but they're quite central to how translation hosting works, and we need to work out how to bridge that gap). Hopefully in 2021 though!

[Alkis Georgopoulos (alkisg)]

Same here with https://code.launchpad.net/~epoptes/epoptes/epoptes
Until it's fixed, we'll probably need to manually upload the translations.pot periodically...

[Olly Betts (ojwb)]

It seems this wasn't fixed in 2021 as I'm still seeing it at https://code.launchpad.net/~ojwb/survex/master and after deleting and recreating the bzr mirror it imports OK and then seems fails to ever update after that:

 Last successful import was on 2022-02-21.
[Failure] Import started 14 hours ago on juju-1e3bde-prod-lp-code-import-15 and finished 14 hours ago taking 5 seconds — see the log
[Failure] Import started on 2022-02-23 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-23 taking 5 seconds — see the log
[Failure] Import started on 2022-02-22 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-22 taking 5 seconds — see the log
[Failure] Import started on 2022-02-22 on juju-1e3bde-prod-lp-code-import-12 and finished on 2022-02-22 taking 5 seconds — see the log
[Failure] Import started on 2022-02-21 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-21 taking 5 seconds — see the log
[Success] Import started on 2022-02-21 on juju-1e3bde-prod-lp-code-import-12 and finished on 2022-02-21 taking 2 minutes — see the log

So at least for my case, it doesn't seem that pre-existing corruption in the bzr repo is to blame, because the bzr repo is brand new.

I don't actually want a bzr mirror of my git repo as such, but that seems to be required by launchpad's translation feature since the bzr mirror got automatically created for me when I pointed to translations in my git repo. The only workaround there seems to be (repeatedly deleting and recreating the bzr mirror) isn't really practical, and if I'm going to do that I might as well just upload the translation files manually instead.

This hasn't been fixed in approaching 8 years now and I wonder if that reflects that bzr support isn't a priority any more and/or that fixing this is hard. Perhaps the translation feature could be changed to work directly from git repos more easily?

[Jelmer Vernooij (jelmer)]

Are the code importers still using Bazaar rather Breezy? If so, this is probably just a matter of moving over to Breezy.

[Colin Watson (cjwatson)]

I believe the remaining blocker to getting lp-codeimport moved to Breezy is finishing the cscvs port. cscvs is extremely old and crufty code, but there are still a few CVS-to-bzr imports on Launchpad and I'd rather not break them if I can avoid it; at present our architecture doesn't allow migrating these separately from other import types. I have most of a port, which isn't really very complicated in itself, but last time I tried I had extreme difficulty getting the test suite to pass so that I could verify that I hadn't broken anything obvious. I guess I'll give it another try at some point.

[Jelmer Vernooij (jelmer)]

On Fri, Feb 25, 2022 at 10:27:01PM -0000, Colin Watson wrote:
> I believe the remaining blocker to getting lp-codeimport moved to Breezy
> is finishing the cscvs port. cscvs is extremely old and crufty code,
> but there are still a few CVS-to-bzr imports on Launchpad and I'd rather
> not break them if I can avoid it; at present our architecture doesn't
> allow migrating these separately from other import types. I have most
> of a port, which isn't really very complicated in itself, but last time
> I tried I had extreme difficulty getting the test suite to pass so that
> I could verify that I hadn't broken anything obvious. I guess I'll give
> it another try at some point.

Is your port and/or the test output available somewhere public?