git-to-git import from gerrit with many changes "fails"

Bug #1642699 reported by Michael Hudson-Doyle on 2016-11-17
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Launchpad itself
High
Colin Watson

Bug Description

https://code.launchpad.net/~gophers/golang/+git/golang is an import of https://go.googlesource.com/go, which is a gerrit instance with many thousands of changes. The log: (http://launchpadlibrarian.net/293800925/gophers-golang-+git-golang.log) shows launchpad importing each of these changes as a new branch, and then failing to send them to lp with "2016-11-17 11:37:11 INFO fatal: cannot exec 'send-pack': Argument list too long".

The Go team had similar problems with their syncing-to-github thing (https://github.com/golang/go/issues/16388), and afaict ended up hacking something up to push refs in batches (https://github.com/golang/build/commit/f2d63a86dd9cd149ab59a47b7ade41bb32d09b56) although comments also suggest that switching to git+ssh rather than git+https would have been enough (which I guess wouldn't be the easiest change ever for you to implement).

It's worth noting that the import of master succeeds, which is what I actually care about for now :)

summary: - git-to-git import from gerrit with many changes fails
+ git-to-git import from gerrit with many changes "fails"
description: updated
Colin Watson (cjwatson) wrote :

I've requested what should hopefully be the necessary set of git cherry-picks: https://rt.admin.canonical.com/Ticket/Display.html?id=98938

Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
importance: Undecided → High
status: New → In Progress
Colin Watson (cjwatson) wrote :

(It's not clear whether we'll need to use SSH as well. We'll see.)

Colin Watson (cjwatson) wrote :

We've now upgraded all the importds to xenial, so the cherry-picks are no longer relevant. However, it's now failing like this instead:

  http://launchpadlibrarian.net/319074739/gophers-golang-+git-golang.log

One thing that could be the cause is that we don't enable progress output on the push, so that if it takes a long time the worker monitor is going to time it out even though it's still making progress. It may be worth fixing that.

I have a feeling there may be something else going on, though, as I can't find a record of the hypothetical long push request in the git.launchpad.net logs.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.