use persistent svn_ra sessions and reconnect on failure
Bug #120992 reported by
Michael Hudson-Doyle
This bug affects 5 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Launchpad CSCVS |
Won't Fix
|
High
|
Unassigned |
Bug Description
Making the initial import of a large repository requires something in the region of 10K connections to the server to all go through successfully. A server doesn't have to be all that flaky to drop just one of these connections, and if it's towards the end, we currently (to the best of my knowledge) have to start over again. If instead we could restart the import at that 95% point (which certainly ought to be possible), we'd be much more likely to be able to complete the import.
I'm looking at you, google code hosting.
Related branches
Changed in launchpad-bazaar: | |
importance: | Undecided → High |
status: | New → Confirmed |
Changed in launchpad-cscvs: | |
assignee: | nobody → rockstar |
Changed in launchpad-cscvs: | |
status: | Confirmed → In Progress |
Changed in launchpad-cscvs: | |
status: | In Progress → Triaged |
Changed in launchpad-cscvs: | |
assignee: | rockstar → nobody |
tags: | added: svn |
To post a comment you must log in.
The svn import code of cscvs should:
* Use the svn_ra API instead of the svn_client API, to use persistent sessions for multiple queries and avoid creating that silly number of connections.
* Be able to create a new session, with increasing retry delays if the server has a hiccup and the session errors out, just as its native cvs pserver client does.
That should adequately address the issue with flaky svn servers, and would also make svn imports much faster and put less load on the community servers we import from.
Handling flaky servers by allowing importd to restart failed imports would probably be less programming work, but it would require more handiwork and I do not think it would be a Good Thing.
I suggest changing the title of this bug to "use persistent svn_ra sessions and reconnect on failure" and retargetting to launchpad-cscvs.