Launchpad itself

Fails to detect Mercurial repository if remote does not have valid http 404

Bug #674581 reported by Yuv
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Bazaar Hg Plugin
Fix Released
High
Jelmer Vernooij
Bazaar Hg Plugin 0.1.1
Launchpad itself
Fix Released
Medium
Jelmer Vernooij

Bug Description

Hello,

I am not sure if this is a bug or a question but I have not found a way to file a question.

I am trying to update the code imports for two projects:
https://launchpad.net/hugin
https://launchpad.net/enblend

Both projects are hosted at SourceForge. Both used SVN and the SVN mirroring into Launchpad worked well.

Both projects changed from SVN to Mercurial. I tried to import their respective Mercurial repositories but they both seem to fail with a timeout:

https://code.launchpad.net/~hugin/hugin/trunk
https://code.launchpad.net/~hugin/enblend/trunk

What did I do wrong? I entered the following two URLs, which are good for checking out (read-only) with any Hg client

http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin
http://enblend.hg.sourceforge.net:8000/hgroot/enblend/enblend

I tried to google for documentation of what I intend to do (there must be other projects on SF that use Hg and mirror on LP?) but have not found anything. I've tried to find documentations of Hg imports in general but found no solution to my problem.

Hence my request. Can you help?

Thank you very much for taking your time to read/reply to my questions.

Yuv

Related branches

lp:bzr-hg
Launchpad code reviewers from Canonical: Pending requested
Diff: 729 lines (+257/-65)
17 files modified
Makefile (+3/-11)
NEWS (+21/-1)
README (+12/-0)
__init__.py (+48/-7)
branch.py (+19/-10)
commands.py (+55/-0)
fetch.py (+6/-4)
idmap.py (+2/-2)
info.py (+2/-2)
mapping.py (+8/-4)
parsers.py (+8/-5)
repository.py (+6/-1)
setup.py (+1/-1)
tests/__init__.py (+9/-2)
tests/test_dir.py (+42/-0)
tests/test_mapping.py (+1/-1)
tests/test_parsers.py (+14/-14)
lp:~jelmer/launchpad/newer-bzr
Gavin Panella (community): Approve
Diff: 592 lines (+136/-125)
12 files modified
lib/canonical/config/schema-lazr.conf (+2/-0)
lib/lp/code/bzr.py (+6/-0)
lib/lp/code/interfaces/tests/test_branch.py (+2/-2)
lib/lp/codehosting/codeimport/tests/servers.py (+40/-27)
lib/lp/codehosting/codeimport/tests/test_worker.py (+4/-15)
lib/lp/codehosting/codeimport/worker.py (+41/-42)
lib/lp/codehosting/puller/tests/__init__.py (+4/-2)
lib/lp/codehosting/puller/worker.py (+4/-3)
lib/lp/codehosting/tests/test_bzrutils.py (+3/-3)
utilities/sourcedeps.cache (+23/-23)
utilities/sourcedeps.conf (+6/-6)
versions.cfg (+1/-2)
Curtis Hovey (sinzui)
affects: launchpad → launchpad-code
Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 674581] [NEW] Sourceforge Mercurial Import Fails

Hi,

I see there is an import branch
<https://code.launchpad.net/~hugin/enblend/tip> and it is failing as
shown in <http://launchpadlibrarian.net/59116794/hugin-enblend-tip.log>,
specifically

urllib2.URLError: <urlopen error [Errno 111] Connection refused>

I don't know if this indicates that sourceforge is blocking
connections, or if there is an outgoing ip rule blocking the
connection.

--
Martin

Revision history for this message
Yuv (yuv) wrote :

Hi,

> I see there is an import branch
> <https://code.launchpad.net/~hugin/enblend/tip> and it is failing as
> shown in <http://launchpadlibrarian.net/59116794/hugin-enblend-tip.log>,

yes, I tried a different URL than
<https://code.launchpad.net/~hugin/enblend/trunk> to check if it was me
entering the wrong URL.

> urllib2.URLError: <urlopen error [Errno 111] Connection refused>

both fail at the same point, so it is not (only) me entering the wrong URL.

> I don't know if this indicates that sourceforge is blocking
> connections, or if there is an outgoing ip rule blocking the
> connection.

Question to the experts: has Hg import from SourceForge worked in the past?

If yes, here is a possible avenue to explore in fixing the error.

At some point in the not so distant past (IIRC less than a year ago) SF
started allowing multiple Hg repositories. This resulted in a change of the
URL to the main repository

from <http://enblend.hg.sourceforge.net:8000/hgroot/enblend>

to <http://enblend.hg.sourceforge.net:8000/hgroot/enblend/enblend>

so that we can host a second repository, e.g. at

to <http://enblend.hg.sourceforge.net:8000/hgroot/enblend/enblend-web>

Maybe the import script is trying to access according to the old schema?

I'd like to help analyze / debug the error but I know too little about
Launchpad. Can you point me to a place where I can browse LP's source code?
and within that place to the specific file that fails at the moment?

thanks

Revision history for this message
Tim Penhey (thumper) wrote : Re: Sourceforge Mercurial Import Fails

hg clone http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin works fine.

bzr branch http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin bzr-hugin
  fails with the following:

http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin/ is permanently redirected to http+urllib://hugin.hg.sourceforge.net/hgweb/hugin/hugin/
bzr: ERROR: Unknown bzrdir format: '<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE html PUBLIC "-//W3C

and then the rest of the html file.

Changed in launchpad-code:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 674581] Re: Sourceforge Mercurial Import Fails

On 18 November 2010 14:07, Tim Penhey <email address hidden> wrote:
> hg clone http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin works
> fine.
>
> bzr branch  http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin bzr-hugin
>  fails with the following:
>
> http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin/ is permanently redirected to http+urllib://hugin.hg.sourceforge.net/hgweb/hugin/hugin/
> bzr: ERROR: Unknown bzrdir format: '<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE html PUBLIC "-//W3C
>
> and then the rest of the html file.

That seems to be a different bug to what the user reported, which I
suspect is caused by a firewall either at the Canonical end or the
SourceForge end. However, the bug Tim points out probably means even
if the firewall was fixed, the import would not proceed.

--
Martin

Jelmer Vernooij (jelmer)
Changed in bzr-hg:
importance: Undecided → High
status: New → Triaged
Revision history for this message
Jelmer Vernooij (jelmer) wrote : Re: Sourceforge Mercurial Import Fails

These appear to be two separate issues.

Unfortunately the local failure to clone that branch is caused by the fact that:

$ wget http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin/.bzr/branch-format

doesn't error with a 404 but instead returns a HTML page.

The simplest workaround for this would be to probe for Mercurial repositories before we probe for Bazaar repositories, like we do with Subversion branches over HTTP. This has a clear disadvantage though - an extra roundtrip is added for accessing a native Bazaar repository over HTTP when bzr-hg is installed.

We could also add a workaround in bzr and treat any branch that has a .bzr/branch-format file that contains more than one newline as not being a bzr branch. That's rather ugly, but would mean we can avoid probing for Mercurial repositories first.

Revision history for this message
Yuv (yuv) wrote :

The one issue seem to affect SVN imports too. See
http://launchpadlibrarian.net/59306469/hugin-panotools-trunk.log

Revision history for this message
Jelmer Vernooij (jelmer) wrote : Re: [Bug 674581] Re: Sourceforge Mercurial Import Fails

On Sat, 2010-11-20 at 18:24 +0000, Yuv wrote:
> The one issue seem to affect SVN imports too. See
> http://launchpadlibrarian.net/59306469/hugin-panotools-trunk.log
 It almost looks like we've been firewalled by sourceforge.

Cheers,

Jelmer

Jelmer Vernooij (jelmer)
Changed in bzr-hg:
status: Triaged → Fix Committed
assignee: nobody → Jelmer Vernooij (jelmer)
Changed in launchpad:
assignee: nobody → Jelmer Vernooij (jelmer)
status: Triaged → In Progress
Jelmer Vernooij (jelmer)
Changed in bzr-hg:
milestone: none → 0.2
Diogo Matsubara (matsubara)
tags: added: sourceforge
removed: souorceforge
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

Fixed in stable r13158 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/13158>.

tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
William Grant (wgrant)
tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
Revision history for this message
Yuv (yuv) wrote : Re: Sourceforge Mercurial Import Fails

this seems to be still failing - I have recently received emails that the import for hugin is requested and failed

https://code.launchpad.net/~hugin/hugin/trunk

i tried to look at the interface to set up a new import but it says "This foreign branch URL is already specified for the imported branch" and does not let me continue.

On https://code.launchpad.net/hugin/default/+setbranch I am missing a field to set the branch of the Mercurial repo I want to mirror, similar to the "module" field for CVS.

If you look at http://hugin.hg.sourceforge.net:8000/hgroot/hugin/hugin you will see listed our many development branches, release branches, and the default branch which we use for integration and bug fixes.

Changed in launchpad:
status: Fix Released → New
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Hi Yuv,

This bug report seemed to cover two different bugs; one has been fixed, the other is still open. I've opened a new bug about the connection refused issue.

summary: - Sourceforge Mercurial Import Fails
+ Fails to detect Mercurial repository if remote does not have valid http
+ 404
Changed in launchpad:
status: New → Fix Released
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

bug 812033

Jelmer Vernooij (jelmer)
Changed in bzr-hg:
status: Fix Committed → Fix Released
milestone: 0.2 → 0.1.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.