urllib2.urlopen fails to parse the URI when user:password is specified but no port - update-manager fails to download changelog

[Zygmunt Krynicki]

Binary package hint: update-manager

update-manager encountered unhandled exception when attempting to download changelog for the ubuntu-private-fonts package. This package comes from a private ppa. Private PPAs have different URL entry with user:pass@... that confuses some of update-manager code.

Unhandled exception in thread started by <bound method MyCache.get_news_and_changelog of <UpdateManager.Core.MyCache.MyCache object at 0x2955590>>
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/UpdateManager/Core/MyCache.py", line 247, in get_news_and_changelog
    self.get_changelog(name)
  File "/usr/lib/python2.6/dist-packages/UpdateManager/Core/MyCache.py", line 271, in get_changelog
    changelog = self._get_changelog_or_news(name, "changelog", False, changelogs_uri)
  File "/usr/lib/python2.6/dist-packages/UpdateManager/Core/MyCache.py", line 193, in _get_changelog_or_news
    changelog = urllib2.urlopen(uri)
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1169, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1107, in do_open
    h = http_class(host, timeout=req.timeout) # will parse host:port
  File "/usr/lib/python2.6/httplib.py", line 1101, in __init__
    HTTPConnection.__init__(self, host, port, strict, timeout)
  File "/usr/lib/python2.6/httplib.py", line 657, in __init__
    self._set_hostport(host, port)
  File "/usr/lib/python2.6/httplib.py", line 682, in _set_hostport
    raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '<email address hidden>'

Note, above I replaced the actual password with 'XXX' sequence

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: update-manager 1:0.134.10
ProcVersionSignature: Ubuntu 2.6.32-24.42-generic 2.6.32.15+drm33.5
Uname: Linux 2.6.32-24-generic x86_64
NonfreeKernelModules: nvidia
Architecture: amd64
Date: Thu Aug 26 17:53:24 2010
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release amd64 (20100429)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, user)
 LANG=pl_PL.utf8
 SHELL=/bin/bash
SourcePackage: update-manager

[Jean-Baptiste Lallement (jibel)]

Thanks for your report.

This is a problem with python failing to parse a valid scheme according to RFC 3986 (http://tools.ietf.org/html/rfc3986)
To reproduce:
$ python
Python 2.6.6 (r266:84292, Aug 24 2010, 21:47:18)
[GCC 4.4.5 20100816 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> urllib2.urlopen('http://username:password@domain/path?query_string#anchor')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1170, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1116, in do_open
    h = http_class(host, timeout=req.timeout) # will parse host:port
  File "/usr/lib/python2.6/httplib.py", line 661, in __init__
    self._set_hostport(host, port)
  File "/usr/lib/python2.6/httplib.py", line 686, in _set_hostport
    raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: 'password@domain'

I guess that the parser cuts the scheme at the first ':' to find the port but it's the authority part.
Interestingly it doesn't fail if you provide the port:
>>> urllib2.urlopen('http://username:password@domain:80/path?query_string#anchor')
[...]
urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

I'm adding a python task and confirming the update-manager task because it should correctly handle the exception.

[Matthias Klose (doko)]

python3 has urllib.parse which seems to do what you want. Won't be fixed for python2.x. Tempted to close as won't fix.

[Zygmunt Krynicki (zyga)]

Doko: while fixing this in python is not required it should _really_ be fixed in update manager.

[Felix Möller (felix-derklecks)]

I just hit this issue with Oneiric. Would be really cool to have this fixed.

[dino99 (9d9)]

This version is no more supported