Buildout -- Software for automating application assembly

downloading through proxy not working

Reported by Huub Bouma on 2009-11-18
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Buildout
Undecided
Unassigned

Bug Description

I'm working on a suse unix system, and I have an environment variable:

http_proxy=http://user:passwd@localhost:11233/

When I do a python ./bootstrap, I get the following error:

Traceback (most recent call last):
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1659, in main
    user_defaults, windows_restart, command)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 182, in __init__
    data['buildout'].copy(), override))
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1333, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1333, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/buildout.py", line 1283, in _open
    path, is_temp = download(filename)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/download.py", line 96, in __call__
    local_path, is_temp = self.download(url, md5sum, path)
  File "/tmp/tmpXwJvtd/zc.buildout-1.4.2-py2.4.egg/zc/buildout/download.py", line 173, in download
    tmp_path, headers = urllib.urlretrieve(url, tmp_path)
  File "/usr/lib64/python2.4/urllib.py", line 85, in urlretrieve
    return _urlopener.retrieve(url, filename, reporthook, data)
  File "/usr/lib64/python2.4/urllib.py", line 218, in retrieve
    fp = self.open(url, data)
  File "/usr/lib64/python2.4/urllib.py", line 186, in open
    return getattr(self, name)(url)
  File "/usr/lib64/python2.4/urllib.py", line 309, in open_http
    h.endheaders()
  File "/usr/lib64/python2.4/httplib.py", line 795, in endheaders
    self._send_output()
  File "/usr/lib64/python2.4/httplib.py", line 676, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.4/httplib.py", line 643, in send
    self.connect()
  File "/usr/lib64/python2.4/httplib.py", line 611, in connect
    socket.SOCK_STREAM):
IOError: [Errno socket error] (-2, 'Name or service not known')

After a little debugging it seems that urllib.urlretrieve seems to be the culprit. It can't handle this kind of proxy setting. I have also tried this in a little standalone python script, and it has the same bad result.

urllib2 handles the proxy setting just fine. So if I change this line:

  tmp_path, headers = urllib.urlretrieve(url, tmp_path)

into this:

  import urllib2
  tmp_sock = urllib2.urlopen(url)
  tmp_file = open(tmp_path, 'w')
  tmp_file.write(tmp_sock.read())
  tmp_file.close()

then all works fine.
Now this code might not be great, but is it possible that urllib2 will be used as a workaround for this proxy issue?

Cheers, Huub

Clayton (claytonc-sousa) wrote :

This error is not detected in machines with python2.4.

The error is detected in zc.buildout 1.4.4 and 1.5.2 with python2.6.

When I use cntlm version 0.35 I dont't detect the problem, but when i use cntlm version > 0.9 the error occurs.

With ntlmaps it's working.

I did what is described above and it worked.

Clayton

Clayton (claytonc-sousa) wrote :

The above tests were done on Ubuntu 11.04 and 12.04.

Clayton

Marius Gedminas (mgedmin) wrote :

This sounds like a bug in the Python standard library. Have you reported it at http://bugs.python.org, or found an existing bug report for this issue? Can you please provide a link to it here?

Huub Bouma (bouma-w20e) wrote :

The problem I think is that I used a proxy with authentication, which is not supported in urllib:

"Proxies which require authentication for use are not currently supported; this is considered an implementation limitation."

http://docs.python.org/2/library/urllib.html

Marius Gedminas (mgedmin) wrote :

Still, the error message from urllib is misleading. Is it trying to resolve a host named 'user:password@localhost' and failing? Why doesn't urllib.urlopen() fail in the same fashion?

Please file a urllib bug at http://bugs.python.org/

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers