package-data-downloader hangs forever when attempting to download through cntlm proxy
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cntlm (Debian) |
Fix Released
|
Unknown
|
|||
cntlm (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Precise |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[SRU] The debdiff attached to comment #22 backports cntlm-0.
It fixes the following bugs in Precise:
- package-
- Can not play radio streams any more (LP: #659809)
- error when downloading files >2GB (LP: #1031670)
- cntlm does not work at reboot (LP: #825593)
- cntlm gpg error The following signatures were invalid: NODATA 2 (LP: #257210)
[IMPACT]
When behind a corporate proxy requiring NTLM authentication, users are unable to:
- install packages which download external files, e.g. flashplugin-
- play internet radio streams (worked in Lucid)
- download files larger than 2GB in size
- download and install GPG keys through apt-get and apt-add-repository
[Test Cases]
Cntlm should be correctly configured and network proxy applied system wide as 127.0.0.1 port 3128 for HTTP, HTTPS and FTP, but not Socks.
- package-
Run 'sudo apt-get install flashplugin-
0.91 behaviour: flashplugin-
0.92 behaviour: flashplugin-
- Can not play radio streams any more (LP: #659809)
Open Rhythmbox and add http://
0.91 behaviour: no music is heard and a red No Entry icon appears next to the station.
0.92 behaviour: music starts playing.
- error when downloading files >2GB (LP: #1031670)
Run 'wget http://
0.91 behaviour: the message 'Connection closed at byte 0. Retrying' appears repeatedly.
0.92 behaviour: the download proceeds normally.
- cntlm does not work at reboot (LP: #825593)
Restart the computer with no network cable attached, open Firefox and enter a URL.
0.91 behaviour: the message 'The proxy server is refusing connections' appears.
0.92 behaviour: the message '502 connection timed out. cntlm proxy failed to complete the request' appears.
- cntlm gpg error The following signatures were invalid: NODATA 2 (LP: #257210)
Run 'sudo add-apt-repository ppa:ginggs/ppa', press Enter to continue when promtped.
0.91 behaviour: the message 'gpg: requesting key 08CC41D2 from hkp server keyserver.
0.92 behaviour: the key is downloaded and installed normally.
[Regression Potential]
Minimal: cntlm has no dependants and no dependencies besides libc6.
I am proposing a backport instead of cherry-picking individual patches for Precise because of the difficulties I experienced in trying to cherry-pick r306. I found that the current commits relied on other changes that were not present in the two-year old version 0.91 in Precise. For example, cntlm with r306 broke in subtle ways (certain pages were not rendered correctly) when r281 was not included.
In addition, I found unrelated changes in the commits, for example, more detailed debug logging and dummy checks introduces to suppress compiler warnings, which were difficult to extract.
Lastly, while running the 0.91 version with debugging information on, cntlm would segfault every couple of days while under heavy load (several workstations sharing one cntlm gateway), whereas this did not occur in the 0.92 version from Quantal.
In short, I believe backporting the Quantal version will give us a more stable base, without the increased regression potential of cherry-picking multiple patches onto a two-year old version.
-------
I previously reported this in #983559, but it seems that it is a completely unrelated issue.
1) Ubuntu 12.04 LTS
2) 0.119ubuntu8.4
3) I expected to be able to install packages which need to download external files (e.g. flashplugin-
4) package-
At university, I can only access the outside world through an NTLM proxy, so I direct everything through a cntlm proxy running locally on my machine. Everything used to work as expected, but since the change in Precise to the way that packages download external files, I am unable to install these packages unless I hack the package-
This is definitely not an issue with finding the proxy settings when using sudo -- I have verified that the environment variables are found as expected. I have also reproduced this issue with a minimal python example which I have run as my normal user.
If I monitor the /tmp/ directory after issuing the command to install one of these packages, I can see that a temporary file is created and grows until it reaches the expected size of the file to be downloaded. However, the script does not detect that the file has finished downloading, and hangs forever, without giving any indication of an error, until it is terminated.
Since I am able to download these files without any difficulty using wget, I have patched my script to use wget instead of urllib. When I do this, everything works as expected. I therefore believe that there is a bug in urllib which is triggered by cntlm. I have looked for a known bug which could be responsible, but I haven't been able to find anything yet.
In the meantime, using wget instead of urllib fixes the problem. I don't know if this is considered a tidy solution, but it works.
Changed in cntlm (Debian): | |
status: | Unknown → New |
Changed in cntlm (Ubuntu): | |
importance: | Undecided → Medium |
description: | updated |
description: | updated |
description: | updated |
Changed in cntlm (Debian): | |
status: | New → Fix Released |
I can't seem to attach two attachments to one report, so here's my minimal urllib example. As an aside, setting the proxies from the environment explicitly seems to be completely unnecessary, since urllib should do that by default unless proxies are explicitly turned off.
I tried downloading different things to see if it made any difference. I haven't conducted an exhaustive search, but I have found that I can download the Google homepage, but not a specific image file. I don't know how the upstream NTLM proxy is configured exactly, so this could just be a coincidence.