CherryPy fails to download even if *external* is set to True

Bug #385387 reported by Sridhar Ratnakumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
PyPI Mirror
Fix Committed
Undecided
Unassigned

Bug Description

Other than CherryPy-2.1.1.tar.gz and CherryPy-2.2.0beta.tar.gz, all other versions (including 3.1.x) are missing. This makes it impossible to install, say, TurboGears from a pypi mirror.

> pypimirror -I -c etc/pypimirror.cfg
/home/sridharr/as/pypm/eggs/z3c.pypimirror-1.0.14-py2.6.egg/z3c/pypimirror/mirror.py:28: DeprecationWarning: the sets module is deprecated
  import sets
/home/sridharr/as/pypm/eggs/z3c.pypimirror-1.0.14-py2.6.egg/z3c/pypimirror/mirror.py:32: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  from md5 import md5
2009-06-09 15:23:46,492 DEBUG Processing package CherryPy
http://www.cherrypy.org
http://download.cherrypy.org/cherrypy/3.1.0rc1/
http://trac.cherrypy.org/cgi-bin/trac.cgi/wiki/CherryPyDownload
2009-06-09 15:23:49,628 WARNING Unload downloading http://trac.cherrypy.org/cgi-bin/trac.cgi/wiki/CherryPyDownload ()
http://download.cherrypy.org/cherrypy/3.1beta/
2009-06-09 15:23:49,825 WARNING Unload downloading http://download.cherrypy.org/cherrypy/3.1beta/ ()
http://download.cherrypy.org/cherrypy/3.0.2/
http://download.cherrypy.org/cherrypy/3.0.3/
http://download.cherrypy.org/cherrypy/2.3.0/
http://download.cherrypy.org/cherrypy/3.0.1/
http://download.cherrypy.org/cherrypy/3.1.0beta3/
http://download.cherrypy.org/cherrypy/3.0.0/
http://download.cherrypy.org/cherrypy/3.1.2/
http://download.cherrypy.org/cherrypy/2.2.1/
http://download.cherrypy.org/cherrypy/3.1.0/
http://download.cherrypy.org/cherrypy/3.0RC1/
2009-06-09 15:23:51,904 WARNING Unload downloading http://download.cherrypy.org/cherrypy/3.0RC1/ ()
http://download.cherrypy.org/cherrypy/3.1.1/
http://sourceforge.net/project/showfiles.php?group_id=56099
2009-06-09 15:23:52,726 DEBUG Package not available: HTML parse error: bad end tag: u"</sc'+'ript>", at line 530, column 38
2009-06-09 15:25:59,661 DEBUG Statistics
2009-06-09 15:25:59,661 DEBUG ----------
2009-06-09 15:25:59,662 DEBUG Found (cached): 0
2009-06-09 15:25:59,662 DEBUG Stored (downloaded): 0
2009-06-09 15:25:59,662 DEBUG Not found (404): 1
2009-06-09 15:25:59,662 DEBUG Invalid packages: 0
2009-06-09 15:25:59,663 DEBUG Invalid URLs: 0
2009-06-09 15:25:59,663 DEBUG Runtime: 2m13s
(pypm:^/trunk:r302*) ~/as/pypm
> ls /net/nas/data/languages/pypm/pypimirror/CherryPy/
CherryPy-2.1.1.tar.gz CherryPy-2.2.0beta.tar.gz index.html
>

Revision history for this message
Sridhar Ratnakumar (srid) wrote :

from the config file,

external_links = True
follow_external_index_pages = True

Revision history for this message
Sridhar Ratnakumar (srid) wrote :

Further analysis uncovered the root cause of this error:

The HTML of http://sourceforge.net/project/showfiles.php?group_id=56099 contains the following:

   if(typeof(urchinTracker)!='function')document.write('<sc'+'ript src="'+

BeautifulSoup consideres the JavaScript string '<sc'+'ript src="' as a HTML tag and fails. So much for a permissive parser?

Revision history for this message
Andreas Jung (ajung) wrote : Re: [Pypi-mirror] [Bug 385387] [NEW] CherryPy fails to download even if *external* is set to True

On 10.06.09 00:28, Sridhar Ratnakumar wrote:
> Public bug reported:
>
> Other than CherryPy-2.1.1.tar.gz and CherryPy-2.2.0beta.tar.gz, all
> other versions (including 3.1.x) are missing. This makes it impossible
> to install, say, TurboGears from a pypi mirror.
>
>
>> pypimirror -I -c etc/pypimirror.cfg
>>
> /home/sridharr/as/pypm/eggs/z3c.pypimirror-1.0.14-py2.6.egg/z3c/pypimirror/mirror.py:28: DeprecationWarning: the sets module is deprecated
> import sets
> /home/sridharr/as/pypm/eggs/z3c.pypimirror-1.0.14-py2.6.egg/z3c/pypimirror/mirror.py:32: DeprecationWarning: the md5 module is deprecated; use hashlib instead
> from md5 import md5
> 2009-06-09 15:23:46,492 DEBUG Processing package CherryPy
> http://www.cherrypy.org
> http://download.cherrypy.org/cherrypy/3.1.0rc1/
> http://trac.cherrypy.org/cgi-bin/trac.cgi/wiki/CherryPyDownload
> 2009-06-09 15:23:49,628 WARNING Unload downloading http://trac.cherrypy.org/cgi-bin/trac.cgi/wiki/CherryPyDownload ()
> http://download.cherrypy.org/cherrypy/3.1beta/
> 2009-06-09 15:23:49,825 WARNING Unload downloading http://download.cherrypy.org/cherrypy/3.1beta/ ()
> http://download.cherrypy.org/cherrypy/3.0.2/
> http://download.cherrypy.org/cherrypy/3.0.3/
> http://download.cherrypy.org/cherrypy/2.3.0/
> http://download.cherrypy.org/cherrypy/3.0.1/
> http://download.cherrypy.org/cherrypy/3.1.0beta3/
> http://download.cherrypy.org/cherrypy/3.0.0/
> http://download.cherrypy.org/cherrypy/3.1.2/
> http://download.cherrypy.org/cherrypy/2.2.1/
> http://download.cherrypy.org/cherrypy/3.1.0/
> http://download.cherrypy.org/cherrypy/3.0RC1/
> 2009-06-09 15:23:51,904 WARNING Unload downloading http://download.cherrypy.org/cherrypy/3.0RC1/ ()
> http://download.cherrypy.org/cherrypy/3.1.1/
> http://sourceforge.net/project/showfiles.php?group_id=56099
> 2009-06-09 15:23:52,726 DEBUG Package not available: HTML parse error: bad end tag: u"</sc'+'ript>", at line 530, column 38
> 2009-06-09 15:25:59,661 DEBUG Statistics
> 2009-06-09 15:25:59,661 DEBUG ----------
> 2009-06-09 15:25:59,662 DEBUG Found (cached): 0
> 2009-06-09 15:25:59,662 DEBUG Stored (downloaded): 0
> 2009-06-09 15:25:59,662 DEBUG Not found (404): 1
> 2009-06-09 15:25:59,662 DEBUG Invalid packages: 0
> 2009-06-09 15:25:59,663 DEBUG Invalid URLs: 0
> 2009-06-09 15:25:59,663 DEBUG Runtime: 2m13s
> (pypm:^/trunk:r302*) ~/as/pypm
>
>> ls /net/nas/data/languages/pypm/pypimirror/CherryPy/
>>
> CherryPy-2.1.1.tar.gz CherryPy-2.2.0beta.tar.gz index.html
>

Which BeautifoulSoup version are you running? I think this is caused
by some issue within the newest BeautifulSoup version. Try to install
BeautifulSoup 3.0.7 (instance of the newest one).

Andreas

Revision history for this message
Sridhar Ratnakumar (srid) wrote :

On 09-06-09 10:26 PM, Andreas Jung wrote:
> Which BeautifoulSoup version are you running?

BeautifulSoup-3.1.0.1-py2.6.egg

> I think this is caused
> by some issue within the newest BeautifulSoup version.

Indeed. http://www.crummy.com/software/BeautifulSoup/3.1-problems.html

 > Try to install
> BeautifulSoup 3.0.7 (instance of the newest one).

I will.

Revision history for this message
Sridhar Ratnakumar (srid) wrote :

Ok, that fixed the problem. Should install_requires be locked to <3.1.0 in setup.py, then?

Revision history for this message
Andreas Jung (ajung) wrote :

Fixed on the trunk

Changed in pypi-mirror:
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers