[exchange] BeautifulSoup 3.1 is incompatible with xe.com

[Stefano Rivera]

Query: currencies for South Africa
ERROR:scripts.ibid-plugin:Exception occured in Currency processor of lookup plugin
Traceback (most recent call last):
  File "../ibid-plugin-333213/scripts/ibid-plugin", line 81, in <module>
    processor.process(event)
  File "ibid/plugins/__init__.py", line 54, in process
    event = method(event, *match.groups()) or event
  File "/home/stefanor/projects/ibid/misc/ibid/plugins/lookup.py", line 195, in currency
    self._load_currencies()
  File "/home/stefanor/projects/ibid/misc/ibid/plugins/lookup.py", line 176, in _load_currencies
    for tr in soup.find('table', attrs={'class': 'tbl_main'}).table.findAll('tr'):
AttributeError: 'NoneType' object has no attribute 'table'
Response: Excuse me?

ii python-beautifulsoup 3.1.0.1-1 error-tolerant HTML parser for Python

[Stefano Rivera (stefanor)]

Oh, and:
Query: exchange 100 zar to gbp
ERROR:scripts.ibid-plugin:Exception occured in Currency processor of lookup plugin
Traceback (most recent call last):
  File "../ibid-plugin-333213/scripts/ibid-plugin", line 81, in <module>
    processor.process(event)
  File "ibid/plugins/__init__.py", line 54, in process
    event = method(event, *match.groups()) or event
  File "/home/stefanor/projects/ibid/misc/ibid/plugins/lookup.py", line 190, in exchange
    event.addresponse(soup.findAll('span', attrs={'class': 'XEsmall'})[1].contents[0])
IndexError: list index out of range
Response: Excuse me?

[Stefano Rivera (stefanor)]

The blame for this one looks to be squarely on the shoulders of BeautifulSoup >= 3.1:

# Beautiful Soup is now based on HTMLParser rather than SGMLParser, which is gone in Python 3. There's some bad HTML that SGMLParser handled but HTMLParser doesn't, usually to do with attribute values that aren't closed or have brackets inside them:

  <a href="foo</a>, </a><a href="bar">baz</a>
  <a b="<a>">', '<a b="<a>"></a><a>"></a>

A later version of Beautiful Soup will allow you to plug in different parsers to make tradeoffs between speed and the ability to handle bad HTML.