Nothing happens when entering a city name in weather

[kritikal]

When I type in "Montreal" without the double quotes nothing happens in the content manager but I get this error in the console

Traceback (most recent call last):
  File "/home/sergeh/Desktop/entertainer-0.1/src/utils/content_management_dialog.py", line 550, in on_location_find_button_clicked
    self.weather.refresh()
  File "/home/sergeh/Desktop/entertainer-0.1/src/utils/weather.py", line 151, in refresh
    self.find_forecast(self.__location)
  File "/home/sergeh/Desktop/entertainer-0.1/src/utils/weather.py", line 53, in find_forecast
    dom = minidom.parse(urllib.urlopen(url))
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse
    return expatbuilder.parse(file)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse
    result = builder.parseFile(file)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile
    parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 121

tried entering different cities and some just give me a Not found in the content manager (or something like that) So I don't see why it gives that error with Montreal

[Joshua Scotton (joshuascotton)]

This happens on mine as well

[Matt Layman (mblayman)]

This is just speculation, but I'm wondering if the problem has to do with the accent character that is coming from the e in Montreal. Could this be an encoding problem? Maybe the xml parser doesn't know how to handle the character properly.

[kritikal (gamecore)]

actually I didn't use an accent since in english it's spelled without the accent

[Matt Layman (mblayman)]

kritikal, I was actually thinking more internally about the weather module code.

I believe you when said that you entered Montreal without the accent, but it is possible that you're query for "Montreal" to the Google service that Entertainer uses might have come back from Google as Montreal with the accent character (evidence of this can probably be seen by searching for "montreal" in Google, my first result was the google map with the accented city name). It is at that level internally that I was speculating about an encoding problem. So it is most certainly not anything that you did incorrectly.

I wonder if we can test this by trying to search for another common city that would have an accent mark in the Google search result.

[kritikal (gamecore)]

I'm sorry Matt, I misunderstood what you meant.

I find it weird that google returns Montréal but while searching for Quebec (which does have an accent in french) returns the city without the accent.
I'm not currently on my linux system so I can't test it out but here are a few cities with accents.

// France
Nîmes
Orléans
Chambéry

// Quebec
Châteauguay
Gaspé
Saint-Jérôme

I guess if none of them work you'll be able to know whether or not the xml parser is at fault.

[kritikal (gamecore)]

Ok so I tried several cities and most of them worked fine even though google returns them with an accent. Like Chambery for instance, google returns Chambéry. I'll try to look at the source code when I have moment and see if I can give you guys a hand.

[Matt Layman (mblayman)]

kritikal, I could be totally wrong. It was speculation about the problem, and I'm not certain about it. I guess we'll need to look into what an ExpatError is. That might shed some light on the problem.

[Joshua Scotton (joshuascotton)]

This bug should be fixed by my working branch.

The problem is that Google returns xml in Latin-1 encoding when searching for montreal but does not say so. Therefore minidom.parse expects utf-8 and can't handle the special characters.
Hence, the following should be replaced:

        url = WEATHER_LOOKUP_URL + search
        url = url.replace(" ", "%20")
        dom = minidom.parsez(urllib.urlopen(url))

with:
        url = WEATHER_LOOKUP_URL + search
        url = url.replace(" ", "%20")
        data = urllib.urlopen(url).read()
        try:
            dom = minidom.parseString(data.encode( "utf-8" ))
        except UnicodeDecodeError:
            data = data.decode("Latin-1")
            dom = minidom.parseString(data.encode( "utf-8" ))

kritikal could you check out my branch and confirm the fix works please?

[kritikal (gamecore)]

confirmed, works like a charm Joshua.