mail applet's gmail login fails when special characters used

[intelx86]

When using Gmail login option (not POP or IMAP), password with special characters fail to login due to the HTTP Basic Authentication method used by feedparser (https://username:<email address hidden>/gmail/feed/atom/unread/). The failure message is misleading (There seem to be problems with our connection to your account. Your best bet is probably to log out and try again.) as it is intended to invoke gnomekeyring prior to login.

Instead you could use an HTTPBasicAuthHandler as described in feedparser documentation (http://www.feedparser.org/docs/http-authentication.html).

To resolve the problem just open mail.py, located at /usr/share/avant-window-navigator/applets/mail in Ubuntu Natty, and find and replace the following lines as described bellow.

==============
 OPEN mail.py
==============

======
 FIND
======
            f = feedparser.parse(\
                "https://%s:%<email address hidden>/gmail/feed/atom/unread/" \
                            % (self.data["username"], self.data['password']))

==============
 REPLACE WITH
==============
            import urllib2
            auth = urllib2.HTTPBasicAuthHandler()
            auth.add_password('New mail feed', 'google.com', \
                self.data["username"], self.data['password'])

            f = feedparser.parse(\
                'https://mail.google.com/gmail/feed/atom/unread/', \
                handlers=[auth])

[intelx86 (intelx)]

I've rewritten the above script as it stopped working after a while.

==============
 OPEN /usr/share/avant-window-navigator/applets/mail/mail.py
==============

======
 FIND
======
            f = feedparser.parse(\
                "https://%s:%<email address hidden>/gmail/feed/atom/unread/" \
                            % (self.data["username"], self.data['password']))

            if "bozo_exception" in f.keys():
                raise LoginError(_("There seem to be problems with our \
connection to your account. Your best bet is probably \
to log out and try again."))
            # Hehe, Google is funny. Bozo exception

==============
 REPLACE WITH
==============
            import urllib2
            password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
            password_mgr.add_password(None, 'https://mail.google.com', \
            self.data["username"], self.data["password"])

            auth = urllib2.HTTPBasicAuthHandler(password_mgr)
            opener = urllib2.build_opener(auth)

            try:
                src = opener.open('https://mail.google.com/gmail/feed/atom')
            except:
                raise LoginError(_("There seem to be problems with our \
connection to your account. Your best bet is probably \
to log out and try again."))

            feed = src.read()
            f = feedparser.parse(feed)

[Gabor Karsay (gabor-karsay)]

Thank you, which special character(s) don't work (to reproduce the bug)?

[intelx86 (intelx)]

Let's say slash character (/) but I assume that @ and : wouldn't work.

[onox (onox)]

@Intel X86:

1) It seems HTTPBasicAuthHandler has a procedure add_password, so you don't need to manually construct a HTTPPasswordMgrWithDefaultRealm

2) Change except: to except IOError, e:

3) You can inline feed = src.read()

If you make these changes and attach a proper .patch file to this bug report, I'll commit it.

[onox (onox)]

Could you also test whether it's possible to use HTTPDigestAuthHandler instead of HTTPBasicAuthHandler? It seems to be more secure according to the feedparser documentation.

[intelx86 (intelx)]

HTTPDigestAuthHandler could not be used as Gmail feed does not use digest authentication.
Digest authentication constists of the realm's (typically a description of the computer or system being accessed) and the password's MD5 (or other algorithm) digest. To check if a password protected page uses Basic Auth or Digest Auth simply:

>>> import feedparser
>>> d = feedparser.parse('http://feedparser.org/docs/examples/digest_auth.xml')
>>> d.status
401
>>> d.headers['www-authenticate']
'Basic realm="Use test/basic"'
>>> d = feedparser.parse('http://feedparser.org/docs/examples/digest_auth.xml')
>>> d.status
401
>>> d.headers['www-authenticate']
'Digest realm="DigestTest",
 nonce="+LV/uLLdAwA=5d77397291261b9ef256b034e19bcb94f5b7992a",
 algorithm=MD5,
 qop="auth"'

As you see from the example above, the authentication method is stated in the response headers.
For more information on Digest Authentication look here (http://en.wikipedia.org/wiki/Digest_access_authentication).

I don't use HTTPBasicAuthHandler because it requires the specific realm ('New mail feed' for Gmail) otherwise it won't get authorized.
The example shown at urllib2 manual (http://docs.python.org/library/urllib2.html) needs the specific realm or it fails.

import urllib2
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm='PDQ Application',
                          uri='https://mahler:8092/site-updates.py',
                          user='klem',
                          passwd='kadidd!ehopper')
opener = urllib2.build_opener(auth_handler)

If we use that method we need first to acquire the right realm and then proceed to the login.
>>> import feedparser, re
>>> d = feedparser.parse('http://feedparser.org/docs/examples/basic_auth.xml')
>>> d.status
401
>>> d.headers['www-authenticate']
'Basic realm="BasicTest"'
>>> realm = re.findall('realm="([^"]*)"', d.headers['www-authenticate'])[0]
>>> realm
'BasicTest'

I applied the suggested changes.

[onox (onox)]

Please move import urllib2 to _below_ import socket (which is on line 22) and provide the patch in the format like bzr diff does. Otherwise than that the patch is fine.

[onox (onox)]

You can do bzr checkout lp:awn-extras or bzr branch lp:awn-extras. Modify mail.py and then do bzr diff applets/maintained/mail/mail.py

[intelx86 (intelx)]

Hope it's ok!

[onox (onox)]

Fix committed in revision 1516. Thanks for your contribution, Intel X86!

[Nicholas Bodley (nbodley)]

This problem seems to be more general than just with Ubuntu (and its derivative, Mint Linux). Nevertheless, it's very helpful to see this, because my Verizon FiOS router's access-password creator utility will accept all sorts of special characters, and create a new password from what you type, but when you try to use it, you're out of luck; it's rejected. I have been thinking that the developers were, well, Not Nice; seems that I'm wrong.

Well, I finally "caved in" and made a straight alphanumeric-only password.

Launchpad, using Mint Linux 11, showed this problem, as well.

Then, some time ago, I bought a Craig Android CMP738b tablet (quite low cost, lovely display, often-infuriating firmware) and very recently tried to use Google Play. I tried to sign in with my Gmail account, and it said username and password don't match, a rather-bad error message (misleading). Google's advice on creating passwords is well-written, but says nothing much about special characters. It just rejected a password containing alphanumerics and one [&] character.

From the standpoint of a semi-technical long-term* non-developer computer user, it appears just now that, often, software that creates new passwords tends to accept all sorts of characters, but when one has defined the new password and attempts to use it, the password evaluator rejects it. Worse, the password creator apparently sometimes can recommend non-alphanumeric characters, thereby permitting the user to create a useless password.
* 1962: Attempted heuristic assembler programming on a Philco 2000, but the DORMS instruction didn't work, and owner used an high-level language (no Booleans) exclusively.

I apologize if my post is somewhat inappropriate. It's based on a limited number of OSes and limited variety of hardware, but to this user, it has been a serious horror show.

Regards,
[nb]