AccessToken.from_string() crashes on python3

[Zygmunt Krynicki]

[Impact]

 * Unable to renew/relogin with a new oauth token, using python3 scripts
 * python2 APIs work (because python2 does not care about str vs bytes)

[Test Case]

 * Attempt login_with() using python3 and an expired/invalid existing token
 * Relogin should be successful, without crashing scripts

[Regression Potential]

 * No change in behaviour on python2 (which is what used by reverse dependencies)
 * this is a backport to fix python3 behaviour, for all the newly ported maintainance scripts in python3 that prefer to run on "stable" systems

[Other Info]

 * Well tested in later releases, and with users running from trunk.
 * I believe this still is not published in the cheeseshop.

[Original bug report]

Trying to use Launchpad.login_with() on python3, after fixing the earlier bug yields this issue:

Traceback (most recent call last):
  File "./foo.py", line 31, in <module>
    lb = launchpad.projects[project]
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 1001, in __getitem__
    shim_resource._ensure_representation()
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 382, in _ensure_representation
    representation = self._root._browser.get(self._wadl_resource)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 436, in get
    response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 387, in _request
    str(url), method=method, body=data, headers=headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 357, in _request_and_retry
    url, method=method, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1291, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 132, in _request
    return self.retry_on_bad_token(response, content, *args)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 144, in retry_on_bad_token
    self.launchpad.credentials, self.launchpad.credential_store)
  File "/usr/lib/python3/dist-packages/launchpadlib/credentials.py", line 545, in __call__
    self.make_end_user_authorize_token(credentials, request_token_string)
  File "/usr/lib/python3/dist-packages/launchpadlib/credentials.py", line 658, in make_end_user_authorize_token
    self.web_root)
  File "/usr/lib/python3/dist-packages/launchpadlib/credentials.py", line 233, in exchange_request_token_for_access_token
    self.access_token = AccessToken.from_string(content)
  File "/usr/lib/python3/dist-packages/launchpadlib/credentials.py", line 251, in from_string
    key = params['oauth_token']
KeyError: 'oauth_token'

The problem is that params has b'oauth_token', not 'oauth_token', which matters on python3

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-launchpadlib - 1.10.4-1

---------------
python-launchpadlib (1.10.4-1) unstable; urgency=medium

  [ Ondřej Nový ]
  * Fixed VCS URL (https)

  [ Dimitri John Ledkov ]
  * New upstream release
  * Drop deprecated warnings patch, applied upstream
  * Cherrypick upstream patch to fix LP: #1471927

 -- Dimitri John Ledkov <email address hidden> Sat, 13 Aug 2016 04:18:24 +0100

[Brian Murray (brian-murray)]

Thanks for uploading the fix for this bug report to -proposed. However, when reviewing the package in -proposed and the details of this bug report I noticed that the bug description is missing information required for the SRU process. You can find full details at http://wiki.ubuntu.com/StableReleaseUpdates#Procedure but essentially this bug is missing some of the following: a statement of impact, a test case and details regarding the regression potential. Thanks in advance!

[Brian Murray (brian-murray)]

Hello Zygmunt, or anyone else affected,

Accepted python-launchpadlib into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-launchpadlib/1.10.3-3ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

Fixed in launchpadlib 1.10.5.

[Dimitri John Ledkov (xnox)]

Attempted to login using python3 on existing client and got the traceback:

/usr/lib/python3/dist-packages/launchpadlib/credentials.py in from_string(cls, query_string)
    249 """Create and return a new `AccessToken` from the given string."""
    250 params = cgi.parse_qs(query_string, keep_blank_values=False)
--> 251 key = params['oauth_token']
    252 assert len(key) == 1, (
    253 "Query string must have exactly one oauth_token.")

KeyError: 'oauth_token'

Upgraded to:
Get:1 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 python3-launchpadlib all 1.10.3-3ubuntu0.1 [36.5 kB]

And that traceback is gone.

However, keyring integration is still broken:

In [1]: from launchpadlib.launchpad import Launchpad

In [2]: lp = Launchpad.login_with('just testing', 'production', version='devel')
The authorization page:
 (https://launchpad.net/+authorize-token?oauth_token=G81S4r3H7ppz29Zr4z6r&allow_permission=DESKTOP_INTEGRATION)
should be opening in your browser. Use your browser to authorize
this program to access Launchpad on your behalf.
Waiting to hear from Launchpad about your decision...
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-2-469be45530f4> in <module>()
----> 1 lp = Launchpad.login_with('just testing', 'production', version='devel')

/usr/lib/python3/dist-packages/launchpadlib/launchpad.py in login_with(cls, application_name, service_root, launchpadlib_dir, timeout, proxy_info, authorization_engine, allow_access_levels, max_failed_attempts, credentials_file, version, consumer_name, credential_save_failed, credential_store)
    542 cache_path, timeout, proxy_info, authorization_engine,
    543 allow_access_levels, credential_store,
--> 544 credential_save_failed, version)
    545
    546 @classmethod

/usr/lib/python3/dist-packages/launchpadlib/launchpad.py in _authorize_token_and_login(cls, consumer_name, service_root, cache, timeout, proxy_info, authorization_engine, allow_access_levels, credential_store, credential_save_failed, version)
    349 # They're not there. Acquire new credentials using the
    350 # authorization engine.
--> 351 credentials = authorization_engine(credentials, credential_store)
    352 else:
    353 # We acquired credentials. But, the application name

/usr/lib/python3/dist-packages/launchpadlib/credentials.py in __call__(self, credentials, credential_store)
    550 return None
    551 # save() invokes the callback on failure.
--> 552 credential_store.save(credentials, self.unique_consumer_id)
    553 return credentials
    554

/usr/lib/python3/dist-packages/launchpadlib/credentials.py in save(self, credentials, unique_consumer_id)
    305 except Exception as e:
    306 if self.credential_save_failed is None:
--> 307 raise e
    308 self.credential_save_failed()
    309 return credentials

/usr/lib/python3/dist-packages/launchpadlib/credentials.py i...

[Adam Conrad (adconrad)]

The verification of the Stable Release Update for python-launchpadlib has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-launchpadlib - 1.10.3-3ubuntu0.1

---------------
python-launchpadlib (1.10.3-3ubuntu0.1) xenial; urgency=medium

  * Cherrypick python3 crash fixes:
    - _bad_oauth_token (LP: #1471894)
    - AccessToken.from_string() (LP: #1471927)

 -- Dimitri John Ledkov <email address hidden> Thu, 22 Dec 2016 15:05:54 +0000