Gmail webapp account renewal not successful

[Pat McGowan]

MX4 running rc-proposed 167

I ran the gmail webapp which I have done in the past.
 I had a google account configured, and gmail was permitted to use it

Open gmail, get directed to login with ubuntu one
login into ubuntu one
redirect to a webbrowser instance with my google account home displayed and no option for mail
Close the browser
the ubuntu one screen is still up but has no navigation available, so close that too
close the gmail webapp

I repeated that entire thing more than once, but third time I reopened the webapp it showed me the mail using my account.

[Pat McGowan (pat-mcgowan)]

I think this is the one from the second run

[Pat McGowan (pat-mcgowan)]

from the first run

[David Barth (dbarth)]

I think we need 2 things first :

1. a reliable way to reproduce the account credentials renewal process; this is where the cookie renewal happens

2. Additional logs and tools to trace down the various redirections and cookie operations that the renewal process requires

3. Once we are there, the solution should be clearer. It may take relaxing the url pattern policy even further, or manage the redirection / overlays in a different way

#1 For reproducing, I think it is key to be able to store valid "old" cookies and account configurations, taken from different places in the system (the OA central base, the OA cookies, also the app cookies, the OA/app permissions), and be able to save that into an archive that can then be re-injected on a new device. Because we have to re-install our phones quite frenquently, it has been difficult to reproduce regularly, but really there is nothing preventing us from restoring cookies from 3 months ago, and get forced by Google to re-authenticate as a result.

#2 For tracing the redirection and cookie operations, it will take a few more log statements. But that should be the occasion to remove the remainder of the other useless debug lines that end up in the webapp-container logs

[Alberto Mardegan (mardy)]

I'm afraid that the attached logs are not relative to the first execution of the webapp, where the SAML login is performed.

#1 We do have this already, it's just a matter of copying the ~/.cache/online-accounts-ui/id-* files and restoring them.

#2 We also have useful debugging messages (and indeed, some less useful ones too) when the login happens; but in these logs we don't see them, because with all probability they are not the right log files.

I don't think that it's related to this bug at all, but there's a suspicious line in the logs:
   Ignoring empty or invalid webapp URL pattern: "https?://accounts.google.co.*/*"
Indeed, the code in url-pattern-utils.cpp doesn't seem to support this kind or pattern, and that should be fixed.

[Alexandre Abreu (abreu-alexandre)]

This log

Ignoring empty or invalid webapp URL pattern: "https?://accounts.google.co.*/*"

is a no-op, it "just" means that one of the patterns specified in the command line is invalid (which it is and should be updated),

Could you also attach the content of :

~/.config/com.ubuntu.developer.webapps.webapp-gmail/com.ubuntu.developer.webapps.webapp-gmail.conf

where the patterns are saved?

[David Barth (dbarth)]

Did that pattern acceptance change, or what is the correct form? This would
be a good explanation for some of the re-authentication process to be
ejected into the browser, and break.

On Wed, Nov 18, 2015 at 3:41 PM, Alexandre Abreu <
<email address hidden>> wrote:

> This log
>
> Ignoring empty or invalid webapp URL pattern:
> "https?://accounts.google.co.*/*"
>
> is a no-op, it "just" means that one of the patterns specified in the
> command line is invalid (which it is and should be updated),
>
> Could you also attach the content of :
>
> ~/.config/com.ubuntu.developer.webapps.webapp-
> gmail/com.ubuntu.developer.webapps.webapp-gmail.conf
>
> where the patterns are saved?
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1517102
>
> Title:
> Gmail webapp account renewal not successful
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/webapps-core/+bug/1517102/+subscriptions
>

[Alexandre Abreu (abreu-alexandre)]

Nothing changed, just that the pattern is found in the Exec line and I think might be a left over from an earlier version (or just a mistake), it was never an allowed form. For google, we allow e.g. www.google.* where the * is expanded to [^\\./] after but google.co.* is not a valid pattern (which probably should be treated as a separate "bug"),

And yes it should be fixed,

[Alexandre Abreu (abreu-alexandre)]

I created

https://bugs.launchpad.net/webapps-core/+bug/1517527

to cover the google.co.* case

[David Barth (dbarth)]

What I observed with a set of old tokens is that our own (canonical.com)
token renewal process gets ejected into the browser. The redirects are part
of the SAML / 3rd party auth. process, but not all of it is trapped
unfortunately.
Eventually, after restarting the webapp 3 times to finalize the process,
the tokens get refreshed and the Gmail webapp redirects to the main UI, but
there are at least 2 redirects that end up in the browser.

Here is what I found in the logs :

qml: Opening: https://login.ubuntu.com/two_factor_auth?next=/saml/process
in the
 browser window.
...
and later
...
qml: Opening: https://login.ubuntu.com/saml/process in the browser window.

I also spotted a few:
process 23654: arguments to dbus_connection_close() were incorrect,
assertion "c
onnection->generation == _dbus_current_generation" failed in file
../../dbus/dbu
s-connection.c line 2935.

if that matters.

On Wed, Nov 18, 2015 at 4:46 PM, Alexandre Abreu <
<email address hidden>> wrote:

> Nothing changed, just that the pattern is found in the Exec line and I
> think might be a left over from an earlier version (or just a mistake),
> it was never an allowed form. For google, we allow e.g. www.google.*
> where the * is expanded to [^\\./] after but google.co.* is not a valid
> pattern (which probably should be treated as a separate "bug"),
>
> And yes it should be fixed,
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1517102
>
> Title:
> Gmail webapp account renewal not successful
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/webapps-core/+bug/1517102/+subscriptions
>

[David Barth (dbarth)]

A few more observations from our debug session.

I can reproduce the issue with a backup of my tokens. I had removed .local/share/com...webapp-gmail, and once restarting Gmail the app was starting straight on the login.ubuntu.com page, asking for my user/pass.
The issue is that after validating correct user/pass creds, the new URL on login.ubuntu.com is not accepted by the container and re-opened in the browser.

On further inspection, the .config/com...webapp-gmail file which should contain the URL for login.ubuntu.com was not containing it.

This means that the SAMLRequest was never being seen by the container code. Otherwise, it would have added it in the config file.

Others tried to reproduce the problem with my own credentials backup, but in there case the app was opening on the initial Google login screen, not the 2nd login screen managed by Ubuntu.

Finally, I tried removing not only .local/share but also my .cache/com...webapp-gnail directory.

This resulted in the app opening on the initial Google login screen, not the ubuntu one. And as a result, when entering my username on this initial screen and validating, the subsequent SAMLRequest was properly detected and I was able to login and proceed to the Gmail inbox screen without issues nor browser re-directs.

So this indicates that the webapp cache, ie webapp-container/oxide cache, contains something like the last URL navigated to. And on app startup, the webview is directly and silently redirected there, without any onNavigationRequest signal being fired. This prevents the SAMLRequest pattern from being detected, and leads to the browser re-directs creating the quite messy user experience.

At this stage, we are looking for the cause for this cache and .config file to be out of sync. Normally, if ever the cache contains a login.ubuntu.com url, it should be as a result of an initial login, which should have been detected and added to the .config file.

[Alberto Mardegan (mardy)]

One thing that we noticed while reproducing this bug is that sometimes the URL patterns are not updated:
================
qml: SAML request detected. Adding host pattern: https?://login\.ubuntu\.com/*
qml: Invalid JSON content found in url patterns file
qml: Invalid JSON content type found in url patterns file (not an array)
================

Indeed, if the URL pattern were correctly stored into the config file, this bug would not happen.

[Alberto Mardegan (mardy)]

Actually, disregard my last comment: those error messages are quite misleading, but they don't point at a real error: they'll also be emitted when there are no URL patterns stored in the config file.

Pat, could you please paste your ~/.config/com.ubuntu.developer.webapps.webapp-gmail/com.ubuntu.developer.webapps.webapp-gmail.conf file?

[Alexandre Abreu (abreu-alexandre)]

Just to keep track of them, I entered

https://bugs.launchpad.net/ubuntu/+source/webbrowser-app/+bug/1522585

to address the misleading error messages.

And

https://bugs.launchpad.net/ubuntu/+source/webbrowser-app/+bug/1522562

as a workaround to help get the applications' state back to normal.