Account-polld reports no updates, so no gmail notifications

[Roascio Paolo]

Account-polld works only once if i delete and recreate my gmail account. In this case i get a notification for only one random (a month old) mail in my account (in my test i sent 10 mail that were marked unread).

after that i never get mail notification and account-polld reports success with 0 updates.

here my account-polld log:

http://paste.ubuntu.com/12416594/

system-image-cli -i

current build number: 27
device name: krillin
channel: ubuntu-touch/stable/bq-aquaris.en
last update: 2015-11-20 17:49:34
version version: 27
version ubuntu: 20151118.2
version tag: OTA-8
version device: 20150821-736d127
version custom: 20151111--36-46-vivid

[Roascio Paolo (roa)]

Some more tests (sorry for my bad english):

if in the received mail page there is a couple of mail (~50 in my case, some read and others no), account-polld reports only the last mail which sender don't already exists in the list (so only the first time the sender sends a mail, after this it's address is already in the page and new messages from him are ignored)
I receive periodic mails from my home automation system and them are never notified, the same for mails from google or launchpad. only new senders, only once.

I expect somehing like account-polld caches the timestamp (or something like) of the last mail notified and then periodically check only for more recent matches, if the match is true, then notifies the mail and caches the new timestamp. And so on. This regardless of the sender.

Although i admit that my test results are in contrast with the case when the account is deleted and then recreated in the phone. In that case, only the oldest unread mail is notified.

[Roascio Paolo (roa)]

Other test:

I completely emptied my gmail account so i expected a notification on new mails.
well... no. no notifications at all.
My home automation system sent me the statistics mail at 00:00 (sender me) and it was ignored.
Now i wait for other mails, but this case distorts the way i thought account-polld works. For my experience the gmail notification is quite random

[Mikko Majander (mikko-majander)]

I have been noticing the same issue on Meizu MX4. I would say that most of the emails will go without notification so I decided to test if I can reproduce the issue or find some common nominator in the cases where the notification is not shown. To my surprise it worked quite well and the only way to reproduce it was to leave the gmail web app open and then lock the phone.

I don't know if this is useful or not, but here's what I tested:

-All emails sent from zoho
-I have @iki.fi address which is directed to my @gmail.com address. Most of the test emails we're sent directly to the @gmail.com.
-Only one google account configured, no other email accounts in use
-logs from account-polld: http://paste.ubuntu.com/12940752/
-OTA-7 installed

The test results:

Starting with phone locked, gmail web app closed
07:50 email sent, 07:51 notification received: phone kept locked
07:52 email sent, 07:56 notification received: notifications cleared, phone kept locked
07:57 email sent, 08:01 notification received: unlock phone, open notification in gmail web app, read email, go to inbox in gmail web app, leave it open and lock phone
08:03 email sent, 08:10 no notification: unlock phone, gmail web mail shows email in inbox. Close gmail web app, open dekko, lock phone
08:12 email sent, 08:16 notification received: turn off wifi and use 3G
08:20 email sent to gmail, 08:22 notification received
08:20 email sent to iki, 08:22 notification received: clear notifications, open gmail web app, go to home screen / scope, lock phone
08:28 email sent to iki, 08:32 notification received: unlock phone, open gmail web app and leave it in foreground, lock phone
08:32 email sent to iki, 08:40 no notification

As a bonus:
09:04 received unrelated email (heard notification sound from my work phone which syncs also my personal email). After waiting about 5 minutes, I unlocked the Meizu and there still was no notification. There was no gmail web app running.

Leaving gmail web app open and locking the phone seemed to cause the issue, but I believe that most of the time I don't have it open, not even in the background. The account-polld log shows 09:11 token expired, but otherwise I don't see anything special in it. The gmail account is #2 and disabled Facebook is #3.

I'm happy to provide more logs and test further if needed.

[Roascio Paolo (roa)]

The issue that there is notifications if an application is in foreground and the phone is locked is another story (https://bugs.launchpad.net/canonical-devices-system-image/+bug/1507510) and the solution is planned in ota-8...

I tried too to find some common nominator.

In my tests the only mails notified are those related to this bug (and not generated by me), so, yours and the two from Pat.
Other mails are never notified, no matter if are marked as important, with or without attachments, any sender. Date and time also doesn't seem to affect the missing notification. simply account-polld ignores all mails in my account.

Since i bought the phone on june, i received a total of 9 notifications, three as mentioned above, two in my tests (delete and recreate the gmail account in the phone) and four random and unrelated...

[Bill Filler (bfiller)]

@jgdx
please test this and see if it's the same issue (dupe) of application not getting notification when in foreground, and if not, lets try to debug the root cause.

[Jonas G. Drange (jonas-drange)]

@roa, mikko-majander: Thanks for reporting this. Could you attach the ubuntu-push-client log as well, from when you are able to reproduce this bug?

@bfiller: sure!

[Roascio Paolo (roa)]

@jgdx Thank you for looking into this, requested log is here:

http://paste.ubuntu.com/13315534

This bug is always reproducible, since i bought the phone. I confirm what i wrote in #4, mails are notified only from this thread and few spare others (usually 7-10 days old)

[Jonas G. Drange (jonas-drange)]

@roa, I can reproduce this behaviour when I receive emails that bypass my Inbox. We just check the Inbox folder, so please, make doubly sure the e-mails that you are trying to receive notifications for are in the Inbox and are not moved around by a filter.

The reason for bringing this up is that you mention e-mails from your home automation system and launchpad bugs---which might be the common denominator you were looking for.

If this is not the case, I still need a way to surely reproduce this issue, because right now I can't.

(Please note that gmail notifications will not wake up the screen in OTA < 9)

[Roascio Paolo (roa)]

Sorry, something went wrong :( and i posted the attachment without comment...

I double checked in my gmail account and all emails are in the inbox folder, there are only two senders filtered as spam (and obviously i don't want be notified for them) and no redirections set.

In my inbox, actually, there are 102 mail, none has been notified, even your.

since I have not secrets, if you think can be useful, i can send via p.m. my gmail credentials so you can check if the problem is an account misconfiguration or another reason.

In the meanwhile i send a couple of account-polld logs

[Jonas G. Drange (jonas-drange)]

@roa, the logs indicate that no account is being considered, which is odd. I'll take a look at the code and see how one would end up in this state.

And, could you add the output of $ system-image-cli -i in the bug description? Also, have you made any changes to the system image (i.e. making it writable and installing stuffs)?

Thank you for your patience and efforts.

[Roascio Paolo (roa)]

> And, could you add the output of $ system-image-cli -i in the bug description?

Done.

> Also, have you made any changes to the system image (i.e. making it writable and installing stuffs)?

No, it's equal as when i bought it, never mounted rw and updates are only via OTA. My customizations are:

- uninstalled amazon, twitter, facebook, cinco dìas, el paìs scopes and pathwind app.
- installed terminal, file manager, calendar, cloud music, logviewer, sudoku, utorch from the ubuntu store
- set ssh access as in this thread (post #7): http://askubuntu.com/questions/348714/how-can-i-access-my-ubuntu-phone-over-ssh

no other changes.

Thank you for looking into this and, since i have ssh access to the phone, feel free to ask every log/config file/wathever you think may be useful for your work.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jonas G. Drange (jonas-drange)]

I have requested some additional details from @roa and will set the status to incomplete until I've received these details.

[Jonas G. Drange (jonas-drange)]

User reports that the command

    curl --header "Authorization: Bearer yaXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXlpA" https://www.googleapis.com/gmail/v1/users/me/messages?q=is%3Aunread+category%3Apersonal+in%3Ainbox

yields no emails, and that

    curl --header "Authorization: Bearer yaXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXlpA" https://www.googleapis.com/gmail/v1/users/me/messages?q=is%3Aunread+in%3Ainbox

yields emails. The phone is making the former query, which is the cause of this bug.

A suggested solution is to make this query instead:

    "in:inbox is:unread after:$last_check"

where $last_check is the timestamp of the last check.

The down side with this is that some inbox configurations can produce a lot of notifications (of which at most 10 will be displayed in the notification center at once). On the other hand, this will solve the bug for the reporter, me, and many others.

[Roascio Paolo (roa)]

After ota-9 update, i tried the package attached in comment #18. After two weeks of testing i can write that all works as expected, so, at least for me, this bug can be closed.

Thank you for your great work ;)

[Launchpad Janitor (janitor)]

This bug was fixed in the package account-polld - 0.1+16.04.20160308.1-0ubuntu1

---------------
account-polld (0.1+16.04.20160308.1-0ubuntu1) xenial; urgency=medium

  [ Niklas Wenzel ]
  * Fix vibrations format for notifications (LP: #1431887)

  [ jonas-drange ]
  * Drop filter on category, limit the query in time (the same day).
    Also, fix bug where emails violating rfc2047 would not produce
    notifications. (LP: #1495904)
  * do not ignore the mangled address

 -- <email address hidden> (Jonas G. Drange) Tue, 08 Mar 2016 14:12:37 +0000