Today scope needs attention: slow to load and missing items

[TenLeftFingers]

Prerequisites:
E4.5 device with OTA-5 installed.
Turn on the phone, enter pin and observe Today scope (or just pull-down refresh to see the same behaviour).

Current situation:
Orange progress throbber takes 60 seconds to stop.
Some items never load (News, Twitter) and some only load sometimes (Day of week part that used to be on top).

Ideal situation:
Today scope is the focal point of Ubuntu Touch and should load more quickly that sixty seconds as it gives a poor first impression and mars the overall experience as this scope is a regular bounce point during day-to-day usage.
All scopes should load.
Automated testing should ensure that this kind of behaviour doesn't escape yet another OTA release.

Let me know which logs to attach.

[TenLeftFingers (tenleftfingers)]

Registry log. Thanks.

[Ilonka (ilonka-o)]

On my BQ 4.5 is actual date in today scope missing. I saw it after each reboot but then it dissapeared.

[Cédric Bellegarde (gnumdk)]

Slow on meizu too...

Scope need update code need to be rewritten:
- No progress bar
- Only update each sub scope when datas are available

For now, it's really bad looking.

[Cédric Bellegarde (gnumdk)]

Here a video showing top while today scope is updating. At the end, blank page in scope.

https://www.youtube.com/watch?v=rIQ7vbIySlM

[Kyle Nitzsche (knitzsche)]

Hi TenLeftFingers.

Thank you for filing the bug!

Can you please disable "Upcoming" in Today scope settings.

This should result in fast Today scope loading.

Please comment back.

Thanks - Kyle

[TenLeftFingers (tenleftfingers)]

Hi Kyle, thanks for making contact. I tried your suggestion but it made no
difference. I rebooted but also no difference. Sorry :/

On Tue, Jul 21, 2015 at 3:42 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi TenLeftFingers.
>
> Thank you for filing the bug!
>
> Can you please disable "Upcoming" in Today scope settings.
>
> This should result in fast Today scope loading.
>
> Please comment back.
>
> Thanks - Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Sebastien Bacher (seb128)]

getting the "day" section missing here, the scope log has

: unity::scopes::TimeoutException: Request timed out after 500 milliseconds (endpoint = ipc:///run/user/32011/zmq/priv/com.canonical.scopes.day_day, op = search)

[Kyle Nitzsche (knitzsche)]

I installed OTA5 with a --wipe on E4.5 this morning and have been running it all day. I have never seen a blank or a partially blank Today scope. I tested with network on and off and with Today scope location enabled and disabled. I do not see the exception mentioned in comment #7. There is a crash from the telegram scope but it has not lead to a blank or partially blank Today scope for me.

I am not saying a blank or partially blank today scope this is not a real issue :) Just saying I cannot reproduce it so far.

Let's ensure the following:
1) Please ensure you have the Day child scope installed. If Day scope is installed, you should see the following item in Today settings: "Display results from Day Info <checkbox>". Please comment here if you do NOT see that.
2) Please also ensure Day scope is enabled n Today by placing a check there in settings, and then return to a the Today scope and refresh.

On the question of long load time: we aware of a bug in which the Events scope causes the orange progress bar to continue moving back and forth at the bottom of the Today scope until it times out (40 seconds). All results should display (and do for me), but that moving progress bar is annoying :). You can disable "Upcoming" in Today scope settings (that is the events scope) if you want. That will remove Upcoming (events) from Today, so only do this if you do not rely on this functionality. Of course, if the Today scope is blank or partially blank, this won't make much difference.

[TenLeftFingers (tenleftfingers)]

Thanks Kyle,

1) The Day child scope must be installed because sometimes it shows. But
how can I verify to be sure? I have the terminal app installed and can
check anything you tell me with it.
2) The Today scope is definitely ticked (starred) in the Manage screen and
I moved it to the top of the list :)

I don't see an Events scope in the Management section for the Today scope.
Turning off Upcoming, My Activity and Telegram seemed to reduce the time by
at least half. But the throbber takes 60 seconds to stop, not 40. I'm using
a stopwatch to check. In the mean-time, the results display before this,
but not all.

Disabling Upcoming alone does not improve or change the situation.

Hope this is helpful. Let me know if I can provide more info/logs.

Thank you.

On Wed, Jul 22, 2015 at 3:21 AM, Yuan-Chen Cheng <email address hidden>
wrote:

> ** Changed in: today-scope
> Assignee: (unassigned) => Ethan Chang (ethan.chang)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Chris, have you seen this bug? If so it would be very helpful to get some info from you including whether there are any dashboard crashes (/var/crash/), the syslog, and scoperegistry.log at the time of the incident and the output of "click list".

[Kyle Nitzsche (knitzsche)]

HI TenLeftFingers,

1) You wrote:
"I don't see an Events scope in the Management section for the Today scope.
Turning off Upcoming,..."

Please note that Upcoming *is* the events scope. (which is confusing....)

So disabling Upcoming *alone* should disable events and significantly decrease the time the orange progress bar moves back and forth.

2) Just to be sure, please check whether you see the Day scope in Today scope settings. I've attached a screenshot showing Today settings. Note "Display Results from Day Info". Do you have that?

3) Let's confirm your system image settings. In terminal please enter this:

COMMAND: system-image-cli -i
current build number: 24
device name: krillin
channel: ubuntu-touch/stable/bq-aquaris.en
last update: 2015-07-21 17:05:57
version version: 24
version ubuntu: 20150713
version device: 20150529-8e13c5f
version custom: 20150709-814-29-21-vivid

Can you please confirm that the results are the same (except for "last update" which should be different)

Report any differences.

4) Did you ever set "writable-image"? (If you don't understand this question, the answer is probably "No".) We can test this as follows in Terminal. Enter the following command
COMMAND: touch /test
touch: cannot touch ‘/test’: Read-only file system

That is the normal and correct result, a read-only file system.

Please let me know if you see some thing else.

Thanks a lot for your help debugging this TenLeftFingers!

[Kyle Nitzsche (knitzsche)]

TenLeftFingers:

Your original scope-registry.log is just a short fragment.

Is it possible for you to get and add to this bug the /home/phablet/.cache/upstart/scope-registry.log immediately AFTER you see a blank or partially blank Today scope?

It might contain very useful information.

Thanks again,
Kyle

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

"So disabling Upcoming *alone* should disable events and significantly
decrease the time the orange progress bar moves back and forth."
Okay, this is not the case in my instance.

" Just to be sure, please check whether you see the Day scope in Today
scope settings. I've attached a screenshot showing Today settings. Note
"Display Results from Day Info". Do you have that?"
Yes, I have Day Info and it is ticked.

" In terminal please enter
this:

COMMAND: system-image-cli -i"
With the exception of "last update" (which has the same date, but different
time) the output is identical to what you posted.

" Did you ever set "writable-image"? (If you don't understand this
question, the answer is probably "No".) We can test this as follows in
Terminal. Enter the following command
COMMAND: touch /test
touch: cannot touch ‘/test’: Read-only file system"
I didn't do anything like that. The output on my device is identical to
what you posted.

Thanks again for your time. I have attached registry.log after a scope
refresh with Upcoming disabled. All other 'scopelets' are enabled.

Jarlath

On Wed, Jul 22, 2015 at 3:53 PM, Kyle Nitzsche <email address hidden>
wrote:

> TenLeftFingers:
>
> Your original scope-registry.log is just a short fragment.
>
> Is it possible for you to get and add to this bug the
> /home/phablet/.cache/upstart/scope-registry.log immediately AFTER you
> see a blank or partially blank Today scope?
>
> It might contain very useful information.
>
> Thanks again,
> Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Hi Jarlath,

That log you posted shows the following two lines, one right after the other (in all but one case):

: unity::scopes::TimeoutException: Request timed out after 500 milliseconds (endpoint = ipc:///run/user/32011/zmq/priv/com.canonical.scopes.day_day, op = search)

"file:///custom/click/com.canonical.scopes.tasks/1.3/sctasks/tasks.png"

Can you please disable Tasks from Today settings and refresh your scope.

Then please attach the new scope-registry.log?

And please say whether this caused Day to show (I doubt it will, I just want to clearly rule tasks out as the cause).

(Probably the timeout is the cause.)

thanks,
Kyle

[TenLeftFingers (tenleftfingers)]

Thanks again Kyle,

As you expected, it made no difference. Here's the latest log:

On Wed, Jul 22, 2015 at 6:34 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi Jarlath,
>
> That log you posted shows the following two lines, one right after the
> other (in all but one case):
>
> : unity::scopes::TimeoutException: Request timed out after 500
> milliseconds (endpoint =
> ipc:///run/user/32011/zmq/priv/com.canonical.scopes.day_day, op =
> search)
>
> "file:///custom/click/com.canonical.scopes.tasks/1.3/sctasks/tasks.png"
>
> Can you please disable Tasks from Today settings and refresh your
> scope.
>
> Then please attach the new scope-registry.log?
>
> And please say whether this caused Day to show (I doubt it will, I just
> want to clearly rule tasks out as the cause).
>
> (Probably the timeout is the cause.)
>
> thanks,
> Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Jarlath,

Can you please provide the versions of the Day scope and the Holidays scope in terminal as follows:

COMMAND: click list | grep day
com.canonical.scopes.day 0.1.10
com.canonical.scopes.holidays 1.1.4

Thanks,
Kyle

[Kyle Nitzsche (knitzsche)]

NOTE: all people who have experienced this issue and from whom we have a timely log have the following exception in their scope-registry.log:

unity::scopes::TimeoutException: Request timed out after 500 milliseconds (endpoint = ipc:///run/user/32011/zmq/priv/com.canonical.scopes.day_day, op = search)

Seems like a probable cause.

[TenLeftFingers (tenleftfingers)]

Its the exact same as what you posted.

On Wednesday, July 22, 2015, Kyle Nitzsche <email address hidden>
wrote:
> Jarlath,
>
> Can you please provide the versions of the Day scope and the Holidays
> scope in terminal as follows:
>
> COMMAND: click list | grep day
> com.canonical.scopes.day 0.1.10
> com.canonical.scopes.holidays 1.1.4
>
> Thanks,
> Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Hi Jarlath,

If you disable Day scope in Today scope settings, does Today seem normal?

[Kyle Nitzsche (knitzsche)]

Noting that I have tried to reproduce the bug on a mako with a fresh install as follows and have not been able to reproduce:

phablet@ubuntu-phablet:~$ system-image-cli -i
current build number: 21
device name: mako
channel: ubuntu-touch/stable/ubuntu
last update: 1970-01-01 01:29:36
version version: 21
version ubuntu: 20150713
version device: 20150210
version custom: 20150713

Onto which I installed four key OTA5 scopes:

com.canonical.scopes.calls_1.3.1_armhf.click
com.canonical.scopes.dashboard_1.8.7_armhf.click
com.canonical.scopes.day_0.1.10_armhf.click
com.canonical.scopes.holidays_1.1.4_armhf.click
com.canonical.scopes.texts_1.3.5_armhf.click
com.ubuntu.telegram_1.3.18.112_armhf.click

With and without location and network.

Day scope always shows. No timeout exception in scoperegistry.log

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

No difference. I tried twice just to be sure. So now Day, Upcoming and
Tasks are disabled and the behaviour hasn't changed.

On Thursday, July 23, 2015, Kyle Nitzsche <email address hidden>
wrote:
> Hi Jarlath,
>
> If you disable Day scope in Today scope settings, does Today seem
> normal?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Hi Jarlath,

Thanks for keeping up with my requests!

Can you please take a photo (and attach it) of Today scope with Day and Upcoming disabled so that I can see what you see?

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

My pleasure! I'll send the photo in a few minutes. I've spent the last few
minutes running a test, starting with every option in Today scope
Management disabled and then adding one by one. This shows some culprits:

Disable all: 3 seconds
+Location data: 3 seconds
+Contacts: 2.5 seconds
+Day: 2.5 seconds
(Skipping Upcoming as it bumps the total to 56 seconds)
+My Activity: 3 seconds
+Holidays: 3 seconds
+Tasks: 4 seconds
+Twitter: 3 seconds
+Weather: 4 seconds
+BBC: 4 seconds
(Skipping Calls as it bumps the total to 10 seconds)
+Euronews: 3 seconds
(Skipping Messages as it bumps the total to 12 seconds)
(Skipping Telegram as it bumps the total to 49 seconds)

On Thu, Jul 23, 2015 at 6:15 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi Jarlath,
>
> Thanks for keeping up with my requests!
>
> Can you please take a photo (and attach it) of Today scope with Day and
> Upcoming disabled so that I can see what you see?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

When I disable Upcoming, Calls, Messages and Telegram, the Day info
consistently displays correctly.

On Thu, Jul 23, 2015 at 6:40 PM, Jarlath Reidy <email address hidden>
wrote:

> Here's a short video. I start with everything enabled and then disable the
> Day and Upcoming. Then I wait for the throbber to stop and check that items
> still don't show.
>
> On Thu, Jul 23, 2015 at 6:35 PM, Jarlath Reidy <email address hidden>
> wrote:
>
>> Hi Kyle,
>>
>> My pleasure! I'll send the photo in a few minutes. I've spent the last
>> few minutes running a test, starting with every option in Today scope
>> Management disabled and then adding one by one. This shows some culprits:
>>
>> Disable all: 3 seconds
>> +Location data: 3 seconds
>> +Contacts: 2.5 seconds
>> +Day: 2.5 seconds
>> (Skipping Upcoming as it bumps the total to 56 seconds)
>> +My Activity: 3 seconds
>> +Holidays: 3 seconds
>> +Tasks: 4 seconds
>> +Twitter: 3 seconds
>> +Weather: 4 seconds
>> +BBC: 4 seconds
>> (Skipping Calls as it bumps the total to 10 seconds)
>> +Euronews: 3 seconds
>> (Skipping Messages as it bumps the total to 12 seconds)
>> (Skipping Telegram as it bumps the total to 49 seconds)
>>
>> On Thu, Jul 23, 2015 at 6:15 PM, Kyle Nitzsche <
>> <email address hidden>> wrote:
>>
>>> Hi Jarlath,
>>>
>>> Thanks for keeping up with my requests!
>>>
>>> Can you please take a photo (and attach it) of Today scope with Day and
>>> Upcoming disabled so that I can see what you see?
>>>
>>> --
>>> You received this bug notification because you are subscribed to the bug
>>> report.
>>> https://bugs.launchpad.net/bugs/1476282
>>>
>>> Title:
>>> Today scope needs attention: slow to load and missing items
>>>
>>> To manage notifications about this bug go to:
>>>
>>> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>>>
>>
>>
>

[Kyle Nitzsche (knitzsche)]

also please a photo with Day enabled.

[TenLeftFingers (tenleftfingers)]

My previous two messages didn't seem to make it. Maybe because a video was attached to the first one. I'm uploading it to YouTube at this URL: http://youtu.be/rq53WI5_yDc

[TenLeftFingers (tenleftfingers)]

This one has Day enabled (so, only Upcoming is disabled): http://youtu.be/M-YcS3bZMqI

[TenLeftFingers (tenleftfingers)]

With only Day enabled (no others), Day info displays quickly.

[Kyle Nitzsche (knitzsche)]

Hi Jarlath,

Regarding the load times:
* Upcoming (events) is broken regarding the load time/progress, and has a bug, so lets ignore this.
* calls and messages are a little slow and could be improved, so I filed these two bugs:
   * https://bugs.launchpad.net/calls-scope/+bug/1477700
  * https://bugs.launchpad.net/texts-scope/+bug/1477702
* telegram: I was not aware of a performance issue here. Can you file a bug saying your scope loads slows against https://bugs.launchpad.net/libqtelegram

So with these points set aside from the conversation, my main concern is if Day scope does not display when enabled. That is a critical issue I would need to fix and is the remaining focus of this bug.

Therefore, can you do the following:
1) enable Day scope
2) in terminal run this command: tail -F /home/phablet/.cache/upstart/scope-registry.log | grep -i exception
3) refresh the Today scope
4) If day scope does NOT display (which is the bug), I want to know whether the terminal shows any exceptions (like the the Timeout exception mentioned above.) If itdoes, please grab your scoperegistry.log and attach it here.

[TenLeftFingers (tenleftfingers)]

Sure Kyle, just to be clear - you want me to enable _only_ the Day scope?

On Thu, Jul 23, 2015 at 7:02 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi Jarlath,
>
> Regarding the load times:
> * Upcoming (events) is broken regarding the load time/progress, and has a
> bug, so lets ignore this.
> * calls and messages are a little slow and could be improved, so I filed
> these two bugs:
> * https://bugs.launchpad.net/calls-scope/+bug/1477700
> * https://bugs.launchpad.net/texts-scope/+bug/1477702
> * telegram: I was not aware of a performance issue here. Can you file a
> bug saying your scope loads slows against
> https://bugs.launchpad.net/libqtelegram
>
> So with these points set aside from the conversation, my main concern is
> if Day scope does not display when enabled. That is a critical issue I
> would need to fix and is the remaining focus of this bug.
>
> Therefore, can you do the following:
> 1) enable Day scope
> 2) in terminal run this command: tail -F
> /home/phablet/.cache/upstart/scope-registry.log | grep -i exception
> 3) refresh the Today scope
> 4) If day scope does NOT display (which is the bug), I want to know
> whether the terminal shows any exceptions (like the the Timeout exception
> mentioned above.) If itdoes, please grab your scoperegistry.log and attach
> it here.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Jarlath. I want want to be sure that when Day scope is enabled it displays AND you see no exceptions. You should disable Upcoming (because it slows things down) and any others if you want to speed things up, but it is not required for this test.

[Kyle Nitzsche (knitzsche)]

In the case when you have Day enabled and you return to the main view and Day does not show. Does it show after you refresh with a pull down? Or does it just never show again?

[Kyle Nitzsche (knitzsche)]

A.lso:
What version of telegram scope do you have? is it this:
phablet@ubuntu-phablet:~$ click list | grep tele
com.ubuntu.telegram 1.3.18.112

[TenLeftFingers (tenleftfingers)]

It threw this exception (file also attached):
: unity::scopes::TimeoutException: Request timed out after 500
milliseconds (endpoint =
ipc:///run/user/32011/zmq/priv/com.canonical.scopes.day_day, op = search)

output of $ click list | grep tele:
com.ubuntu.telegram 1.3.20.114

After 2 refreshes, still no Day scopelet. But after 3 refreshes, the day
scope has appeared. I refreshed a further 10 times and it survives all of
them.

On Thu, Jul 23, 2015 at 7:41 PM, Kyle Nitzsche <email address hidden>
wrote:

> In the case when you have Day enabled and you return to the main view
> and Day does not show. Does it show after you refresh with a pull down?
> Or does it just never show again?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

I have filed the Telegram bug at https://bugs.launchpad.net/libqtelegram/+bug/1477727

[Kyle Nitzsche (knitzsche)]

Jarlath,

Now that Day scope is showing, can you determine what circumstances cause it to not display (when it is enabled)?

For example, does it depend on whether any particular other child scopes are enabled?

Does it only happen when Day is Disabled and Enable Day and the you return to the main view?

Thanks again for your persistence!

[Kyle Nitzsche (knitzsche)]

Jarlath,

Can you please do these two things and attach the two created files:

click list > myclicks
dpkg -l > mydebs

Cheers

[TenLeftFingers (tenleftfingers)]

Thanks to you too Kyle!

I've tried disabling and re-enabling Day Info (no others are enabled) and
the scope now behaves as expected.

I've tried re-enabling all scopelets one-by-one but Day Info displays
reliably for now.

On Thu, Jul 23, 2015 at 8:30 PM, Kyle Nitzsche <email address hidden>
wrote:

> Jarlath,
>
> Now that Day scope is showing, can you determine what circumstances
> cause it to not display (when it is enabled)?
>
> For example, does it depend on whether any particular other child scopes
> are enabled?
>
> Does it only happen when Day is Disabled and Enable Day and the you
> return to the main view?
>
> Thanks again for your persistence!
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

Sure, here you go

On Thu, Jul 23, 2015 at 8:45 PM, Jarlath Reidy <email address hidden>
wrote:

> Thanks to you too Kyle!
>
> I've tried disabling and re-enabling Day Info (no others are enabled) and
> the scope now behaves as expected.
>
> I've tried re-enabling all scopelets one-by-one but Day Info displays
> reliably for now.
>
> On Thu, Jul 23, 2015 at 8:30 PM, Kyle Nitzsche <
> <email address hidden>> wrote:
>
>> Jarlath,
>>
>> Now that Day scope is showing, can you determine what circumstances
>> cause it to not display (when it is enabled)?
>>
>> For example, does it depend on whether any particular other child scopes
>> are enabled?
>>
>> Does it only happen when Day is Disabled and Enable Day and the you
>> return to the main view?
>>
>> Thanks again for your persistence!
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1476282
>>
>> Title:
>> Today scope needs attention: slow to load and missing items
>>
>> To manage notifications about this bug go to:
>>
>> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>>
>
>

[Kyle Nitzsche (knitzsche)]

HI Jarlath - curious. Are you still seeing the Day scope sometimes not display in Today when it is enabled? Cheers

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

Yes, it has not shown itself today even though it is enabled. I have
Upcoming, My Activity, Calls, Messages and Telegram disabled but still, Day
Info doesn't show.

Thanks!

On Fri, Jul 24, 2015 at 8:02 PM, Kyle Nitzsche <email address hidden>
wrote:

> HI Jarlath - curious. Are you still seeing the Day scope sometimes not
> display in Today when it is enabled? Cheers
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Jarlath:

Can you provide the output of two commands:
1) ls /var/cash
2) locale

And please, where are you physically? (country and city)

[Kyle Nitzsche (knitzsche)]

Jarlath,

typo: I meant:

ls /var/crash

My apologies.

[TenLeftFingers (tenleftfingers)]

I'm based it Galway city, Ireland. The output of that commad is:

_opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.crash
_opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.upload
_opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.uploaded
_opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.crash
_opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.upload
_opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.uploaded
_usr_bin_dialer-app.32011.crash
_usr_bin_dialer-app.32011.upload
_usr_bin_dialer-app.32011.uploaded
_usr_bin_media-hub-server.32011.crash
_usr_bin_media-hub-server.32011.upload
_usr_bin_media-hub-server.32011.uploaded
_usr_bin_pulseaudio.32011.crash
_usr_bin_pulseaudio.32011.upload
_usr_bin_pulseaudio.32011.uploaded
_usr_bin_webbrowser-app.32011.crash
_usr_bin_webbrowser-app.32011.upload
_usr_bin_webbrowser-app.32011.uploaded
_usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.crash
_usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.upload
_usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.uploaded
_usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.crash
_usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.upload
_usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.uploaded

On Fri, Jul 24, 2015 at 9:26 PM, Kyle Nitzsche <email address hidden>
wrote:

> Jarlath,
>
> typo: I meant:
>
> ls /var/crash
>
> My apologies.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

I forgot locale:

LANG=en_US.UTF-8
LANGUAGE=en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_IE.UTF-8
LC_TIME=en_IE.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_IE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_IE.UTF-8
LC_NAME=en_IE.UTF-8
LC_ADDRESS=en_IE.UTF-8
LC_TELEPHONE=en_IE.UTF-8
LC_MEASUREMENT=en_IE.UTF-8
LC_IDENTIFICATION=en_IE.UTF-8
LC_ALL=

On Fri, Jul 24, 2015 at 10:39 PM, Jarlath Reidy <email address hidden>
wrote:

> I'm based it Galway city, Ireland. The output of that commad is:
>
>
> _opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.crash
>
> _opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.upload
>
> _opt_click.ubuntu.com_com.ubuntu.docviewer_0.3.146_lib_arm-linux-gnueabihf_bin_ubuntu-docviewer-app.32011.uploaded
> _opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.crash
> _opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.upload
>
> _opt_click.ubuntu.com_com.ubuntu.telegram_1.3.20.114_telegram.32011.uploaded
> _usr_bin_dialer-app.32011.crash
> _usr_bin_dialer-app.32011.upload
> _usr_bin_dialer-app.32011.uploaded
> _usr_bin_media-hub-server.32011.crash
> _usr_bin_media-hub-server.32011.upload
> _usr_bin_media-hub-server.32011.uploaded
> _usr_bin_pulseaudio.32011.crash
> _usr_bin_pulseaudio.32011.upload
> _usr_bin_pulseaudio.32011.uploaded
> _usr_bin_webbrowser-app.32011.crash
> _usr_bin_webbrowser-app.32011.upload
> _usr_bin_webbrowser-app.32011.uploaded
> _usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.crash
> _usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.upload
> _usr_lib_arm-linux-gnueabihf_qt5_bin_qmlscene.32011.uploaded
> _usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.crash
> _usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.upload
> _usr_lib_arm-linux-gnueabihf_unity-scopes_scoperunner.32011.uploaded
>
> On Fri, Jul 24, 2015 at 9:26 PM, Kyle Nitzsche <
> <email address hidden>> wrote:
>
>> Jarlath,
>>
>> typo: I meant:
>>
>> ls /var/crash
>>
>> My apologies.
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1476282
>>
>> Title:
>> Today scope needs attention: slow to load and missing items
>>
>> To manage notifications about this bug go to:
>>
>> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>>
>
>

[Kyle Nitzsche (knitzsche)]

Thank you Jarlath.

I have an idea as to the root cause and am running by some folks here. We shall see :)

I may not be available until Tuesday (holiday Monday).

I will return, and thanks again for all of your work to help solve this issue!

Cheers,
Kyle

[TenLeftFingers (tenleftfingers)]

Thanks to you too Kyle, I appreciate your time and effort. Enjoy the long
weekend :)

On Friday, July 24, 2015, Kyle Nitzsche <email address hidden> wrote:
> Thank you Jarlath.
>
> I have an idea as to the root cause and am running by some folks here.
> We shall see :)
>
> I may not be available until Tuesday (holiday Monday).
>
> I will return, and thanks again for all of your work to help solve this
> issue!
>
> Cheers,
> Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Update:
The scope framework expects certain methods to return very quickly, notably Scope::search(). Otherwise there's a timeout exception and there are issues.

The code path from my Scope::search() method had included reading a json file that defines the agg scope and querying the registry to see which scopes/keywords are installed and enabled in order to populate the list of current scopes.

I modified the code to run that stuff elsewhere, and two test users report improvements (fewer day scope no-shows). But it is not a complete solution: there are still some no shows and some timeoutexceptions.

I've attached that click here.

[Kyle Nitzsche (knitzsche)]

Another update: I asked my kind tester (who has the Day scope no-show issue) to install an earlier version of the Day scope (which provides the Date at the top and is sometimes missing). It always appears during normal operations (Except not right after a boot, which may be this scoperunner issue: https://errors.ubuntu.com/problem/21d9e7ddf91a26b21abfb2758315ad41fcfd3fa9

[Kyle Nitzsche (knitzsche)]

This issue is fix-committed with day-scope release 0.1.11 (bzr rev 24) here:
https://code.launchpad.net/~day-scope-team/day-scope/trunk

[Kyle Nitzsche (knitzsche)]

I am going to mark this bug fix-committed because the the main issue was with the Day scope not displaying and it is fix-committed. The other issues about slow load times are tracked in other bugs:
* https://bugs.launchpad.net/calls-scope/+bug/1477700
* https://bugs.launchpad.net/texts-scope/+bug/1477702
* https://bugs.launchpad.net/libqtelegram/+bug/1477727
* internal bug on events (Upcoming) progress bar takes a long time

[Kavoura (kavoura)]

I am also having a problem with the Today scope. Since the latest update, it keeps going blank, nothing displayed at all other than Today and icons at the top, rest is white/blank, on BQ Aquarius E45

[Cédric Bellegarde (gnumdk)]

@Kavoura Current version is 0.1.10, so bug fix isn't released.

[TenLeftFingers (tenleftfingers)]

OTA6 is giving me a consistent Today child-scope so far, but I'm now only seeing:
Today child-scope,
Weather,
Holiday
Favorite Contacts,

Everything else is truncated / doesn't show.

Are there no selenium type automated tests for this, or am I alone in experiencing this issue?

[Kyle Nitzsche (knitzsche)]

Hi TenLeftFingers,

Sorry you are having this issue.

1) Please verify all Today's child scopes are enabled in Today scope settings and then that the problem continues.

2) Can you please attach:

* The "clicks" file created by this command:
click list >clicks

* Your /home/phablet/.cache/upstart/scope-registery.log (get this right after you refresh the Today scope and you see the problem)

3) After Contacts is the "Recent" category in the Today scope. I would expect that to contain one result from each of the following:
* Calls
* Messages
* Telegram

Can you please disable all three of these in the Today scope and enable them one at a time and let me know if that results in everything working. This may narrow the problem to one of those child scopes.

Thanks,
Kyle

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

Thanks for your response!

When I disable calls, messaging and Telegram, the scope loads in 3 seconds
- which is an improvement. Activity, News and Twitter show up in this case.
When I enable Calls (but not the other two), it takes about 30 seconds but
I eventually see Recent Calls, Activity, News and Twitter.
When I disable Calls again, it takes 3 seconds AND My Activity, News and
Twitter still show!
When I enable Messages (but not the other two), it takes about 55 seconds
and again see nothing after Favorite Contacts. Messages do not show.
When I enable Telegram (but not the other two), it takes about 5 seconds
and I see Recent Telegrams, Activity, News and Twitter.

I will attach files later unless you instruct me otherwise. But, I think
the Today scope needs a way to recognise a problem scope and not let that
result in other healthy scopes not loading.

Many thanks,
Jarlath

On Fri, Sep 4, 2015 at 7:08 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi TenLeftFingers,
>
> Sorry you are having this issue.
>
> 1) Please verify all Today's child scopes are enabled in Today scope
> settings and then that the problem continues.
>
> 2) Can you please attach:
>
> * The "clicks" file created by this command:
> click list >clicks
>
> * Your /home/phablet/.cache/upstart/scope-registery.log (get this right
> after you refresh the Today scope and you see the problem)
>
> 3) After Contacts is the "Recent" category in the Today scope. I would
> expect that to contain one result from each of the following:
> * Calls
> * Messages
> * Telegram
>
> Can you please disable all three of these in the Today scope and enable
> them one at a time and let me know if that results in everything
> working. This may narrow the problem to one of those child scopes.
>
> Thanks,
> Kyle
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

I've attached those. And to answer you more directly:

1) Yes, with all enabled the issue exists.
2) Please see attached.
3) Answered in previous post.

Many thanks!

On Fri, Sep 4, 2015 at 7:36 PM, Jarlath Reidy <email address hidden>
wrote:

> Hi Kyle,
>
> Thanks for your response!
>
> When I disable calls, messaging and Telegram, the scope loads in 3 seconds
> - which is an improvement. Activity, News and Twitter show up in this case.
> When I enable Calls (but not the other two), it takes about 30 seconds but
> I eventually see Recent Calls, Activity, News and Twitter.
> When I disable Calls again, it takes 3 seconds AND My Activity, News and
> Twitter still show!
> When I enable Messages (but not the other two), it takes about 55 seconds
> and again see nothing after Favorite Contacts. Messages do not show.
> When I enable Telegram (but not the other two), it takes about 5 seconds
> and I see Recent Telegrams, Activity, News and Twitter.
>
> I will attach files later unless you instruct me otherwise. But, I think
> the Today scope needs a way to recognise a problem scope and not let that
> result in other healthy scopes not loading.
>
> Many thanks,
> Jarlath
>
> On Fri, Sep 4, 2015 at 7:08 PM, Kyle Nitzsche <<email address hidden>
> > wrote:
>
>> Hi TenLeftFingers,
>>
>> Sorry you are having this issue.
>>
>> 1) Please verify all Today's child scopes are enabled in Today scope
>> settings and then that the problem continues.
>>
>> 2) Can you please attach:
>>
>> * The "clicks" file created by this command:
>> click list >clicks
>>
>> * Your /home/phablet/.cache/upstart/scope-registery.log (get this right
>> after you refresh the Today scope and you see the problem)
>>
>> 3) After Contacts is the "Recent" category in the Today scope. I would
>> expect that to contain one result from each of the following:
>> * Calls
>> * Messages
>> * Telegram
>>
>> Can you please disable all three of these in the Today scope and enable
>> them one at a time and let me know if that results in everything
>> working. This may narrow the problem to one of those child scopes.
>>
>> Thanks,
>> Kyle
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1476282
>>
>> Title:
>> Today scope needs attention: slow to load and missing items
>>
>> To manage notifications about this bug go to:
>>
>> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>>
>
>

[Kyle Nitzsche (knitzsche)]

Hi Jarlath,

I am attaching a *dev* version of Messages that is much faster. (In Today scope, it should prevent the long pause .) I have the same fix for the calls scope, but let's see if this works.

Can you please install the attached, disable Calls and see if things are better?

The files will be much appreciated when you get around to it!

Cheers,
Kyle

[Kyle Nitzsche (knitzsche)]

This is an interesting exception:

: ReplyObject::push(VariantMap): unity::InvalidArgumentException: register_category(): duplicate category: not_logged_infitbit_localId

This indicates the agg scope is trying to register a category that already exists. To avoid this, in most cases I timestamp the new category_id to ensure it is unique. This should always be done so I will chase this down.

[Kyle Nitzsche (knitzsche)]

Tacking the issue in comment #59 here:
https://bugs.launchpad.net/scope-aggregator/+bug/1492428

[TenLeftFingers (tenleftfingers)]

Thanks Kyle,

I'm glad it makes sense to one of us :)

Could you guide me on the click installation process? The last click I
tried to install, the terminal told me that if I didn't 'register' it, it
would be garbage collected on restart. So I don't know how to do it
properly yet.

On Fri, Sep 4, 2015 at 8:01 PM, Kyle Nitzsche <email address hidden>
wrote:

> This is an interesting exception:
>
> : ReplyObject::push(VariantMap): unity::InvalidArgumentException:
> register_category(): duplicate category: not_logged_infitbit_localId
>
> This indicates the agg scope is trying to register a category that
> already exists. To avoid this, in most cases I timestamp the new
> category_id to ensure it is unique. This should always be done so I will
> chase this down.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[TenLeftFingers (tenleftfingers)]

I've installed the click using:
sudo click install --allow-unauthenticated <package>.click

But I notice no difference after disabling calls.

I got a warning that the clilck wasn't registered to a user and would be
lost at restart. Not sure if that counts for anything?

On Fri, Sep 4, 2015 at 8:12 PM, Kyle Nitzsche <email address hidden>
wrote:

> Tacking the issue in comment #59 here:
> https://bugs.launchpad.net/scope-aggregator/+bug/1492428
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

Hi,

If you are using the terminal app, and you have the click in the /home/phablet/ directory and that is where you are you can do this:

(I am not sure if this requires that the phone be in developer mode (from settings) but I don't think so)

pkcon install-local --allow-untrusted com.canonical.scopes.texts_1.3.4.2_armhf.click

If you have your phone connected by usb to your laptop and you have terminal over adb (from phablet-tools pkg and certain other steps) you can do this:

adb push com.canonical.scopes.texts_1.3.4.2_armhf.click /tmp
adb shell pkcon install-local --allow-untrusted /tmp/com.canonical.scopes.texts_1.3.4.2_armhf.click

[Kyle Nitzsche (knitzsche)]

Hi,

Your method of installation does not work because click packages are installed and also must be registered to a user. So while you installed it, it is not available for you.

Please try one of the methods in comment #63.

[TenLeftFingers (tenleftfingers)]

Hi Kyle,

It's much better with your click. I can disable calls and the others will
show. Even with calls enabled, after much delay, the other scopes still
show.

On Fri, Sep 4, 2015 at 8:45 PM, Kyle Nitzsche <email address hidden>
wrote:

> Hi,
>
> Your method of installation does not work because click packages are
> installed and also must be registered to a user. So while you installed
> it, it is not available for you.
>
> Please try one of the methods in comment #63.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>

[Kyle Nitzsche (knitzsche)]

awesome.

So a theory is that messages scope was timing out (child scopes timeout at 40 seconds and you said it took 55 seconds).

I don't see this because I don't have that many messages.

With that dev messages click, it returns the result to Today scope quite quickly, so no timeout. That (and the same changes to calls scope) needs to be published. I have two merge proposals for this:

MESSAGES:
https://code.launchpad.net/~knitzsche/texts-scope/texts-responsiveness-bug-1477702/+merge/270209

CALLS:
https://code.launchpad.net/~knitzsche/calls-scope/slow-load-fix-bug-1477700/+merge/270096

Yes, a child scope timing out should NOT cause other child scopes to fail to show. this needs investigating.

Thanks for your help in debugging this!

[TenLeftFingers (tenleftfingers)]

And thanks to you!

On Fri, Sep 4, 2015 at 9:22 PM, Kyle Nitzsche <email address hidden>
wrote:

> awesome.
>
> So a theory is that messages scope was timing out (child scopes timeout
> at 40 seconds and you said it took 55 seconds).
>
> I don't see this because I don't have that many messages.
>
> With that dev messages click, it returns the result to Today scope quite
> quickly, so no timeout. That (and the same changes to calls scope) needs
> to be published. I have two merge proposals for this:
>
> MESSAGES:
>
> https://code.launchpad.net/~knitzsche/texts-scope/texts-responsiveness-bug-1477702/+merge/270209
>
> CALLS:
>
> https://code.launchpad.net/~knitzsche/calls-scope/slow-load-fix-bug-1477700/+merge/270096
>
> Yes, a child scope timing out should NOT cause other child scopes to
> fail to show. this needs investigating.
>
> Thanks for your help in debugging this!
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1476282
>
> Title:
> Today scope needs attention: slow to load and missing items
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/canonical-devices-system-image/+bug/1476282/+subscriptions
>