High CPU usage observed across EDS and indicator-datetime causing clock app tests to fail

Bug #1321775 reported by Nekhelesh Ramananthan on 2014-05-21
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Indicator Date and Time
Incomplete
Undecided
Charles Kerr
qtorganizer5-eds (Ubuntu)
Undecided
Unassigned

Bug Description

We have been observing random clock app test failures in the qa dashboard in image #39 and onwards. On further investigation we were able to reproduce to consistently,

1. Run the clock app tests. You will notice that they pass with the exception of 1 test.
2. Run the calendar app tests now.
3. Run the clock app tests again. Notice that there are about 5-6 test failures related to Alarms (EDS).

Timo noticed high CPU usages of EDS and indicator-datetime after the calendar app test finished. Somehow the calendar app test create events which are then messing up clock app results.

In the QA Dashboard, the clock app test failures are only noticed randomly. This is because the tests are not executed in the same order all the time. So sometimes if the calendar is run after the clock, then the clock app tests have a good time.

Related branches

summary: - High CPU usage observed across EDS and indicator-datetime causing tests
- clock app tests to fail
+ High CPU usage observed across EDS and indicator-datetime causing clock
+ app tests to fail
description: updated
Łukasz Zemczak (sil2100) wrote :

One interesting observation made today though: this happened once during smoketesting, but we didn't notice any failures on system-settle after the calendar-app tests. Normally when the CPU usage is really high (even inbetween reboots) this should have been caught by system settle tests, which generally check if the CPU usage levels don't go crazy. We didn't see those. Any ideas how that could have happened?

Charles Kerr (charlesk) wrote :

I'm not sure that I understand this report as it pertains to indicator-datetime.

I'll rerun the steps above to see if I can reproduce and address the high CPU load in indicator-datetime. However this seems like a red herring to me for the problem described in the summary:

(1) indicator-datetime and qt5organizer-eds both run their unit tests in isolated EDS sandboxes so that they can always guarantee that its EDS data is always the same when the tests are run. Do the calendar and clock apps not also do this?

(2) I don't see how it follows that high CPU load from a 3rd party app that only reads from EDS, never writes, will cause another app's tests to fail. nik90, you can try removing indicator-datetime from the equation by running "stop indicator datetime" from a shell before running the steps you outlined in the bug description.

Nekhelesh Ramananthan (nik90) wrote :

@Charles,

[1] Clock and Calendar app *do not* run their AP tests in isolated EDS sandboxes and that is the main issue. We already have a bug report in the clock app requesting for mocking the EDS collection to avoid interacting with the user's alarms and events, let alone the events of another app. But unfortunately I do not have the expertise to add the mocking server for EDS. I will see if I can get the QA team to implement that for the clock and calendar app.

[2] Timo observed high CPU load requested by EDS and indicator-datetime which indicates that these 2 services are clearly busy performing something in the background. As a result, when the clock app runs the tests after the calendar app, while saving the alarm, EDS does not respond to the clock resulting in the clock app crashing and causing the test to fail. Tomorrow morning I will test stopping the indicator-datetime and then running the tests again.

Nekhelesh Ramananthan (nik90) wrote :

@sil2100, I am not sure why the system settle tests are not failing. But if the high CPU loads were observed manually and the tests dint fail, then I suppose that would indicate the tests need to be modified/fixed to detect these issue as a failure.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in qtorganizer5-eds (Ubuntu):
status: New → Confirmed
Allan LeSage (allanlesage) wrote :

@nik90 , I'm finding that the autopilot tests are failing in trunk, this is blocks Charles from investigating further:

Traceback (most recent call last):
  File "/home/alesage/workspace/ubuntu-clock-app/tests/autopilot/ubuntu_clock_app/tests/test_clock.py", line 95, in test_delete_world_city_must_delete_saved_world_city_list
    self.page.add_world_city(index=0)
  File "/usr/lib/python2.7/dist-packages/autopilot/logging.py", line 46, in inner
    return f(instance, *args, **kwargs)
  File "/home/alesage/workspace/ubuntu-clock-app/tests/autopilot/ubuntu_clock_app/emulators.py", line 149, in add_world_city
    world_clock_page = self._click_add_city_button()
  File "/home/alesage/workspace/ubuntu-clock-app/tests/autopilot/ubuntu_clock_app/emulators.py", line 159, in _click_add_city_button
    header.click_action_button('addCityAction')
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/dbus.py", line 525, in __getattr__
    (self.__class__.__name__, name))
AttributeError: Class 'Header' has no attribute 'click_action_button'.

. . . I'm investigating but your help would be appreciated.

Allan LeSage (allanlesage) wrote :

Correction: these tests are working, thanks nik90 for your help getting them running.

Charles Kerr (charlesk) wrote :

I'm not able to reproduce the reported high CPU in indicator-datetime in build 53.

Setup:
I took a phablet freshly flashed to build 53, ran phablet-click-test-setup on the desktop followed by "click-buddy --dir . --provision" in the root directories of ubuntu-clock-app and ubuntu-calendar-app.
On the phablet I tweaked the autopilot directory to allow the ubuntu-calendar-app's tests to run (see lp:~nskaggs/ubuntu-calendar-app/fix-ap-env-setup).
In a separate phablet-shell, "powerd-cli display on bright"
Finally, I was ready to run the tests: "phablet-test-run -v ubuntu_clock_app", "phablet-test-run -v calendar_app", and "phablet-test-run -v ubuntu_clock_app".

Results:

1. During the first run of clock-app's autopilot tests, indicator-datetime's cpu stayed in the [0.0 .. 0.1%] range in htop and its cumulative time went from 0:00.75 to 0:01.91, a cumulative time growth of 1.16.

2. During the calendar-app autopilot tests, datetime's cpu use stayed in the [0.0 .. 0.1%] range in htop and its cumulative time went from 0:01.91 to 0:02.08,.

3. During the second run of clock-app's autopilot tests, datetime's cpu use stayed in the [0.0 .. 0.1%] range in htop and its cumulative time went from 0:02.08 to 0:03.18, an increase of 1.10, which is fairly consistent with the levels in the first test.

So nik90 or timo, if you're still able to reproduce this high CPU load in indicator-datetime is there a chance you could provide some information on either what steps I need to change to reproduce the behavior, or (say) a cachegrind profile of indicator-datetime taken during one of these high CPU sessions...

Changed in indicator-datetime:
assignee: nobody → Charles Kerr (charlesk)
status: New → Incomplete
Nekhelesh Ramananthan (nik90) wrote :

We were able to reproduce the bug when the calendar and clock app were not isolated when running the AP tests. However nskaggs's branch attached to this bug report isolates the calendar app tests to its own environment. That could probably be why you are not seeing the high CPU usage. While discussing this timo, we agreed that while the way to fix this bug would be to isolate the two apps, it is also important that indicator-datetime and the eds-plugin handle the high-cpu usages more gracefully.

I suppose one way for you to confirm this bug would be to try out older build of the clock and calendar. If however you feel that the app isolation is sufficient and a prerequisite, then this bug can be considered resolved.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers