Autopilot Tests fail for month view

[Nicholas Skaggs]

Test monthview fails in the lab, with the following error:

(most recent call last):
  File "/usr/lib/python2.7/dist-packages/calendar_app/tests/test_monthview.py", line 59, in test_monthview_go_to_today_next_month
    self._test_go_to_today(1)
  File "/usr/lib/python2.7/dist-packages/calendar_app/tests/test_monthview.py", line 49, in _test_go_to_today
    self.change_month(delta)
  File "/usr/lib/python2.7/dist-packages/calendar_app/tests/test_monthview.py", line 43, in change_month
    Eventually(Equals(after.month)))
MismatchError: After 10.0 seconds test failed: 10 != 9

A full trace can be seen here:

http://reports.qa.ubuntu.com/smokeng/saucy/image/3963/calendar-app-autopilot/340014/
http://reports.qa.ubuntu.com/smokeng/saucy/image/3963/calendar-app-autopilot/

[Nicholas Skaggs (nskaggs)]

This seems to work fine in the VM; videos would be so nice for this :-(

http://91.189.93.70:8080/job/generic-mediumtests/269/console

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:ubuntu-calendar-app at revision 114, scheduled for release in ubuntu-calendar-app, milestone alpha-1

[Nicholas Skaggs (nskaggs)]

Tests are still failing in lab, changing status to reflect

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:ubuntu-calendar-app at revision 117, scheduled for release in ubuntu-calendar-app, milestone alpha-1

[Nicholas Skaggs (nskaggs)]

This still fails on the latest tests, third time is the charm Olivier :-)

[Olivier Tilloy (osomon)]

So it seems the first test in the suite (calendar_app.tests.test_calendar.TestMainView.test_new_event) hangs towards the end of its execution, resulting in screen blanking on devices (by lack of simulated user interaction) and thus all the subsequent tests fail.

I instrumented the test, and it appears the test hangs in MainView.get_title_label(…). I’ll dig further.

[Olivier Tilloy (osomon)]

The implementation of get_title_label(…) does this:

    labels = self.select_many("Label", text=title)

Apparently, autopilot silently chokes on this, I suspect because there are too many Labels in the tree. Removing the "text=title" parameter doesn’t help.
However, restricting the root of the search to the day view does the trick:

    labels = self.get_day_view().select_many("Label", text=title)

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:ubuntu-calendar-app at revision 120, scheduled for release in ubuntu-calendar-app, milestone alpha-1

[Nicholas Skaggs (nskaggs)]

Olivier, I think we should get some feedback from AP in these cases, I'll chase this down with the autopilot folks.

[Olivier Tilloy (osomon)]

After thinking a bit more about the issue, and considering that the first test in the suite eventually succeeded, my guess is that autopilot ended up finding the label, but it took a very long time to do so, enough for the screen blanking to kick in and compromise the results of the subsequent tests.

So it’s not really an issue in autopilot itself (although it could be made faster), the problem was in the test itself (and probably in the app that has a huge hierarchy of instantiated objects at all times, making it generally slow).