webapp_container tests started failing reliably with rc-proposed image #92

[Olivier Tilloy]

Running the webapp_container autopilot test suite on image #91 on krillin (ubuntu-touch/rc-proposed/bq-aquaris.en), several times in a row, all tests pass reliably.

Upgrading the device to image #92 from the same channel, running the same test suite, I consistently get 5 or 6 failures, not always on the same tests. I’m also seeing the app seemingly exiting before the failing tests complete. After running the entire test suite and getting those failures, if I swipe from the right edge to reveal the list of running apps, I’m seeing semi-transparent black surfaces (see attached screenshot). Their number seems to be the same as the number of failures. I can swipe them down to close them.

Note that the webbrowser_app test suite doesn’t exhibit this issue, only the webapp_container suite (which has a total of 20 tests).

The changes between image #91 and image #92 are the following: http://people.canonical.com/~lzemczak/landing-team/ubuntu-touch/rc-proposed/92.commitlog.

I’m initially filing this bug against webbrowser-app (as it is the source package for webapp-container), but the issue likely lies elsewhere, as there was no webbrowser-app update between those two images.

[Olivier Tilloy (osomon)]

Attaching the list of changes between image #91 and image #92 for future reference, in case it disappears from people.canonical.com.

[Olivier Tilloy (osomon)]

The unity8/qtmir uploads look like they could potentially have triggered the issue.

[Olivier Tilloy (osomon)]

I’m attaching the full XML output of a typical test run on my krillin (now at image #94, same issue).

[Gerry Boland (gerboland)]

<oSoMoN> greyback, it’s only the webapp_container suite that has failing tests (only 20 tests)
<oSoMoN> the webbrowser_app suite is unaffected

<oSoMoN> greyback, in my experience it’s not always the same tests that fail reliably, but I always get ~ 6 failures out of 20 tests