Autopilot test failures when run on device

[Alan Pope 🍺🐧🐱 🦄]

Ran the autopilot tests on the mako device running 20130801 image.

I started the test by using "phablet-test-run -n -p ubuntu-weather-app-autopilot weather" and it appeared to lock up after a while. I pressed CTRL+C and the following output was produced:

Tests running...
======================================================================
ERROR: ubuntu_weather_app.tests.test_locationmanager.TestLocationManager.test_cancel_adding_location(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Database created
Settings table added, Database upgraded to v0.2
callbacks 0x403f5521 0x403f54d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 108, in test_cancel_adding_location
    self._open_add_location_page()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 34, in _open_add_location_page
    self.assertThat(addLocPage.visible, Eventually(Equals(True)))
AttributeError: 'NoneType' object has no attribute 'visible'

======================================================================
ERROR: ubuntu_weather_app.tests.test_locationmanager.TestLocationManager.test_search_city(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Database created
Settings table added, Database upgraded to v0.2
callbacks 0x40408521 0x404084d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 78, in test_search_city
    self._open_add_location_page()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 34, in _open_add_location_page
    self.assertThat(addLocPage.visible, Eventually(Equals(True)))
AttributeError: 'NoneType' object has no attribute 'visible'

======================================================================
FAIL: ubuntu_weather_app.tests.test_locationmanager.TestLocationManagerWithLocation.test_toolbar_opened_when_location_is_defined(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
[London] returning cached data, time from last fetch: 1.484
callbacks 0x400c2521 0x400c24d9
[Hamburg] returning cached data, time from last fetch: 1.451
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 201, in test_toolbar_opened_when_location_is_defined
    self._open_location_manager()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 144, in _open_location_manager
    self.assertThat(self.main_window.get_toolbar().opened, Eventually(Equals(True)))
MismatchError: After 10.0 seconds test on Toolbar.opened failed: True != dbus.Boolean(False, variant_level=1)

======================================================================
FAIL: ubuntu_weather_app.tests.test_locationmanager.TestLocationManagerWithLocation.test_remove_location(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
callbacks 0x40097521 0x400974d9
creating surface at (0, 58) with size (768, 1222)[London] returning cached data, time from last fetch: 0.874
[Hamburg] returning cached data, time from last fetch: 0.846
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 168, in test_remove_location
    self._open_location_manager()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 144, in _open_location_manager
    self.assertThat(self.main_window.get_toolbar().opened, Eventually(Equals(True)))
MismatchError: After 10.0 seconds test on Toolbar.opened failed: True != dbus.Boolean(False, variant_level=1)

======================================================================
FAIL: ubuntu_weather_app.tests.test_settings.TestSettings.test_switch_scale(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
callbacks 0x401ad521 0x401ad4d9
[London] returning cached data, time from last fetch: 0.89
creating surface at (0, 58) with size (768, 1222)[Hamburg] returning cached data, time from last fetch: 0.855
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_settings.py", line 52, in test_switch_scale
    self._open_settings_sheet()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_settings.py", line 30, in _open_settings_sheet
    self.assertThat(self.main_window.get_toolbar().opened, Eventually(Equals(True)))
MismatchError: After 10.0 seconds test on Toolbar.opened failed: True != dbus.Boolean(False, variant_level=1)

Ran 6 tests in 1186.711s
FAILED (failures=5)

[Martin Borho (martin-borho)]

Hmm, I'm trying to get the autopilot tests to run on the device (GNex), but for all apps autopilot is failing with:

_StringException: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/test_locationmanager.py", line 21, in setUp
    super(TestLocationManager, self).setUp()
  File "/usr/lib/python2.7/dist-packages/ubuntu_weather_app/tests/__init__.py", line 34, in setUp
    self.pointing_device = Pointer(self.input_device_class.create())
  File "/usr/lib/python2.7/dist-packages/autopilot/input/__init__.py", line 354, in create
    return _pick_backend(backends, preferred_backend)
  File "/usr/lib/python2.7/dist-packages/autopilot/utilities.py", line 56, in _pick_backend
    "Unable to instantiate any backends\n%s" % '\n'.join(failure_reasons))
RuntimeError: Unable to instantiate any backends
UInput: UInputError('"/dev/uinput" cannot be opened for writing',)

[Martin Borho (martin-borho)]

Okay, after fiddling around with the permissions of /dev/uinput the autopilot tests are running and except two tests all looks fine.

And it seems the reasons why these two tests are failing on my device are more related to the test code than a general setup problem or something.

Which device are you using?

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:ubuntu-weather-app at revision 85, scheduled for release in ubuntu-weather-app, milestone alpha-1