Calculator app autopilot tests fail on device

[Alan Pope 🍺🐧🐱 🦄]

I ran the autopilot tests on my device running image from 20130801 and got 7 of 8 failing...

I started the test by using "phablet-test-run -n -p ubuntu-calculator-app-autopilot ubuntu_calculator_app" and it appeared to lock up after a while. I pressed CTRL+C and the following output was produced:-

Tests running...
======================================================================
ERROR: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_click_history(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Database upgraded to 0.1.1
callbacks 0x40a4d521 0x40a4d4d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 263, in test_click_history
    self.pointing_device.click_object(calc)
  File "/usr/lib/python2.7/dist-packages/autopilot/input/__init__.py", line 536, in click_object
    self.move_to_object(object_proxy)
  File "/usr/lib/python2.7/dist-packages/autopilot/input/__init__.py", line 553, in move_to_object
    x, y = get_center_point(object_proxy)
  File "/usr/lib/python2.7/dist-packages/autopilot/input/_common.py", line 65, in get_center_point
    object_proxy)
ValueError: Object 'None' does not have any recognised position attributes

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_three_negative_numbers_multiplication(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Database created
callbacks 0x4060f521 0x4060f4d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 213, in test_three_negative_numbers_multiplication
    self._assert_result("-140")
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: '-140' != String(u'')

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_two_negatives_in_three_numbers_multiplication(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
callbacks 0x401e4521 0x401e44d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 209, in test_two_negatives_in_three_numbers_multiplication
    self._assert_result("30")
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: '30' != String(u'')

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_addition(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
callbacks 0x410a9521 0x410a94d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 181, in test_addition
    self._assert_result("6")
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: '6' != String(u'')

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_divide_priority(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
callbacks 0x4018b521 0x4018b4d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 156, in test_divide_priority
    self._assert_result("8")
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: '8' != String(u'')

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_two_negative_numbers_multiplication(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
callbacks 0x401e9521 0x401e94d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 201, in test_two_negative_numbers_multiplication
    self._assert_result("6")
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: '6' != String(u'')

======================================================================
FAIL: ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_divide_by_zero(with touch)
----------------------------------------------------------------------
_StringException: Empty attachments:
  process-stdout

process-return-code: {{{-15}}}
process-stderr: {{{
unknown option -- t
unknown option -- e
unknown option -- s
unknown option -- t
unknown option -- a
unknown option -- b
unknown option -- i
unknown option -- l
unknown option -- i
unknown option -- t
unknown option -- y
loaded the dummy plugin
loaded the Linux plugin
Registered the AalSensorPlugin types

Loading testability driver.
unity::action::ActionManager::ActionManager(QObject*):
 Could not determine application identifier. HUD will not work properly.
 Provide your application identifier in $APP_ID environment variable.
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
Fail to load themed icon for: "user-trash-symbolic"
file:///usr/lib/arm-linux-gnueabihf/qt5/qml/Ubuntu/Components/Icon.qml:77:5: QML Image: Failed to get image from provider: image://gicon/user-trash-symbolic
callbacks 0x406e9521 0x406e94d9
creating surface at (0, 58) with size (768, 1222)QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
QOpenGLShader::link: "--From Fragment Shader:
--From Vertex Shader:
Link was successful.
"
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 233, in test_divide_by_zero
    self._assert_result(u'\u221e') # Unicode for "infinite" result
  File "/usr/lib/python2.7/dist-packages/ubuntu_calculator_app/tests/test_simple_page.py", line 98, in _assert_result
    get_result, Eventually(Equals(expected_result)))
MismatchError: After 10.0 seconds test failed: u'\u221e' != String(u'')

Ran 8 tests in 616.619s
FAILED (failures=7)

[Nicholas Skaggs (nskaggs)]

I believe this to be working now, but we'll need alan to confirm it works on his device. Alan, wait a day for the build to get into the ppa and report back if it's working. Note ,the smoke tests in the lab seem to agree it's working :-) Cheers!

[Paul Larson (pwlars)]

I still count 7 failures on http://reports.qa.ubuntu.com/smokeng/saucy/image/3514/ubuntu-calculator-app-autopilot/ for today's image (20130814).

[Nicholas Skaggs (nskaggs)]

A new version of autopilot tests from the team has broken things again :-(

[Nicholas Skaggs (nskaggs)]

Negative operations are broken

https://bugs.launchpad.net/ubuntu-calculator-app/+bug/1210082

[David Planella (dpm)]

Looking at http://reports.qa.ubuntu.com/smokeng/saucy/image/3896/ it seems that the tests are passing now. Alan, could you check if you can still reproduce it, and if not, mark it as Fix Released? Thanks!

[Alan Pope 🍺🐧🐱 🦄 (popey)]

Yup! Test completed on my local device!