Autopilot test applications get killed

[Thomi Richards]

Not sure if this is the correct project to file this against, please re-target as appropriate.

I've noticed that when running unity8 on mir, running several autopilot test suites one after the other gets the phone into a state where the autopilot application under test gets killed. For example, I ran ubuntu_calculator_app test suite, and then gallery_app test suite. Half way through the gallery_app test suite, the gallery application is killed.

The exact behavior seems to change. Sometimes autopilot itself is killed, and does not produce any result log:

"""
04:44:12.668 INFO globals:50 - Starting test ubuntu_calculator_app.tests.test_simple_page.TestSimplePage.test_addition (with touch)
04:44:12.669 WARNING testcase:118 - No tracing available - install the python-autopilot-trace package!
04:44:12.672 WARNING testcase:155 - Process manager backend unavailable, application snapshot support disabled.
04:44:12.673 DEBUG __init__:90 - Backed up database
04:44:12.694 INFO __init__:156 - Launching process: ['/usr/bin/qmlscene', '-testability', '/usr/share/ubuntu-calculator-app/ubuntu-calculator-app.qml', '--desktop_file_hint=/usr/share/applications/ubuntu-calculator-app.desktop']
Killed
phablet@ubuntu-phablet:~$ echo $?
137
"""

 Sometimes the application under test is killed with SIGKILL:

"""
04:43:16.702 INFO globals:50 - Starting test gallery_app.tests.test_events_view.TestEventsView.test_select_button_cancel
04:43:16.707 WARNING testcase:118 - No tracing available - install the python-autopilot-trace package!
04:43:16.726 WARNING testcase:155 - Process manager backend unavailable, application snapshot support disabled.
04:43:16.775 INFO __init__:156 - Launching process: ['/usr/bin/gallery-app', '-testability', '--desktop_file_hint=/usr/share/applications/gallery-app.desktop', '/tmp/gallery-ap_sd']
04:43:18.384 INFO testcase:552 - waiting for process to exit.
04:43:18.390 INFO testcase:554 - Killing process 7452
04:43:18.391 INFO testcase:557 - Appears process has already exited.
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/gallery_app/tests/test_events_view.py", line 28, in setUp
    super(TestEventsView, self).setUp()
  File "/usr/lib/python2.7/dist-packages/gallery_app/tests/__init__.py", line 81, in setUp
    self.launch()
  File "/usr/lib/python2.7/dist-packages/gallery_app/tests/__init__.py", line 117, in launch_test_installed
    emulator_base=toolkit_emulators.UbuntuUIToolkitEmulatorBase)
  File "/usr/lib/python2.7/dist-packages/autopilot/testcase.py", line 284, in launch_test_application
    dbus_bus
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 187, in get_autopilot_proxy_object_for_process
    dbus_bus=dbus_bus,
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 262, in get_proxy_object_for_existing_process
    process
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 292, in _get_dbus_addresses_from_search_parameters
    % return_code
ProcessSearchError: Process exited with exit code: -9

Ran 1 test in 2.077s
FAILED (failures=1)
"""

My suspicion is that this is the OOM checker, but the memory looks fine to me:

             total used free shared buffers cached
Mem: 1.8G 1.5G 321M 0B 328K 741M
-/+ buffers/cache: 807M 1.0G
Swap: 511M 7^C

[Thomi Richards (thomir-deactivatedaccount)]

Forgot to mention: this is being run on a mako, system image #87

[Alexander Sack (asac)]

fwiw, this doesn't affect our automation as we reboot between autopilots

[Oliver Grawert (ogra)]

some top output (hit shift+M to sort by memeory and check the highest RSS rates) would be intresting for seeing the memory consumption while this happens

[Alexander Sack (asac)]

is this also happening with SF?

[Thomi Richards (thomir-deactivatedaccount)]

Ogra: Here you go:

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 2269 phablet 20 0 793m 473m 391m S 0.0 25.3 1:08.77 unity8
 2324 phablet 20 0 192m 38m 22m S 0.0 2.0 0:05.23 maliit-server
 2543 phablet 20 0 113m 29m 5940 S 0.0 1.6 0:00.85 evolution-calen
 2501 phablet 20 0 99924 10m 6628 S 0.0 0.6 0:02.90 unity-scope-hom
 2570 phablet 20 0 105m 10m 6376 S 0.0 0.6 0:01.64 unity-scope-loa
 1935 phablet 20 0 93032 10m 8388 S 0.0 0.6 0:00.54 mediascanner-se
 4312 phablet 20 0 94884 9328 7980 S 0.0 0.5 0:00.29 unity-scope-loa
 2388 phablet 20 0 97868 8280 6736 S 0.0 0.4 0:00.26 address-book-se
 2399 phablet 20 0 90724 7320 5488 S 0.0 0.4 0:00.35 evolution-sourc
 2408 phablet 20 0 98.2m 6980 5436 S 0.0 0.4 0:00.39 evolution-addre
  952 root 20 0 67844 6360 5156 S 0.0 0.3 0:03.70 powerd
 2371 phablet 20 0 29840 5512 4568 S 0.0 0.3 0:01.31 mission-control
 2387 phablet 20 0 38732 5508 4792 S 0.0 0.3 0:04.12 telepathy-ofono
 2506 phablet 20 0 88992 5476 4236 S 0.0 0.3 0:01.67 indicator-datet
 1418 root 20 0 40200 5220 4132 S 0.0 0.3 0:03.43 NetworkManager
  732 root 20 0 46764 5064 4092 S 0.0 0.3 0:00.11 service
 2513 phablet 20 0 63036 4952 2496 S 0.3 0.3 0:24.92 indicator-power

[Renato Araujo Oliveira Filho (renatofilho)]

I am getting this error:

process-stderr: {{{
/usr/bin/address-book-app: invalid option -- 't'
/usr/bin/address-book-app: invalid option -- 'e'
/usr/bin/address-book-app: invalid option -- 's'
/usr/bin/address-book-app: invalid option -- 't'
/usr/bin/address-book-app: invalid option -- 'a'
/usr/bin/address-book-app: invalid option -- 'b'
/usr/bin/address-book-app: invalid option -- 'i'
/usr/bin/address-book-app: invalid option -- 'l'
/usr/bin/address-book-app: invalid option -- 'i'
/usr/bin/address-book-app: invalid option -- 't'
/usr/bin/address-book-app: invalid option -- 'y'
terminate called after throwing an instance of 'std::system_error'
  what(): Operation not permitted
}}}

process-stdout: {{{__pthread_gettid -2}}}
test-log: {{{
21:30:57.831 INFO globals:49 - ************************************************************
21:30:57.833 INFO globals:50 - Starting test address_book_app.tests.test_contactlist.TestContactList.test_contact_list (with touch)
21:30:57.834 WARNING testcase:118 - No tracing available - install the python-autopilot-trace package!
21:30:57.951 WARNING testcase:155 - Process manager backend unavailable, application snapshot support disabled.
21:30:57.971 INFO __init__:156 - Launching process: ['/usr/bin/address-book-app', '-testability', '--desktop_file_hint=/usr/share/applications/address-book-app.desktop']
21:30:59.526 INFO testcase:552 - waiting for process to exit.
21:30:59.532 INFO testcase:554 - Killing process 3378
21:30:59.535 INFO testcase:557 - Appears process has already exited.
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/address_book_app/tests/__init__.py", line 49, in setUp
    self.launch_test_installed()
  File "/usr/lib/python2.7/dist-packages/address_book_app/tests/__init__.py", line 62, in launch_test_installed
    app_type='qt')
  File "/usr/lib/python2.7/dist-packages/autopilot/testcase.py", line 284, in launch_test_application
    dbus_bus
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 187, in get_autopilot_proxy_object_for_process
    dbus_bus=dbus_bus,
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 262, in get_proxy_object_for_existing_process
    process
  File "/usr/lib/python2.7/dist-packages/autopilot/introspection/__init__.py", line 292, in _get_dbus_addresses_from_search_parameters
    % return_code
ProcessSearchError: Process exited with exit code: -6

[I Ahmad (iahmad)]

I can confirm, it reproduces for me when running AP ui-toolkit test cases.

[Olivier Tilloy (osomon)]

I’m seeing lots of similar errors on the webbrowser-app test suite, the process return code is not always the same, but it’s always a negative integer. See for example http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/mako/91:20131010.1:20131010/4658/webbrowser-app-autopilot/472655/.

[Alexandros Frantzis (afrantzis)]

This could be related to https://bugs.launchpad.net/unity-mir/+bug/1240518 , which is fixed in the unity-mir 0.1+13.10.20131016.1-0ubuntu1 package. Is this still a problem in the latest phone image (update/dist-upgraded if you have to, to get the aforementioned unity-mir package, I am not sure if it's in an image yet).

[Daniel van Vugt (vanvugt)]

Incomplete for Mir... unless someone can point to libmir* doing something wrong, or in the stack trace of a core dump.

[Michał Sawicz (saviq)]

This is fixed now.