Comment 8 for bug 1422797

Late in the day I was able to check out some logs, here are my (raw) thoughts and observations regarding what happened in case they might be of use.

Logs found here: http://rtm-dashboard.ci.ubuntu.com/smokeng/utopic/touch_stable/krillin/243:20150224:20150216-fe747ac/349/camera_app/

[camera_app.tests.test_focus.TestFocus.test_focus_invalid]
Passes, but there is the message in the log:
"""
06:43:26.599 ERROR _launcher:206 - Timed out waiting for Application with app_id 'com.ubuntu.camera_camera_3.0.0.516' to stop.
"""

[camera_app.tests.test_flash.TestCameraFlash.test_flash_hdr_mutually_exclusive]
Happens shortly afterward with the error message:

"""
Traceback (most recent call last):
  File "/home/phablet/autopilot/camera_app/tests/test_flash.py", line 24, in setUp
    super(TestCameraFlash, self).setUp()
  File "/home/phablet/autopilot/camera_app/tests/__init__.py", line 43, in setUp
    self.launch_click_installed()
  File "/home/phablet/autopilot/camera_app/tests/__init__.py", line 73, in launch_click_installed
    emulator_base=CameraCustomProxyObjectBase)
  File "/usr/lib/python3/dist-packages/autopilot/testcase.py", line 323, in launch_click_package
    return launcher.launch(package_id, app_name, app_uris)
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 283, in launch
    return super().launch(app_id, app_uris)
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 140, in launch
    state.get('message', '')
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 233, in _check_status_error
    raise RuntimeError(': '.join(message_parts))
RuntimeError: Timed out while waiting for application to launch
"""

Due to there being a screenshot with the camera running either the app is still running from previous test or has been started again.

[camera_app.tests.test_focus.TestFocus.test_focus_valid_and_disappear]
Is the next test to run, it too shows a screenshot that looks the same as the previous test so it's either the still running up or it's been launched again.

It also fails with the message:
"""
Traceback (most recent call last):
  File "/home/phablet/autopilot/camera_app/tests/test_focus.py", line 24, in setUp
    super(TestFocus, self).setUp()
  File "/home/phablet/autopilot/camera_app/tests/__init__.py", line 43, in setUp
    self.launch_click_installed()
  File "/home/phablet/autopilot/camera_app/tests/__init__.py", line 73, in launch_click_installed
    emulator_base=CameraCustomProxyObjectBase)
  File "/usr/lib/python3/dist-packages/autopilot/testcase.py", line 323, in launch_click_package
    return launcher.launch(package_id, app_name, app_uris)
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 283, in launch
    return super().launch(app_id, app_uris)
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 140, in launch
    state.get('message', '')
  File "/usr/lib/python3/dist-packages/autopilot/application/_launcher.py", line 233, in _check_status_error
    raise RuntimeError(': '.join(message_parts))
RuntimeError: Timed out while waiting for application to launch
"""

[camera_app.tests.test_gallery_view.TestCameraGalleryView.test_swipe_to_empty_gallery]
This test has a screenshot of the dash so it's either failed to launch the app or the app has finally closed down.
It fails unable to find the launched application
"""
autopilot.exceptions.ProcessSearchError: Search criteria (object path = '/com/canonical/Autopilot/Introspection', pid = 6281) returned no results
"""

The test run time stamps show it runs from: 06:43:49.918 - 06:44:26.607
We also see in the whoopsie log that inbetween those times it starts parsing a crash file:
[06:44:05] Parsing /var/crash/_usr_share_click_preinstalled_com.ubuntu.camera_3.0.0.516_camera-app.32011.crash.
Which I would take to say that apport has done it's thing by now and it's just getting uploaded?

Overview:
The test 'camera_app.tests.test_focus.TestFocus.test_focus_invalid' passes but autopilot/UbuntuAppLaunch.stop_application times out after 10 attempting to stop the application.

Subsequent attempts in the next 2 tests fail to launch the app (timing out waiting for the app to launch) then it appears that finally the application crashes (although I'm not 100% wait happens at this point).

Is there something further that autopilot should be doing when a call to UbuntuAppLaunch.stop_application times out? (note timeout is set by autopilot, not UbuntuAppLaunch)?