Comment 2 for bug 1160324

First of all, the link pasted in bug description has expired or be removed by LAVA team:

https://validation.linaro.org/dashboard/streams/private/team/linaro/android-daily/bundles/bf6955222bc5e018da0094e03d82d34f0c59e00f/

500 Internal Server Error: [Errno 2] No such file or directory: u'/srv/lava/instances/production/var/lib/lava-server/media/bundles/bundle-61382'

I have however tried CTS locally on my ARM Versatile Express A9 board with Linaro Android build #321:

https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro/#build=321

By configuring ADB over Ethernet with port 5555 (default port), I managed to run 4 test cases there:

05-10 23:26:01 I/192.168.1.70:5555: XML test result file generated at 2014.05.10_23.15.14. Passed 3, Failed 1, Not Executed 24594

In Google's CTS user manual:

https://source.android.com/compatibility/android-cts-manual.pdf

In 3.3 Setting up your device section, when doing Item 14, both android.deviceadmin.cts.CtsDeviceAdminReceiver* can't be enabled. By clicking either of them, pop up window shows, then after clicked Activate button, the tick box is still empty when UI backs to the previous screen.

Those 4 test cases are:
$ ./cts-tradefed
Android CTS 4.4_r1
05-10 23:15:01 I/: Detected new device 192.168.1.70:5555
cts-tf > run cts --plan CTS
05-10 23:15:14 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r1' on device 192.168.1.70:5555
05-10 23:15:14 I/192.168.1.70:5555: Created result dir 2014.05.10_23.15.14
05-10 23:16:44 I/192.168.1.70:5555: Collecting device info
05-10 23:23:07 I/192.168.1.70:5555: -----------------------------------------
05-10 23:23:07 I/192.168.1.70:5555: Test package android.aadb started
05-10 23:23:07 I/192.168.1.70:5555: -----------------------------------------
05-10 23:23:55 I/192.168.1.70:5555: com.android.cts.aadb.TestDeviceFuncTest#testBugreport PASS
05-10 23:23:56 I/192.168.1.70:5555: com.android.cts.aadb.TestDeviceFuncTest#testExecuteShellCommand PASS
05-10 23:23:59 I/192.168.1.70:5555: com.android.cts.aadb.TestDeviceFuncTest#testGetLogcat_size PASS
05-10 23:26:00 I/192.168.1.70:5555: com.android.cts.aadb.TestDeviceFuncTest#testGetScreenshot FAIL
com.android.tradefed.device.DeviceUnresponsiveException: Attempted screenshot multiple times on device 192.168.1.70:5555 without communication success. Aborting.
    at com.android.tradefed.device.TestDevice.performDeviceAction(TestDevice.java:1318)
    at com.android.tradefed.device.TestDevice.getScreenshot(TestDevice.java:1534)
    at com.android.cts.aadb.TestDeviceFuncTest.testGetScreenshot(TestDeviceFuncTest.java:335)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:622)
    at junit.framework.TestCase.runTest(TestCase.java:168)
    at junit.framework.TestCase.runBare(TestCase.java:134)
    at com.android.tradefed.testtype.DeviceTestResult$1.protect(DeviceTestResult.java:81)
    at com.android.tradefed.testtype.DeviceTestResult.runProtected(DeviceTestResult.java:56)
    at com.android.tradefed.testtype.DeviceTestResult.run(DeviceTestResult.java:85)
    at junit.framework.TestCase.run(TestCase.java:124)
    at com.android.tradefed.testtype.DeviceTestCase.run(DeviceTestCase.java:117)
    at com.android.cts.tradefed.testtype.JarHostTest$TestRunnable.run(JarHostTest.java:235)
    at com.android.tradefed.util.RunUtil$RunnableNotifier.run(RunUtil.java:279)

The ADB over Ethernet connection is always active, because I can still get ADB shell after test finished, by simply run "adb shell". The tricky point here is to set the service.adb.tcp.port to 5555, not 6565 as shows in test case description:

https://wiki.linaro.org/Platform/Android/SetupAdbOverTcp

That probably because the CTS test suite will reboot the device during the test, and the connection will be dropped since service.adb.tcp.port can't keep its value after reboot, though "--disable-reboot" in run command may help to walk around this.

For our current daily test dashboard on ARM Versatile Express A9, there is no CTS test at all:

https://validation.linaro.org/dashboard/image-reports/linaro-android_vexpress-linaro

So if we're not planning to run CTS on ARM Versatile Express A9, then I guess this Bug 1160324 can be dropped.

For detailed test result, please refer to attachment.