Timeout happens when run the benchmark apks monkeyrunner scripts

Bug #1081042 reported by Yongqin Liu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Linaro Android
Fix Released
Critical
Yongqin Liu

Bug Description

When run the benchmark apks monkeyrunner scripts,
there are some time out problems occurred on some apks.
Like the vellamo/quadrant/nbench/glbenchmark are times out on the job https://validation.linaro.org/lava-server/scheduler/job/37928 for snowball-jb-gcc47-igloo-stable-blob/#build=117

And vellamo/quadrant/nbench/glbenchmark also time out on the job https://validation.linaro.org/lava-server/scheduler/job/39171/log_file of origen-jb-gcc47-samsunglt-stable-blob/#build=126

vishal (vishalbhoj)
Changed in linaro-android:
assignee: nobody → Yongqin Liu (liuyq0307)
milestone: none → 12.12
Yongqin Liu (liuyq0307)
Changed in linaro-android:
status: New → In Progress
importance: Undecided → Medium
Zach Pfeffer (pfefferz)
Changed in linaro-android:
importance: Medium → Critical
Revision history for this message
Yongqin Liu (liuyq0307) wrote :

because the wifi is not available for 4.2, will use panda-jb-gcc47-tilt-stable-blob/#build=105 to check this problem

Revision history for this message
Yongqin Liu (liuyq0307) wrote :

mainly 4 benchmark application will fail:
 anutu / glbenchmark// quadrant/linpack

glbenchmark:
 the version is outdated, and Vishal will help to update the apk file

quadrant:
mainly timeout for sending result, increase the wait time to 360s from 120s

linpack:
not know why some time it will stop to run, maybe the focus is not right.
will change to use uiautomator to do the run options

antutu:
there is always a version up window popped up, and focus will be wrong by monkeyrunner script.
so will change to use uiautomator for it too

Revision history for this message
Yongqin Liu (liuyq0307) wrote :

the java source of uiautomator of linpack and antutu is located here:
   git://git.linaro.org/people/yongqinliu/linaro-android.git

Revision history for this message
Yongqin Liu (liuyq0307) wrote :

change geekbench and caffeine to use uiautomator

Revision history for this message
Yongqin Liu (liuyq0307) wrote :

Have changed all the apk scripts to use uiautomater(except glbenchmark).
And now is testing them on staging with the #140 and #105.

And from the test on local, there are the possibility that we can't get the all the result displayed on android-build.
1. vellamo test some times will crash on JB4.2(build #140)
2. glbenchmark2.5.1 can't run on JB4.2(build #140) because of the lack of sdcard.
3. one of the test item of glbenchbench will crash.
    now considering to check that item out. not do it in our test.

Revision history for this message
Zach Pfeffer (pfefferz) wrote :

The last thing left to do is finish up the glbenchmark upgrade. Shifting to 13.01.

Changed in linaro-android:
milestone: 12.12 → 13.01
Revision history for this message
Yongqin Liu (liuyq0307) wrote :

source of uiautomator: http://git.linaro.org/gitweb?p=people/yongqinliu/linaro-android.git;a=summary
source of scripts: https://linaro-private.git.linaro.org/gitweb?p=people/yongqinliu/linaro-benchmarks.git;a=summary

On JB4.1 all are work except one test item of glbenchmark2.5.1
On JB4.2 now, mainly 3 problems:
1. vellamo test some times crashes
    http://staging.validation.linaro.org/dashboard/attachment/203594/download
2. glmarkbench2.5.1 can't start because of the sdcard problem
    http://staging.validation.linaro.org/dashboard/attachment/203632/download
3. quadrand sometimes will fail because of the network problem
    http://staging.validation.linaro.org/dashboard/attachment/203645/download

Revision history for this message
Yongqin Liu (liuyq0307) wrote :

because the problems here are not related to the scripts, so change this bug to fix committed.

Changed in linaro-android:
status: In Progress → Fix Committed
vishal (vishalbhoj)
Changed in linaro-android:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.