Total test case numbers changing in PM-QA test suite on Samsung Arndale Linux Linaro ubuntu.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Linaro PMWG QA |
Fix Released
|
Critical
|
sanjay singh rawat |
Bug Description
On Samsung Arndale with Linux Linaro ubuntu image, the number of total test case in pm-qa test suite is changing, on different image types. Inconsistence behaviour observed:
#######
hardware pack + nano rootfs:
7 April 2014:
133 in total, all passed.
6 April 2014:
229 in total, 200 passed.
#######
hardware pack + server rootfs:
25 March 2014 (Linaro 14.03 monthly release image):
263 in total, all passed.
#######
I also made my own LAVA job submission with the hardware pack #365 but replaced nano rootfs to server rootfs #651:
http://
This time it shows the same result as LAVA job 119566, that 7 April one:
These behavours confused me and I have 3 questions here:
1. How many test cases in total of pm-qa test suite should be performed on Samsung Arndale Linux Linaro ubuntu?
2. The monthly release image for Linaro 14.03 is that pre-built image #618:
http://
which in LAVA it shows that all 263 test cases passed. Why there is a huge number (~50%) test cases dropped in the following test run? And even 29 test cases failed on that 6 April run?
3. I have walked through the git log to try to identify the root cause on both test definition and pm-qa repositories, but there is no commit related to the test cases dropping between 25 March 2014 and 7 April 2014. Is there an mechanism inside pm-qa test code to automatically decide how many test cases should be run on a specific hardware pack + rootfs combination? But if so, after I changed the rootfs from nano to server, why there are still only 133 test cases be executed?
#######
The images mentioned above can be downloaded here:
http://
http://
http://
Changed in linaro-power-qa: | |
importance: | Undecided → Critical |
assignee: | nobody → sanjay singh rawat (sanjay-rawat) |
Changed in linaro-power-qa: | |
status: | Fix Committed → Fix Released |
I checked the logs, the issue is cpuhotplug_07 subtest runs infinitely (bug 1304251) which from the logs
- hangs and exits (no test count is less)
- or returns 139 (SIGSEV) and continue running other tests (so test count increasing).
I had posted a patch for bug 1304251 which is under review, this will help to solve the issue.