unigine test fail without any useful info

[Jeffrey Chang]

checkbox 0.15+bzr1867+201301070448~precise1
checkbox-certification 0.15+bzr562+201212191001~precise1

3 unigine tests all failed without any info.
And I could see this graphics benchmark running without problem.

[Daniel Manrique (roadmr)]

OK, so first I ran /usr/share/checkbox/scripts/pts_run unigine-tropics. As you say, this exited with no text or warning as to what had happened.

Looking at the pts_run code, this is how it's designed to run. If any of the commands returns an error, the script will just silently exit.

Could you please manually run these commands, to see the error phoronix-test-suite is having?

export FORCE_TIMES_TO_RUN=1
export OVERRIDE_VIDEO_MODES=800x600
phoronix-test-suite batch-benchmark unigine-sanctuary

Please attach any console output you see here. In my case, it failed because it couldn't contact the phoronix server to download the test definition since I'm behind a proxy. Failure to compile the test would also cause errors and failures.

Also, I found a dependency on ia32-libs for the unigine tests. If you want these tests to run, we will need to add this to the checkbox-ihv-graphics dependencies and rebuild the packages and the tarballs.

So depending on what your results are we need to:

- Add dependencies on ia32-libs.
- Potentially flesh out the script to compensate for phoronix's "fragility" and reliance on the network.

[Jeffrey Chang (modern911)]

I got problem when install ia32-libs.
Console messages below:

The following dependencies are needed and will be installed:

- ia32-libs

This process may take several minutes.
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 ia32-libs : Depends: ia32-libs-multiarch

Phoronix Test Suite v3.6.1

    Installed: pts/unigine-sanctuary-1.5.1

Unigine Sanctuary Test Configuration

System Information

Hardware:
Processor: Intel Core i5-2450M @ 2.50GHz (4 Cores), Motherboard: Dell 03J5KR, Chipset: Intel 2nd Generation Core Family DRAM, Memory: 2048MB, Disk: 500GB Seagate ST9500423AS + 32GB SAMSUNG SSD PM81, Graphics: Intel 2nd Generation Core Family IGP 1024MB, Audio: Conexant CX20590, Network: Realtek RTL8101E/RTL8102E + Atheros AR9285 Wireless

Software:
OS: Ubuntu 12.04, Kernel: 3.5.0-22-generic (x86_64), Desktop: Unity 5.18.0, Display Server: X Server 1.13.0, Display Driver: intel 2.20.9, OpenGL: 3.0 Mesa 9.0, Compiler: GCC 4.6, File-System: ext4, Screen Resolution: 1920x1080

Estimated Run-Time: 13 Minutes

Unigine Sanctuary 2.3:
    pts/unigine-sanctuary-1.5.1 [Resolution: 800 x 600]
    Test 1 of 1
    Estimated Trial Run Count: 1
    Estimated Time To Completion: 13 Minutes
        Started Run 1 @ 03:52:30
        The test did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.

[Daniel Manrique (roadmr)]

Which version of Ubuntu are you running this on?

I tried a fresh install of 12.04.1, and I don't get the same error with ia32-libs:

$ sudo apt-get install ia32-libs
[sudo] password for ubuntu:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  bluez-alsa:i386 esound-common gcc-4.6-base:i386 glib-networking
  glib-networking:i386 glib-networking-common glib-networking-services
(big list of packages....)

Also, of note, adding the ia32-libs dependency would increase tarball size by about 70 MB.

[Daniel Manrique (roadmr)]

Yay, so I tinkered a bit with Phoronix and identified three possible use cases:

- The suite manages to correctly run
- The suite is unable to download test definitions from the internet
- The suite is unable to install dependencies (such as ia32-libs).

I'm attaching sample runs for these three cases, as reference. The script should probably be made a bit more robust to better report these errors.

[Daniel Manrique (roadmr)]

Yay, so I tinkered a bit with Phoronix and identified three possible use cases:

- The suite manages to correctly run
- The suite is unable to download test definitions from the internet
- The suite is unable to install dependencies (such as ia32-libs).

I'm attaching sample runs for these three cases, as reference. The script should probably be made a bit more robust to better report these errors.

[Daniel Manrique (roadmr)]

I proposed a branch with changes to log the full phoronix console output as the comment, this will ease debugging and will be clearer than the current output, which is just a number representing FPS. Without context this number is a bit hard to interpret.

[Jeffrey Chang (modern911)]

I'm testing on a clean OSP1 release based on 12.04.2, which will be the base of all OEM projects from now on.
And I would consider ia32-libs issue is an upstream problem. Will report later.

Another thought is
Do we still need this test in checkbox-ihv?

[Daniel Manrique (roadmr)]

Hi Jefrrey,

I tried installing 12.04.2 from today's daily image, and then installing ia32-libs. It didn't say it was unistallable, but it wants to remove a lot of x11-related packages, which I think would have left the system in a non-working state.

So you're right that this problem lies with ia32-libs and we can't really do much about it.

The branch I proposed for this bug should at least produce more debugging information so this kind of problem (phoronix unable to install packages) will be easier to spot.

We're scheduled to have a call with Chris Gregan on Wednesday, we can discuss if unigine is necessary for checkbox-ihv. Let me know either way.