Full test plan completed immediately when execute plainbox on A+A configs.

[Gabriel Zhi Chen]

Steps:
1. Install image on test machine (A+A).
2. Install plainbox.
3. Execute checkbox-gui, check the box 'All 14.04 Tests for Lenovo Projects' test plan.
4. Click 'Ok' button.
5. When all components listed, click 'Start Testing' button and verify the result.

Actual result:
The test plan completed immediately, the progress bar is 'NaN%'

Expect result:
Test cases could be executed one by one.

Notes:
The issue only occurred on A+A platform. For I, I+N, I+A and A, plainbox exeucted normally.
checkbox-cli does not reproduce the issue.

---------------------------------------------------------

Plainbox version:

ii plainbox 0.22~c1~ppa~ubuntu14.04.1 all toolkit for software and hardware integration testing
ii plainbox-glmark2-es2-meta 0.17~c1~ppa~ubuntu14.04.1 amd64 metapackage to selectively install glmark2-es2
ii plainbox-insecure-policy 0.22~c1~ppa~ubuntu14.04.1 all policykit policy required to use plainbox (insecure version)
ii plainbox-provider-certification-client 0.17~c1~ppa~ubuntu14.04.1 all Client Certification
ii plainbox-provider-checkbox 0.21~c1~ppa~ubuntu14.04.1 amd64 CheckBox provider for PlainBox
ii plainbox-provider-oem 0.4ubuntu52 all plainbox oem provider
ii plainbox-provider-oem-sutton 0.4ubuntu52 all plainbox oem provider for sutton
ii plainbox-provider-resource-generic 0.19~c1~ppa~ubuntu14.04.1 amd64 CheckBox generic resource jobs provider
ii python3-plainbox 0.22~c1~ppa~ubuntu14.04.1 all toolkit for software and hardware testing (python3 module)

[Sylvain Pineau (sylvain-pineau)]

Gabriel, looking at the session you attached, it seems that there no session file inside, only local jobs io logs.
I'l like to know if this error can be reproduced using just the cli interface.

Could you please try to run the same test plan (lenovo) but this time with your cli launcher? (or checkbox-cli if there's no dedicated launcher in your provider that automatically selects the lenovo testplan).

Thanks

[Sylvain Pineau (sylvain-pineau)]

In the logs I see this error from checkbox-gui:

Failed to GenerateDesiredJobList()

The corresponding code in gui-engine.cpp:

QList<QDBusObjectPath> GuiEngine::GenerateDesiredJobList()
{
    QList<QDBusObjectPath> desired_job_list;
    QDBusInterface iface(PBBusName, \
                         PBObjectPathName, \
                         PBInterfaceName, \
                         QDBusConnection::sessionBus());
    if (!iface.isValid()) {
        qDebug("Could not connect to \
               com.canonical.certification.PlainBox.Service1 interface");
        return desired_job_list;
    }
    QList<QDBusObjectPath> whitelists;
    QMap<QDBusObjectPath, bool>::iterator iter = whitelist.begin();
    while (iter != whitelist.end()) {
        if (iter.value()) {
            whitelists.append(iter.key());
        }
        iter++;
    }
    QDBusReply<opath_array_t> reply = \
            iface.call("SelectJobs", \
                       QVariant::fromValue<opath_array_t>(whitelists));
    if (reply.isValid()) {
        desired_job_list = reply.value();
    } else {
        qDebug("Failed to GenerateDesiredJobList()");
    }
    return desired_job_list;
}

So the reply we get over dbus seems to be invalid.
Again having the same testplan executed with checkbox-cli will help debugging this issue.

[Zygmunt Krynicki (zyga)]

Is "plainbox" actually plainbox or some other application here.

I moved this to checkbox-gui as I suspect all those issues to be GUI-related.

[Gabriel Zhi Chen (gabrielzchen)]

Sylvain and Zygmunt, I am sorry for incorrect terminology, that make confusing.

Yes, the issue occurred on 'checkbox-gui'. I also executed plianbox via 'checkbox-cli', the issue could not reproduce.

[Gabriel Zhi Chen (gabrielzchen)]

<spineau> gabrielc: hello
<gabrielc> spineau, hi
<spineau> gabrielc: thanks for the bugs that you submitted after testing the release candidates
* yuning_ is now known as yuning-mtg
<ePierre> spineau, I didn't test checkbox-gui last week when I retested the fixed issues
<gabrielc> spineau, my duty.
<spineau> gabrielc: as they are all due to checkbox-gui I have to decide if we'll fix them as checkbox-touch will soon be a viable replacement
<gabrielc> spineau, sorry for incorrect terminology
<spineau> gabrielc: no problem
<gabrielc> spineau, ok
<spineau> gabrielc: I'd like to confirm that the core dump and the behavior with A+A are regressions or not
<spineau> gabrielc: if possible could you try the same testplans with the stable release instead?
<spineau> gabrielc: I also need to know how you're starting checkbox-gui (boxer and something else) and which testplan/provider was used
<gabrielc> spineau, I tested every release under checkbox-gui. Last public release is good.
<spineau> gabrielc: I tested the certification testplans without any issue so it's maybe related to your testplan
<spineau> gabrielc: even A+A behaves correctly with the last stable?
<gabrielc> spineau, yes, It is good when test last stable release on A+A
<gabrielc> spineau, or let me double confirm that?
<spineau> gabrielc: but running the cli version works (with the rc), that's really weird
<gabrielc> spineau, ok, I will run another full test on A+A and let you know the result for core dump and the behavior with A+A
<gabrielc> spineau, I'll try both cli and gui
<spineau> gabrielc: just focus on gui, that's where bugs are
<spineau> gabrielc: one thing you could do to help debugging
<spineau> gabrielc: is to open two terminals
<spineau> gabrielc: and start the dbus service before the gui
<spineau> gabrielc: so in terminal one type checkbox-gui-service
<spineau> gabrielc: and in terminal two start checbox-gui as usual
<spineau> gabrielc: could you just tell me which testplan did you select?
<spineau> gabrielc: with the two terminal if something logs a warning on stdout will quickly know what's going wrong
<gabrielc> spineau, 'All 14.04 tests for lenovo projects', that is ready for Sutton project
<gabrielc> spineau, ok. thank you.

[Gabriel Zhi Chen (gabrielzchen)]

@Sylvain, first of all, I mis-remembered. Last time I test plainbox release under I+A rather than A+A, I+A is good. Sorry about that.

I tried the public PPA on A+A yesterday, got the same result. The issue can be reproduced

Following your words, before execute '$ checkbox-gui' , I executed '$ checkbox-gui-service' on another terminal first. Please refer to Screenshot.png in attachment.

[Gabriel Zhi Chen (gabrielzchen)]

Another result:

1. Open a terminal, type '$ checkbox-gui-service'
2. Execute '$ canonical-certificaiton-client'
3. Select 'All 14.04 Tests for Lenovo Projects' test plan.
4. Get following output.

----------------------------------------------------------------------------------------

u@u:~$ checkbox-gui-service
WARNING plainbox.ctrl: Automatically breaking via-cycle: Resources -> Resources
ERROR checkbox.ng.dbus_ex: Unable to read property <dbus.service.property 'io_log'> from <checkbox_ng.service.JobResultWrapper at /plainbox/DiskJobResult/139656579751328 at 0x7f0454d39588>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/checkbox_ng/dbus_ex/service.py", line 398, in GetAll
    prop_value = prop.__get__(self, self.__class__)
  File "/usr/lib/python3/dist-packages/checkbox_ng/dbus_ex/service.py", line 198, in __get__
    return self._getf(instance)
  File "/usr/lib/python3/dist-packages/checkbox_ng/service.py", line 551, in io_log
    return dbus.types.Array(self.native.get_io_log(), signature="(dsay)")
  File "/usr/lib/python3/dist-packages/plainbox/impl/result.py", line 508, in get_io_log
    with GzipFile(record_path, mode='rb') as gzip_stream, \
  File "/usr/lib/python3.4/gzip.py", line 181, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/home/u/.cache/plainbox/sessions/pbox-57885f7e.session/io-logs/2013.com.canonical.certification____resource__.record.gz'

[Gabriel Zhi Chen (gabrielzchen)]

I tried 20 times, the bug #1471721 is always reproduced by checkbox-gui. But never see the ERROR output again from checkbox-gui-service...

There is another A+A, execute 'canonical-certification-client', the issue reproduced. Same thing, can never see the Error output again from checkbox-gui-service.