plainbox/impl/exporter/xlsx.py KeyError: 'description'

[Taihsiang Ho]

ubuntu 12.04.5 on Dell Latitude E7240 (CID 201304-13407)

Steps to reproduce this bug:
1. canonical-certification-cli
2. select whitelist cert-12.04.4
3. complete the whole tests

Notes:
Only few tests were run would not reproduce this bug.
For example, if you want to select one or two tests only,
the report could be generated and submitted successfully.
How many tests should be selected to reproduce this bug is still unknown.
Need further tests and study.

Expected result:
The test report will be generated and be submitted to certification.canonical.com

Actual result:
Right after the final tests, the stress related tests, completed. The program crashed because of the python key error and stopped. Trying to resume the session will raise the same error so I can not generate the report and submit it.

------------------------------

More info:

package status:

ubuntu@201304-13407:~$ dpkg -l | grep -e checkbox -e plainbox -e canonical
ii canonical-certification-client 0.6~ppa~ubuntu12.04.1 Client Certification Launcher
ii checkbox 0.17.9.1~ubuntu12.04.1 System testing application
ii checkbox-gui 0.23~ppa~ubuntu12.04.1 QML based interface for system testing based on PlainBox.
ii checkbox-ng 0.9~ppa~ubuntu12.04.1 PlainBox based test runner
ii checkbox-ng-service 0.9~ppa~ubuntu12.04.1 CheckBox D-Bus service
ii checkbox-qt 0.13.10 QT4 interface for checkbox
ii plainbox-glmark2-es2-meta 0.6~ppa~ubuntu12.04.1 metapackage to selectively install glmark2-es2
ii plainbox-insecure-policy 0.11~ppa~ubuntu12.04.1 policykit policy required to use plainbox (insecure version)
ii plainbox-provider-certification-client 0.6~ppa~ubuntu12.04.1 Client Certification
ii plainbox-provider-checkbox 0.10~ppa~ubuntu12.04.1 CheckBox provider for PlainBox
ii plainbox-provider-resource-generic 0.8~ppa~ubuntu12.04.1 CheckBox generic resource jobs provider
ii python3-checkbox 0.17.9.1~ubuntu12.04.1 CheckBox python3 library
ii python3-checkbox-ng 0.9~ppa~ubuntu12.04.1 PlainBox based test runner (Python 3 library)
ii python3-checkbox-support 0.8~ppa~ubuntu12.04.1 collection of Python modules used by PlainBox providers
ii python3-plainbox 0.11~ppa~ubuntu12.04.1 toolkit for software and hardware testing (python3 module)

Kernel:

ubuntu@201304-13407:~$ uname -a
Linux 201304-13407 3.13.0-32-generic #57~precise1-Ubuntu SMP Tue Jul 15 03:51:20 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Distribution and flavor:

ubuntu@201304-13407:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.5 LTS
Release: 12.04
Codename: precise
ubuntu@201304-13407:~$

cache status of Plainbox:

ubuntu@201304-13407:~$ ls -alh .cache/plainbox/sessions/
total 20K
drwxrwxr-x 5 ubuntu ubuntu 4.0K Sep 4 22:48 .
drwxrwxr-x 4 ubuntu ubuntu 4.0K Sep 4 06:39 ..
lrwxrwxrwx 1 ubuntu ubuntu 57 Sep 4 06:39 last-session -> /home/ubuntu/.cache/plainbox/sessions/pbox-ilvqt4.session
drwx------ 4 ubuntu ubuntu 4.0K Sep 4 06:51 pbox-0or6kd.session
drwx------ 4 ubuntu ubuntu 4.0K Sep 4 06:40 pbox-ilvqt4.session
drwx------ 4 ubuntu ubuntu 4.0K Sep 5 06:32 pbox-tk0o70.session
ubuntu@201304-13407:~$

[Taihsiang Ho (tai271828)]

last-session -> /home/ubuntu/.cache/plainbox/sessions/pbox-ilvqt4.session
but I expect the latest session (which I want to resume) is pbox-tk0o70.session.

Is the symbolic last-session correct?

[Daniel Manrique (roadmr)]

Hi Taihsiang, could you please tar.gz and attach your .cache/plainbox/sessions directory?

Also, you said "Trying to resume the session will raise the same error". Could you please provide a capture or log of the error/crash?

Thanks!

[Daniel Manrique (roadmr)]

My hunch is that this happens because the whitelist contains a job with no description, so code like this fails:

                if self.OPTION_WITH_DESCRIPTION in self._option_list:
                    self.worksheet4.write(
                        self._lineno, level + 1,
                        result_map[job]['description'], self.format15)

If the job contains no 'description' attribute, then things will break.

description is used like this in 3 places in xlsx.py.

[Daniel Manrique (roadmr)]

I see there are 12 jobs with no description in the checkbox and resource providers:

jobs/info.txt.in:68-71: advice: job 'meminfo_attachment', field 'description', all jobs should have a description
jobs/stress.txt.in:161-165: advice: job 'stress/reboot_log', field 'description', all jobs should have a description
jobs/stress.txt.in:179-182: advice: job 'stress/reboot_30_log', field 'description', all jobs should have a description
jobs/stress.txt.in:196-200: advice: job 'stress/poweroff_log', field 'description', all jobs should have a description
jobs/stress.txt.in:214-217: advice: job 'stress/poweroff_30_log', field 'description', all jobs should have a description
jobs/stress.txt.in:226-231: advice: job 'stress/reboot_30_check_log', field 'description', all jobs should have a description
jobs/stress.txt.in:240-245: advice: job 'stress/poweroff_30_check_log', field 'description', all jobs should have a description
jobs/stress.txt.in:254-259: advice: job 'stress/reboot_check_log', field 'description', all jobs should have a description
jobs/stress.txt.in:268-273: advice: job 'stress/poweroff_check_log', field 'description', all jobs should have a description
jobs/stress.txt.in:328-332: advice: job 'stress/network_restart_log', field 'description', all jobs should have a description

jobs/resource.txt:79-84: advice: job 'dmi', field 'description', all jobs should have a description
jobs/resource.txt:86-90: advice: job 'efi', field 'description', all jobs should have a description

Now, the only ones in the 12.04.4 cert whitelist are meminfo_attachment, dmi and efi.

However, I was unable to reproduce this problem by using the jobs in this whitelist, so I am still unsure what's happening.

[Daniel Manrique (roadmr)]

Tai, could you try applying this patch and running canonical-certification-cli again, to see if it helps?

=== modified file 'plainbox/plainbox/impl/exporter/xlsx.py'
--- plainbox/plainbox/impl/exporter/xlsx.py 2014-06-03 11:09:20 +0000
+++ plainbox/plainbox/impl/exporter/xlsx.py 2014-09-08 21:42:37 +0000
@@ -474,7 +474,7 @@
                 if self.OPTION_WITH_DESCRIPTION in self._option_list:
                     self.worksheet4.write(
                         self._lineno, level + 1,
- result_map[job]['description'], self.format15)
+ result_map[job].get('description',"PROBLEM"), self.format15)
                 if level:
                     self.worksheet3.set_row(
                         self._lineno, 13, None, {'level': level})
@@ -540,7 +540,7 @@
                         result_map[job]['io_log'].encode()
                     ).decode('UTF-8').rstrip()
                 io_lines = len(io_log.splitlines()) - 1
- desc_lines = len(result_map[job]['description'].splitlines())
+ desc_lines = len(result_map[job].get('description', "PROBLEM2").splitlines())
                 desc_lines -= 1
                 self.worksheet3.write(
                     self._lineno, max_level + 3, io_log,
@@ -548,7 +548,7 @@
                 if self.OPTION_WITH_DESCRIPTION in self._option_list:
                     self.worksheet4.write(
                         self._lineno, max_level + 2,
- result_map[job]['description'],
+ result_map[job].get('description',"PROBLEM3"),
                         self.format16 if self._lineno % 2 else self.format17)
                 if level:
                     self.worksheet3.set_row(

To apply the patch (it's also here http://paste.ubuntu.com/8293781/), save that to a file on the system (say, /tmp/patch.patch) and then:

cd /usr/lib/python3/dist-packages/plainbox/impl/exporter/
sudo patch -p4 </tmp/patch.patch

If that doesn't work, you could simply try removing the xlsx.py file from /usr/lib/python3/dist-packages/plainbox/impl/exporter/ as a workaround. This will allow you to finish the tests you're doing.

I would still appreciate seeing more diagnostic information; if the patch I posted here solves the problem then we can implement that anyway.

As a reference, see revision 2830 for robustness improvements done in the xml exporter, which may also be useful here.

[Taihsiang Ho (tai271828)]

Daniel, thanks for your workaround.
It works and the result of the session pbox-tk0o70.session was submitted to
https://certification.canonical.com/hardware/201304-13407/submission/100400/

I also attached the .cache/.plainbox as the attachment, but
please note the timestamp of the files was changed because
I made the tar ball from the copied folder as a backup when the program crashed.
The session pbox-tk0o70.session was the session to which I applied full test based on 12.04.4 cert white list and
crashed as what I have described in the bug report.

[Taihsiang Ho (tai271828)]

the attachment, plainbox.tar.gz, was the .cache/plainbox mentioned in the comment #6.

[Taihsiang Ho (tai271828)]

Daniel,
based on your comment #4,
I tried to pick up only those tests without the descriptions,
but I could not reproduce the issue as well.

The comment #7 give you the attachment of the cache and log files.
Let's see what is the exact test item without the description.

[Daniel Manrique (roadmr)]

OK, here's the backtrace:

CRITICAL plainbox.crashes: Executable 'checkbox-launcher' invoked with Namespace(command=<checkbox_ng.commands.launcher.LauncherCommand object at 0x7f2d98db9e48>, debug_console=False, debug_interrupt=False, dont_suppress_output=False, dry_run=False, launcher='/usr/bin/canonical-certification-cli', log_level=None, not_interactive=False, pdb=False, trace=[]) has crashed
Traceback (most recent call last):
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/clitools.py", line 529, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/clitools.py", line 525, in dispatch_command
    return ns.command.invoked(ns)
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/launcher.py", line 84, in invoked
    launcher).run()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 112, in run
    return self.do_normal_sequence()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 165, in do_normal_sequence
    self.export_and_send_results()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 340, in export_and_send_results
    exporter.dump(data_subset, stream)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/exporter/xlsx.py", line 634, in dump
    self.write_results(data)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/exporter/xlsx.py", line 592, in write_results
    self._write_job(tree, data['result_map'], max_level)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/exporter/xlsx.py", line 488, in _write_job
    self._write_job(children, result_map, max_level, level + 1)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/exporter/xlsx.py", line 543, in _write_job
    desc_lines = len(result_map[job]['description'].splitlines())
KeyError: 'description'
Traceback (most recent call last):
  File "/tmp/zygoto2/bin/checkbox-launcher", line 9, in <module>
    load_entry_point('checkbox-ng==0.9c2', 'console_scripts', 'checkbox-launcher')()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/main.py", line 62, in launcher
    raise SystemExit(CheckboxLauncherTool().main(argv))
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/clitools.py", line 304, in main
    return self.dispatch_and_catch_exceptions(ns)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/clitools.py", line 529, in dispatch_and_catch_exceptions
    return self.dispatch_command(ns)
  File "/src/checkbox/bzr/v010/plainbox/plainbox/impl/clitools.py", line 525, in dispatch_command
    return ns.command.invoked(ns)
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/launcher.py", line 84, in invoked
    launcher).run()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 112, in run
    return self.do_normal_sequence()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 165, in do_normal_sequence
    self.export_and_send_results()
  File "/src/checkbox/bzr/v010/checkbox-ng/checkbox_ng/commands/newcli.py", line 340, in export_and_send_results
    exporter.dump(data_subset, st...

[Daniel Manrique (roadmr)]

OK, I found it \o/

The version of the provider you're using is plainbox-provider-checkbox 0.10. I checked out that version of the checkbox trunk and found that it's crashing on the graphics/glxgears test. This is in the graphics.legacy.txt job, and the problem is that description is blank (so the JobDefinition probably lacks that attribute). This in turn is due to a problem I introduced when creating graphics-legacy in revision 3226, but Sam found it and we fixed it in checkbox revision 3231.

The problem is that when doing the 0.10 release, I cherry-picked revision 3226 (to add graphics-legacy) but didn't cherry-pick 3231 (as we didn't think it would have a consequence).

So the bad job definition is already fixed in trunk and should be there when we release plainbox-provider-checkbox 0.11 (scheduled for this Friday). Additionally, I'll fix the exporter's code to not assume a description is always present (it *should* be, but it's very easy to remove this assumption).

So there are two possible workarounds:

- Apply the patch I submitted before testing.
- Do not run graphics/glxgears when testing (remove it from your whitelist).

Apologies for this critical bug, which was my fault, I introduced it, then made a mistake when cherry-picking the fix.