HTML file shows UNCATEGORIZED as Failed with no listed failures
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Next Generation Checkbox (CLI) |
Fix Released
|
High
|
Sylvain Pineau | ||
Bug Description
Summary: Looking at the HTML output for a recent test run, the HTML shows "UNCATEGORIZED" as "Failed" but when you expand the section, there is only one thing listed, and it's marked as "Passed". Likely some resource job HAS failed, but since it's not included in the display, the output is confusing and worrysome to reviewers.
Ultimately, there should be no "Uncategorized" tests. Any actual test case should have a category and be appropriately siloed in that category. Any background job (attachment and resource jobs mostly) should be completely invisible, OR should be only displayed when failed and should also be categorized as "Resource" or "Attachment" or whatever.
Steps to reproduce: View the HTML file included in the attached submission tarball.
Expected result: HTML output is accurate
Actual result: HTML output is a bag of lies.
Failure rate: NO idea...
Related branches
- Maciej Kisielewski (community): Approve
- Sylvain Pineau (community): Needs Resubmitting
-
Diff: 727 lines (+603/-5)9 files modifiedcheckbox_ng/launcher/checkbox_cli.py (+2/-0)
checkbox_ng/launcher/merge_reports.py (+128/-0)
checkbox_ng/launcher/subcommands.py (+11/-3)
plainbox/impl/exporter/jinja2.py (+24/-0)
plainbox/impl/providers/exporters/data/Chart.min_2.7.3.js (+10/-0)
plainbox/impl/providers/exporters/data/checkbox.json (+21/-1)
plainbox/impl/providers/exporters/data/multi-page.html (+397/-0)
plainbox/impl/providers/exporters/units/exporter.pxu (+8/-1)
plainbox/impl/session/state.py (+2/-0)
| Jeff Lane (bladernr) wrote | #1 |
| Changed in checkbox-ng: | |
| milestone: | none → 1.3.0 |
| assignee: | nobody → Sylvain Pineau (sylvain-pineau) |
| importance: | Undecided → High |
| Changed in checkbox-ng: | |
| status: | New → In Progress |
| Changed in checkbox-ng: | |
| status: | In Progress → Fix Committed |
| Changed in checkbox-ng: | |
| status: | Fix Committed → Fix Released |
tarball with bad html and full submission files to compare things to