Test runner script gives PASS on segmentation faults

Bug #1567286 reported by Andrea Bernabei on 2016-04-07
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Canonical System Image
Zoltan Balogh
ubuntu-ui-toolkit (Ubuntu)
Christian Dywan
ubuntu-ui-toolkit (Ubuntu RTM)

Bug Description


Today I mistakingly pushed a new unit test in an MR without "bzr add"ing the corresponding QML file.

CI didn't say anything about it, tests were "PASS" on all architectures, the MR was approved, and it was about to land. Later in the day I realized I had not pushed the QML file.
But still, I got all PASS when I should have clearly got FAIL.

So I manually checked the (PASSing) console log from Jenkins, and found:
- a QWARN in the test saying that it couldn't find the QML file
- a segfault in the same test, due to the previous point

The UITK test suite has a script that runs the tests and checks, beside other things, that there were no QWARN issues by tests. That check is done by parsing a per-test result .xml that is written at the end of each test.

I think what is happening in this case is that the .xml file is not being created at all because of the segfault, and the test runner is not detecting that.

So, potentially, there are other segfaults in other test results that have never been caught so far, and could uncover new test failures :)

Related branches

Andrea Bernabei (faenil) on 2016-04-07
Changed in ubuntu-ui-toolkit (Ubuntu):
assignee: nobody → Christian Dywan (kalikiana)
importance: Undecided → Critical
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu-ui-toolkit (Ubuntu):
status: New → Confirmed
Changed in canonical-devices-system-image:
importance: Undecided → Critical
assignee: nobody → Zoltan Balogh (bzoltan)
milestone: none → 11
Changed in ubuntu-ui-toolkit (Ubuntu):
status: Confirmed → Fix Committed
Changed in ubuntu-ui-toolkit (Ubuntu RTM):
importance: Undecided → Critical
status: New → Fix Committed
Changed in canonical-devices-system-image:
status: New → In Progress
Łukasz Zemczak (sil2100) wrote :

This bug was fixed in the package ubuntu-ui-toolkit 1.3.1938+15.04.20160412 in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay


ubuntu-ui-toolkit (1.3.1938+15.04.20160412) vivid; urgency=medium

  [ Tim Peeters ]
  * Hide the PageStack back button when depth == 1. Fixes LP: #1565811
  * Add header subtitle. Fixes LP: #1399289
  * Reveal the header in gallery when changing the layout to two columns.
    Fixes LP: #1556860
  * Set correct colors for disabled actions in the header.
    Fixes LP: #1393485
  * Disable tst_datepicker.qml to unblock the staging.Fixes LP: #1567840
  * Prevent invisible header from setting the flickable topMargin.
    Fixes LP: #1560419, LP: #1560458, LP: #1566231.

  [ Zsombor Egri ]
  * Move MouseTouchAdaptor into UbuntuToolkit library. Fix adaptor code for
    Xenial. Fixes LP: #1561436

  [ Christian Dywan ]
  * Don't use a separate argument to distinguish touch events. If it's touch,
    it has our overloaded methods. Fixes LP: #1530802
  * Use export_qml_dir.sh in qmlapicheck and runtest Also add Usage to runtest.
    Fixes LP: #1567286.
  * Summary style error results with sections. Fixes LP: #1568804

  [ Andrea Bernabei ]
  * Mouse filter: check if mouse is inside the area on mouse moves and
    setHovered accordingly. Fixes LP: #1566378.

  [ Timo Jyrinki ]
  * Add latest changes for GLES.
  * Add back Provides: qtdeclarative5-ubuntu-ui-toolkit-plugin to resolve
    upgrade issue. Fixes LP: #1568817
  * Fix wrapper script auto-generated by Qt that incorrectly tries to execute
    itself. Fixes LP: #1560000
  * bileto_convert_to_gles: sort and add
    qml-module-ubuntu-performancemetrics-gles.install. Fixes LP: #1569217

  * Add additional Breaks as requested by archive admin.
  * Add latest changes for GLES.

  [ Zolán Balogh ]
  * Fix the UITK test plan script.

  [ CI Train Bot ]
  * Resync trunk. added: examples/ubuntu-ui-toolkit-gallery/po/nb.po

  [ Robert Park ]
  * Inline GLES packaging. added: debian/bileto_convert_to_gles
    debian/control.gles debian/rules.gles

 -- Zoltan Balogh <email address hidden> Tue, 12 Apr 2016 11:12:44 +0000

Changed in ubuntu-ui-toolkit (Ubuntu RTM):
status: Fix Committed → Fix Released
Changed in canonical-devices-system-image:
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-ui-toolkit - 1.3.1984+16.10.20160527.2

ubuntu-ui-toolkit (1.3.1984+16.10.20160527.2) yakkety; urgency=medium

  [ Christian Dywan ]
  * Slimmer frame for TextFields: 0.5dp. Fixes LP: #1578190.

  [ Albert Astals Cid ]
  * Add override
    The override specifier (since C++11) specifies that a virtual function
    overrides another virtual function. In a member function declaration or
    definition, override ensures that the function is virtual and is overriding
    a virtual function from the base class.

  [ Tim Peeters ]
  * Fix reference error in PullToRefreshStyle. Fixes LP: #1582843
  * Mark Tab, Tabs, TabBar, PageHeadConfiguration, PageHeadSections,
    PageHeadState, ToolbarButton, ToolbarItems as deprecated in the
    documentation. Fixes LP: #1566735, LP: #1566741.

  [ CI Train Bot ]
  * Resync trunk.

 -- Zoltan Balogh <email address hidden> Fri, 27 May 2016 07:08:44 +0000

Changed in ubuntu-ui-toolkit (Ubuntu):
status: Fix Committed → Fix Released
Changed in canonical-devices-system-image:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers