Test runner script gives PASS on segmentation faults

[Andrea Bernabei]

r1920

Today I mistakingly pushed a new unit test in an MR without "bzr add"ing the corresponding QML file.

CI didn't say anything about it, tests were "PASS" on all architectures, the MR was approved, and it was about to land. Later in the day I realized I had not pushed the QML file.
But still, I got all PASS when I should have clearly got FAIL.

So I manually checked the (PASSing) console log from Jenkins, and found:
- a QWARN in the test saying that it couldn't find the QML file
- a segfault in the same test, due to the previous point

The UITK test suite has a script that runs the tests and checks, beside other things, that there were no QWARN issues by tests. That check is done by parsing a per-test result .xml that is written at the end of each test.

I think what is happening in this case is that the .xml file is not being created at all because of the segfault, and the test runner is not detecting that.

So, potentially, there are other segfaults in other test results that have never been caught so far, and could uncover new test failures :)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Łukasz Zemczak (sil2100)]

This bug was fixed in the package ubuntu-ui-toolkit 1.3.1938+15.04.20160412 in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay

---------------

ubuntu-ui-toolkit (1.3.1938+15.04.20160412) vivid; urgency=medium

  [ Tim Peeters ]
  * Hide the PageStack back button when depth == 1. Fixes LP: #1565811
  * Add header subtitle. Fixes LP: #1399289
  * Reveal the header in gallery when changing the layout to two columns.
    Fixes LP: #1556860
  * Set correct colors for disabled actions in the header.
    Fixes LP: #1393485
  * Disable tst_datepicker.qml to unblock the staging.Fixes LP: #1567840
  * Prevent invisible header from setting the flickable topMargin.
    Fixes LP: #1560419, LP: #1560458, LP: #1566231.

  [ Zsombor Egri ]
  * Move MouseTouchAdaptor into UbuntuToolkit library. Fix adaptor code for
    Xenial. Fixes LP: #1561436

  [ Christian Dywan ]
  * Don't use a separate argument to distinguish touch events. If it's touch,
    it has our overloaded methods. Fixes LP: #1530802
  * Use export_qml_dir.sh in qmlapicheck and runtest Also add Usage to runtest.
    Fixes LP: #1567286.
  * Summary style error results with sections. Fixes LP: #1568804

  [ Andrea Bernabei ]
  * Mouse filter: check if mouse is inside the area on mouse moves and
    setHovered accordingly. Fixes LP: #1566378.

  [ Timo Jyrinki ]
  * Add latest changes for GLES.
  * Add back Provides: qtdeclarative5-ubuntu-ui-toolkit-plugin to resolve
    upgrade issue. Fixes LP: #1568817
  * Fix wrapper script auto-generated by Qt that incorrectly tries to execute
    itself. Fixes LP: #1560000
  * bileto_convert_to_gles: sort and add
    qml-module-ubuntu-performancemetrics-gles.install. Fixes LP: #1569217

  * Add additional Breaks as requested by archive admin.
  * Add latest changes for GLES.

  [ Zolán Balogh ]
  * Fix the UITK test plan script.

  [ CI Train Bot ]
  * Resync trunk. added: examples/ubuntu-ui-toolkit-gallery/po/nb.po

  [ Robert Park ]
  * Inline GLES packaging. added: debian/bileto_convert_to_gles
    debian/control.gles debian/rules.gles

 -- Zoltan Balogh <email address hidden> Tue, 12 Apr 2016 11:12:44 +0000

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-ui-toolkit - 1.3.1984+16.10.20160527.2

---------------
ubuntu-ui-toolkit (1.3.1984+16.10.20160527.2) yakkety; urgency=medium

  [ Christian Dywan ]
  * Slimmer frame for TextFields: 0.5dp. Fixes LP: #1578190.

  [ Albert Astals Cid ]
  * Add override
    The override specifier (since C++11) specifies that a virtual function
    overrides another virtual function. In a member function declaration or
    definition, override ensures that the function is virtual and is overriding
    a virtual function from the base class.

  [ Tim Peeters ]
  * Fix reference error in PullToRefreshStyle. Fixes LP: #1582843
  * Mark Tab, Tabs, TabBar, PageHeadConfiguration, PageHeadSections,
    PageHeadState, ToolbarButton, ToolbarItems as deprecated in the
    documentation. Fixes LP: #1566735, LP: #1566741.

  [ CI Train Bot ]
  * Resync trunk.

 -- Zoltan Balogh <email address hidden> Fri, 27 May 2016 07:08:44 +0000