Hook up 'make test' with autopkgtest infrastructure

[Daniel Holbach]

daniel@daydream:~/reboot-packaging/obj-x86_64-linux-gnu$ make test
Running tests...
/usr/bin/ctest --force-new-ctest-process
Test project /home/daniel/reboot-packaging/obj-x86_64-linux-gnu
    Start 1: basic
1/1 Test #1: basic ............................***Failed 0.29 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 0.29 sec

The following tests FAILED:
   1 - basic (Failed)
Errors while running CTest
Makefile:116: recipe for target 'test' failed
make: *** [test] Error 8
daniel@daydream:~/reboot-packaging/obj-x86_64-linux-gnu$

[Nekhelesh Ramananthan (nik90)]

Marking as high priority as this is blocking replacing the old clock app with the new clock app in the default images.

[Daniel Holbach (dholbach)]

This is working now.

daniel@daydream:~/reboot-packaging/obj-x86_64-linux-gnu$ qmltestrunner -v2 -input ../tests/unit/ -import app
file:///home/daniel/reboot-packaging/tests/unit/tst_hellocomponent.qml:13:5: HelloComponent is not a type
         HelloComponent {
         ^
********* Start testing of qmltestrunner *********
Config: Using QtTest library 5.3.0, Qt 5.3.0
INFO : qmltestrunner::tst_hellocomponent::compile() entering
QWARN : qmltestrunner::tst_hellocomponent::compile()
  /home/daniel/reboot-packaging/tests/unit/tst_hellocomponent.qml produced 1 error(s):
    /home/daniel/reboot-packaging/tests/unit/tst_hellocomponent.qml:13,5: HelloComponent is not a type
  Working directory: /home/daniel/reboot-packaging/obj-x86_64-linux-gnu
  View: QQuickView, import paths:
    '/home/daniel/.local/share'
    '/usr/share'
    '/usr/local/share'
    '/usr/share/gnome'
    '/home/daniel/reboot-packaging/obj-x86_64-linux-gnu/app'
    '/usr/lib/x86_64-linux-gnu/qt5/bin'
    '/usr/lib/x86_64-linux-gnu/qt5/qml'
  Plugin paths:
    '.'

FAIL! : qmltestrunner::tst_hellocomponent::compile() HelloComponent is not a type
   Loc: [/home/daniel/reboot-packaging/tests/unit/tst_hellocomponent.qml(13)]
Totals: 0 passed, 1 failed, 0 skipped
********* Finished testing of qmltestrunner *********
daniel@daydream:~/reboot-packaging/obj-x86_64-linux-gnu$ make test
Running tests...
/usr/bin/ctest --force-new-ctest-process
Test project /home/daniel/reboot-packaging/obj-x86_64-linux-gnu
    Start 1: basic
1/1 Test #1: basic ............................***Failed 0.23 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 0.24 sec

The following tests FAILED:
   1 - basic (Failed)
Errors while running CTest
Makefile:116: recipe for target 'test' failed
make: *** [test] Error 8
daniel@daydream:~/reboot-packaging/obj-x86_64-linux-gnu$

Unfortunately we don't get much information from a "make test" run, even if I add "-v2" to the qmltestrunner bit in CMakeLists.txt, there is little indication of what went wrong. This makes debugging tests harder than it should be.

[Daniel Holbach (dholbach)]

Test output of an entire autopkgtest run.