Some qtpim tests start failing when built against Qt 5.3.2

[Timo Jyrinki]

Update: the bug is that tests fail if the build and test run is not done inside the source tree

Some qtpim tests start failing when built against Qt 5.3.2:

https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-015/+sourcepub/4451482/+listing-archive-extra

We shouldn't be disabling additional tests, so the root causes should be found. All seem to have the following thing in common:
"ASSERT failure in QTest::fetchData(): "Test data requested, but no testdata available.", file qtestcase.cpp, line 2047"

You can test the Qt 5.3.2 yourself by following the instructions at https://wiki.ubuntu.com/Touch/QtTesting

[Timo Jyrinki (timo-jyrinki)]

Update:

I've moved the new qtpim snapshot to https://launchpad.net/~canonical-qt5-edgers/+archive/qt5-daily - you can enable that PPA in addition to the 5.3.2 PPA to use it, or you can just get the sources from there.

The reason is that that new snapshot would also change a symbol requiring a telephony-service rebuild (at least), and the rebuild fails because a unit test starts failing.

So I'd rather see the original problem fixed, ie rebuilding the current qtpim as is has test failures. The newest such build is now at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-015/+sourcepub/4485840/+listing-archive-extra - I'll follow that with a build where running the tests is disabled (commenting out one line in debian/rules) so that there's something that can be used for testing the whole PPA.

[Timo Jyrinki (timo-jyrinki)]

Vivid packages now at https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-002

[Timo Jyrinki (timo-jyrinki)]

The only qtestcase change I can find in 5.3.1 and 5.3.2 is: https://codereview.qt-project.org/#/c/88808/

[Renato Araujo Oliveira Filho (renatofilho)]

Hi timo,

I jus got the last qtpim source from git (my tree: https://<email address hidden>/qt/qtpim), after build it all tests passed on vivid.

But it only passes after install the compiled files (maybe exporting a tons of variable it should works too).

I am using a schroot with vivid and the mainstream qt packages (should work with your packages too).

Check my tests log attached.

[Timo Jyrinki (timo-jyrinki)]

Renato: you'd need the Qt 5.3.2 from the vivid PPA to see the problem. It does not happen with 5.3.0 or vivid only. So please upgrade with https://wiki.ubuntu.com/Touch/QtTesting 's instructions to the 5.3.2 (landing-002 for vivid) and try again.

Also, please preferably test not the git trunk but what we currently have and are trying to rebuild against 5.3.2 (probably both will show the same problem though, as I already tested one git trunk earlier and had same errors and then some more):

dget https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-002/+files/qtpim-opensource-src_5.0%7Egit20140515%7E29475884-0ubuntu5%7Evivid1%7Etest2.dsc

If you want a bazaar environment to more easily use 'bzr bd' etc, you can do the following:
wget https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-002/+files/qtpim-opensource-src_5.0%7Egit20140515%7E29475884.orig.tar.gz
bzr branch lp:~kubuntu-packagers/kubuntu-packaging/qtpim-opensource-src qtpim
cd qtpim
bzr bd

Lastly, the rechecking the latest build logs from vivid 5.3.2 PPA does show two other test failures also besides those "no testdata available" ones:
1. ---
FAIL! : tst_QOrganizerCollection::datastream() 'om.saveCollection(&collectionIn)' returned FALSE. ()
   Loc: [tst_qorganizercollection.cpp(309)]
2. ---
PASS : tst_QVersitOrganizerImporter::testTimeZones(no dst)
FAIL! : tst_QVersitOrganizerImporter::testTimeZones(dst area in standard time) Compared values are not the same
   Actual (actualDatetime): 2010/05/01 23:04:05.000[UTC]
   Expected (expected) : 2010/05/02 00:04:05.000[UTC]
   Loc: [tst_qversitorganizerimporter.cpp(1923)]
PASS : tst_QVersitOrganizerImporter::testTimeZones(dst)
PASS : tst_QVersitOrganizerImporter::testTimeZones(dst specified with rdate - daylight)
FAIL! : tst_QVersitOrganizerImporter::testTimeZones(dst specified with rdate - standard) Compared values are not the same
   Actual (actualDatetime): 1997/12/15 14:00:00.000[UTC]
   Expected (expected) : 1997/12/15 15:00:00.000[UTC]
   Loc: [tst_qversitorganizerimporter.cpp(1923)]

Both were also seen in the utopic rebuild, but my attention just was in the "no testdata found" failures.

[Timo Jyrinki (timo-jyrinki)]

Renato's analysis was at the end that it works locally, if just the qtpim packages are installed system wide before running the tests. This is a change in behavior from when qtpim was built against Qt 5.3.0, where such a requirement did not exist.

This bug should remain open as the core issue should be fixed, but as tests pass locally it could be ok to temporarily disable those additional tests in qtpim.

[Timo Jyrinki (timo-jyrinki)]

A log of a successful run of tests locally without disabling extra tests, when qtpim libraries and the QML modules are already installed.

(a cmake related test was skipped, since that succeeds on the builders but not if using the build directory, a path parsing upstream problem)