[regression] unit test failures after Qt update

Bug #1551145 reported by Olivier Tilloy on 2016-02-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
qtbase-opensource-src (Ubuntu)
Critical
Timo Jyrinki
webbrowser-app (Ubuntu)
Critical
Olivier Tilloy

Bug Description

Today I upgraded my laptop running xenial, rebuilt my local copy of webbrowser-app as there had been Qt5 updates (as well as gcc updates), and now two unit tests are failing:

The following tests FAILED:
   3 - tst_DomainUtilsTests (Failed)
   6 - tst_HistoryDomainListModelTests (Failed)

qt5 was updated from 5.5.1+dfsg-14ubuntu1 to 5.5.1+dfsg-14ubuntu2, but I’m not seeing any relevant changes in https://launchpadlibrarian.net/243321175/qtbase-opensource-src_5.5.1+dfsg-14ubuntu1_5.5.1+dfsg-14ubuntu2.diff.gz.

Olivier Tilloy (osomon) wrote :

It appears that QUrl::topLevelDomain() now always returns an empty string.

Changed in webbrowser-app (Ubuntu):
status: In Progress → Confirmed
Timo Jyrinki (timo-jyrinki) wrote :

This problem, which was picked up by Qt's own unit tests when building unmofified 5.5.1+dfsg-14ubuntu1, is documented in bug #1548686 and seems to come from something beneath Qt.

Olivier Tilloy (osomon) wrote :

Similar to bug #1548686.

Olivier Tilloy (osomon) wrote :

I just downgraded qt5 packages on my laptop from 5.5.1+dfsg-14ubuntu2 to 5.5.1+dfsg-14ubuntu1 (no other packages were touched), and the issue goes away.

Olivier Tilloy (osomon) on 2016-02-29
Changed in qtbase-opensource-src (Ubuntu):
importance: Undecided → Critical
Olivier Tilloy (osomon) wrote :

qIsEffectiveTLD() does a lookup on a well-known list of suffixes generated from https://publicsuffix.org/list/public_suffix_list.dat.
The generated table in Qt’s source tree looks fine, and in fact it’s identical between those two versions of Qt.

Olivier Tilloy (osomon) wrote :

I copied the relevant bits of the Qt source code that implement qTopLevelDomain() (including the generated table in qurltlds_p.h) and rebuilt it as a standalone test program on my machine, and it seems to be working fine. Next step is to do a full rebuild of Qt to compare.

Olivier Tilloy (osomon) wrote :

Attaching the test code with relevant bits of Qt source code copied for future reference. To test, unpack in a directory, then run `qmake` then `make`, and execute the "test" binary.

On my x86-64 laptop running fully up-to-date xenial, I’m getting the following output:

    $ ./test
    QT: QUrl("http://example.org/foobar") "example.org" ""
    local: QUrl("http://example.org/foobar") "example.org" ".org"

Olivier Tilloy (osomon) wrote :

I tried installing the packages from silo 51 (5.5.1+dfsg-14ubuntu3~xenial1~test1, which is a recent full rebuild), they don’t solve the issue.

Olivier Tilloy (osomon) wrote :

My gdb-fu is weak and I haven’t managed to step into qTopLevelDomain() (other than in machine instructions, which I cannot make sense of).

Bill Filler (bfiller) on 2016-03-01
Changed in qtbase-opensource-src (Ubuntu):
assignee: nobody → Timo Jyrinki (timo-jyrinki)
Olivier Tilloy (osomon) wrote :

I finally managed to use gdb to step into Qt code, and what I’m seeing in qTopLevelDomain() is that for e.g. "google.com" the sections are correctly extracted as ["google", "com"], and the for loop calls into qIsEffectiveTLD() for "com" (which should return true because "com" is part of the list at https://publicsuffix.org/list/public_suffix_list.dat), which itself calls into containsTLDEntry(), which computes a chunk index and then compares the entry to entries in that chunk. "com" is being compared to two entries: "accident-prevention.aero" and "net.ag". Those two entries are far apart in the generated table, so this doesn’t make much sense. It looks like somehow the initial computation of the chunk in containsTLDEntry() is broken.

Olivier Tilloy (osomon) wrote :

When stepping into containsTLDEntry(), I’m seeing a value of 267 for index.

When I compute that value separately (entry="com", qt_hash(entry)=27229, tldCount=7150), I get a value of 5779.

Olivier Tilloy (osomon) wrote :

I just found out that the qtbase-opensource-src debian package regenerates src/corelib/io/qurltlds_p.h at build time from /usr/share/publicsuffix/effective_tld_names.dat, which is provided by the publicsuffix package.

Looking at the contents of /usr/include/x86_64-linux-gnu/qt5/QtCore/5.5.1/QtCore/private/qurltlds_p.h (provided by qtbase5-private-dev), they widely differ from the version in the source tree, and indeed the "com" suffix isn’t in there, so that file is clearly corrupted.

Timo Jyrinki (timo-jyrinki) wrote :

I've tried to build qtbase against older versions of a) gccgo-6 (libgcc1), b) gcc5, c) glibc, without change in behavior.

I've now been building against older gnutls28 and publicsuffix, of which the latter indeed could affect this.

Timo Jyrinki (timo-jyrinki) wrote :

Ok it seems like a grep update on 2016-02-17 broke other things too.

Fixing via my guess that is apparently also the correct solution.

Timo Jyrinki (timo-jyrinki) wrote :
Changed in qtbase-opensource-src (Ubuntu):
status: New → In Progress
Olivier Tilloy (osomon) wrote :

This turns out to be a regression in grep (bug #1547466). Building with a UTF-8 locale doesn’t exhibit the problem, but building with LANG=C LC_ALL=C does.

Olivier Tilloy (osomon) on 2016-03-03
Changed in webbrowser-app (Ubuntu):
status: Confirmed → Invalid
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qtbase-opensource-src - 5.5.1+dfsg-14ubuntu3

---------------
qtbase-opensource-src (5.5.1+dfsg-14ubuntu3) xenial; urgency=medium

  * debian/patches/Xcb-fix-crash-on-screen-power-save.patch:
    - Fix additional cases where crash could occur on monitor disconnections
      or power save. (LP: #1551122)
  * Build with C.UTF-8 locale to (LP: #1551145) and re-enable tests (LP: #1548686)

 -- Timo Jyrinki <email address hidden> Wed, 02 Mar 2016 10:28:04 +0000

Changed in qtbase-opensource-src (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers