Qt6 QLibraryInfo returns wrong paths

[tom stevens]

[Impact]

1. QLibraryInfo can return non-existent paths.
2. This results in Qt6 QWebEngine being unusable.

[Test Plan]

1. Check the output of qtdiag from qt6-tools-dev-tools, and verify the paths exist.
a) Run "qtdiag --no-vulkan --no-rhi --no-gl | grep Path"
b) verify that the paths actually exist, in particular DataPath: /share/qt6 does not exist on jammy.

2. Build the Qt supplied simplebrowser example, and verify it launches without error
a) apt-get source libqt6webenginecore6
b) cd qt6-webengine-6.2.4+dfsg/examples/webenginewidgets/simplebrowser
c) mkdir bld
d) cd bld
e) cmake -DCMAKE_BUILD_TYPE=Release ..
f) cmake --build .
g) ./simplebrowser
An example of output from a failure is attached as faillog.txt. In particular note the failure to find resources at /share:
  Qt WebEngine resources not found at /share/qt6/resources. Trying parent directory...
  Qt WebEngine resources not found at /share/qt6. Trying application directory...
and the resulting segmentation fault:
  Segmentation fault (core dumped)

[Where problems could occur]

The change in 6.2.4+dfsg-8 restores the configuration settings that were used to build qtbase-opensource-src (5.14.2+dfsg-3) through qtbase-opensource-src (5.15.3+dfsg-2). The setting was lost on the transition from the Qt5 qmake build to the Qt6 cmake build.

The change causes QLibraryInfo to use paths known at build time instead of trying to dynamically establish the paths. This will fail if the package is relocated to a different location from that which it is build for. Since the package isn't relocatable this shouldn't be a problem.

[Other Info]

The change is currently in The Kinetic Kudu.

[Original Description]

The qtdiag program can be used to find the paths returned by QLibraryInfo:

qtdiag --no-vulkan --no-rhi --no-gl

In this output on jammy (22.04) with qt6-base-dev (6.2.4+dfsg-2ubuntu1) I see:
Library info:
  PrefixPath: /
  DocumentationPath: /share/qt6/doc
  HeadersPath: /include/x86_64-linux-gnu/qt6
  LibrariesPath: /lib/x86_64-linux-gnu
  LibraryExecutablesPath: /lib/qt6/libexec
  BinariesPath: /lib/qt6/bin
  PluginsPath: /lib/x86_64-linux-gnu/qt6/plugins
  QmlImportsPath: /lib/x86_64-linux-gnu/qt6/qml
  ArchDataPath: /lib/x86_64-linux-gnu/qt6
  DataPath: /share/qt6
  TranslationsPath: /share/qt6/translations
  ExamplesPath: /lib/x86_64-linux-gnu/qt6/examples
  TestsPath: /tests
  SettingsPath: /etc/xdg

This is problematic. For example, the DataPath /share/qt6 does not exist. This can lead to various failures. For example a Qt application using the Qt WebEngine might report:

Qt WebEngine resources not found at /share/qt6/resources. Trying parent directory...
Qt WebEngine resources not found at /share/qt6. Trying application directory...
Qt WebEngine resources not found at /app/gpsbabel/gui/GPSBabelFE. Trying fallback directory... The application MAY NOT work.
Path override failed for key base::DIR_QT_LIBRARY_DATA and path '/home/tsteven4/.GPSBabel'
Installed Qt WebEngine locales directory not found at location /share/qt6/translations/qtwebengine_locales. Trying application directory...
Qt WebEngine locales directory not found at location /app/gpsbabel/gui/GPSBabelFE/qtwebengine_locales. Trying fallback directory... Translations MAY NOT not be correct.
Path override failed for key ui::DIR_LOCALES and path '/home/tsteven4/.GPSBabel'
[0423/194717.040426:ERROR:resource_bundle.cc(911)] Failed to load /home/tsteven4/.GPSBabel/qtwebengine_resources_100p.pak
Some features may not be available.
[0423/194717.041440:ERROR:resource_bundle.cc(911)] Failed to load /home/tsteven4/.GPSBabel/qtwebengine_resources_200p.pak
Some features may not be available.
[0423/194717.041481:ERROR:resource_bundle.cc(911)] Failed to load /home/tsteven4/.GPSBabel/qtwebengine_resources.pak
Some features may not be available.
[0423/194717.042243:WARNING:resource_bundle_qt.cpp(119)] locale_file_path.empty() for locale
[3004:3004:0423/194717.389096:ERROR:extension_system_qt.cpp(121)] Failed to parse extension manifest.
Segmentation fault (core dumped)

I note that qtdiag reports the correct paths on debian sid and qt6-base-dev (6.2.4+dfsg-4):

Library info:
  PrefixPath: /usr
  DocumentationPath: /usr/share/qt6/doc
  HeadersPath: /usr/include/x86_64-linux-gnu/qt6
  LibrariesPath: /usr/lib/x86_64-linux-gnu
  LibraryExecutablesPath: /usr/lib/qt6/libexec
  BinariesPath: /usr/lib/qt6/bin
  PluginsPath: /usr/lib/x86_64-linux-gnu/qt6/plugins
  QmlImportsPath: /usr/lib/x86_64-linux-gnu/qt6/qml
  ArchDataPath: /usr/lib/x86_64-linux-gnu/qt6
  DataPath: /usr/share/qt6
  TranslationsPath: /usr/share/qt6/translations
  ExamplesPath: /usr/lib/x86_64-linux-gnu/qt6/examples
  TestsPath: /usr/tests
  SettingsPath: /etc/xdg

It is not clear to me if this is because of the difference in package versions, or some issue with ubuntu packaging.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: libqt6core6 6.2.4+dfsg-2ubuntu1
ProcVersionSignature: Ubuntu 5.13.0-40.45~20.04.1-generic 5.13.19
Uname: Linux 5.13.0-40-generic x86_64
ApportVersion: 2.20.11-0ubuntu82
Architecture: amd64
CasperMD5CheckResult: unknown
Date: Sat Apr 23 19:40:12 2022
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=en_US.UTF-8
SourcePackage: qt6-base
UpgradeStatus: No upgrade log present (probably fresh install)

[tom stevens (tsteven4)]

This seems to be related to the dynamic loader search path, see the comments in https://bugreports.qt.io/browse/QTBUG-82589

[tom stevens (tsteven4)]

This issue appears on jammy because /lib is a symbolic link, and the /lib/x86_64-linux-gnu entry in the loader search path is used to find libQt6Core.
The cmake generated qconfig.cpp contains this line:
#define QT_CONFIGURE_LIBLOCATION_TO_PREFIX_PATH "../../"
So, prefixFromQtCoreLibraryHelper() computes prefixdir as /lib/x86_64-linux-gnu/../../
and returns /.
Note if /usr/lib/x86_64-linux-gnu had been used to find libQt6Core by the loader, then prefixFromQtCoreLibraryHelper() would have computed prefixdir as /usr/lib/x86_64-linux-gnu/../../, and returned /usr, which would have worked.

Instead of messing with the loader search path, or messing with cmake to compute QT_CONFIGURE_LIBLOCATION_TO_PREFIX_PATH correctly given the loader search path, the attached patch just resolves any symbolic link so libdir becomes /usr/lib/x86_64-linux-gnu instead of /lib/x86_64-linux-gnu and prefixFromQtCoreLibraryHelper() returns /usr.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "qlibraryinfo_symlink.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[tom stevens (tsteven4)]

One could imagine a more robust solution where the cmake function file(REAL_PATH <path> <out-var> [BASE_DIRECTORY <dir>] [EXPAND_TILDE]) was used to guarantee that QtQmakeHelpers.cmake resolved the relative path used to compute QT_CONFIGURE_LIBLOCATION_TO_PREFIX_PATH in the same manner that the supplied patch works at run time. This isn't necessary the way the jammy build is set up, but it might be necessary in general.

[tom stevens (tsteven4)]

Here is a more complete patch. The fix is the same, but this includes the updates to changelog, patches/series, as well.

[tom stevens (tsteven4)]

This is another way to fix the problem. It was suggested in https://bugreports.qt.io/browse/QTBUG-82589?focusedCommentId=500548&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-500548: "or disable the relocating feature for your distro". That is what this patch does.

This is a debdiff for qt6-base applicable to 6.2.4+dfsg-2ubuntu1. I built this in pbuilder
and it builds successfully, and I installed it, the patch works as intended.

[tom stevens (tsteven4)]

I note that qtbase-opensource-src-5.15.3+dfsg works correctly on jammy. This is likely because the "-no-feature-relocatable" option is used with configure in debian/rules. 2-qt6-base_6.2.4+dfsg-2ubuntu2.debdiff adds the equivalent cmake option for qt6-base_6.2.4+dfsg.

[tom stevens (tsteven4)]

A bug has been opened with debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010575

[Launchpad Janitor (janitor)]

This bug was fixed in the package qt6-base - 6.2.4+dfsg-8

---------------
qt6-base (6.2.4+dfsg-8) unstable; urgency=medium

  * Team upload.
  * Disable relocatable feature, to fix paths on usr-merged systems.
    Thanks to Steven Trabert for the patch! (Closes: #1010575, LP: #1970057)
  * Mark one inline symbol as optional to fix build on Ubuntu ppc64el.

 -- Dmitry Shachnev <email address hidden> Mon, 23 May 2022 00:56:59 +0300

[tom stevens (tsteven4)]

SRU request:

Please accept qt6-base 6.2.4+dfsg-8 into jammy.

[Impact]

1. QLibraryInfo can return non-existent paths.
2. This results in Qt6 QWebEngine being unusable.

[Test Plan]

1. Check the output of qtdiag from qt6-tools-dev-tools, and verify the paths exist.
a) Run "qtdiag --no-vulkan --no-rhi --no-gl | grep Path"
b) verify that the paths actually exist, in particular DataPath: /share/qt6 does not exist on jammy.

2. Build the Qt supplied simplebrowser example, and verify it launches without error
a) apt-get source libqt6webenginecore6
b) cd qt6-webengine-6.2.4+dfsg/examples/webenginewidgets/simplebrowser
c) mkdir bld
d) cd bld
e) cmake -DCMAKE_BUILD_TYPE=Release ..
f) cmake --build .
g) ./simplebrowser
An example of output from a failure is attached as faillog.txt. In particular note the failure to find resources at /share:
  Qt WebEngine resources not found at /share/qt6/resources. Trying parent directory...
  Qt WebEngine resources not found at /share/qt6. Trying application directory...
and the resulting segmentation fault:
  Segmentation fault (core dumped)

[Where problems could occur]

The change in 6.2.4+dfsg-8 restores the configuration settings that were used to build qtbase-opensource-src (5.14.2+dfsg-3) through qtbase-opensource-src (5.15.3+dfsg-2). The setting was lost on the transition from the Qt5 qmake build to the Qt6 cmake build.

The change causes QLibraryInfo to use paths known at build time instead of trying to dynamically establish the paths. This will fail if the package is relocated to a different location from that which it is build for. Since the package isn't relocatable this shouldn't be a problem.

[Other Info]

The change is currently in The Kinetic Kudu (active development).

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dan Bryant (justdan96)]

When could we see this in Jammy?

[Robert Lipe (robertlipe)]

I'm the project leader of GPSBabel. This problem is preventing our mutual users from successfully building with modern Qt on Jammy and is thus preventing our CI from moving to Jammy. This breakage is apparently somewhat unique to Debian/Ubuntu and isn't in Qt itself, according to Thiago in the Qt project. (Citation provided: https://bugreports.qt.io/browse/QTBUG-82589?focusedCommentId=500548&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-500548)

A fix is apparently upstreamed, but isn't included and referenced here. Can you please apply

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1010575

and then confirm that GPSBabel (and any other affected Qt apps) runs correctly, per the (excellent) description given above in #11 at https://bugs.launchpad.net/ubuntu/+source/qt6-base/+bug/1970057/comments/11?

Dropping Qt5 is a high priority for GPSBabel and Jammy's Qt6 is currently so broken that we can't use it and have to keep falling back to Qt5, which is an increasingly creaky situation for us. Qt5's open source support ended last year. We'd like to be on a current, working, Qt 6.2 or 6.3.

Currently, Ubuntu is providing a broken Qt6 that we're having to work around by falling back to Qt5. This is increasingly untenable for us.

The steps in #11 are thorough and reasonable. Please install that patch, confirm the provided examples work, then graduate to confirming that GPSBabel's map function works.

This is important to GPSBabel's Ubuntu users. We're forcing a painful workaround by using an old version of Qt that isn't supported and isn't a long-term viable plan. Please take the provided update and provide patches to your base.

Thanx to all for listening and (hopefully) for taking action.

[gzotti (georg-zotti)]

Our (Stellarium's) hackaround is recommending to set a symlink /share-->/usr/share on Ubuntu 22.04 so that Qt finds its auxiliary programs. Stupid, ugly, may not be applicable on many systems, and I don't know which side effects this may have.

[Dan Bryant (justdan96)]

I've now made a PPA to backport the Qt6 versions and fixes from Kinetic to Jammy: https://launchpad.net/~justdan96/+archive/ubuntu/qt6-backports

[tom stevens (tsteven4)]

Can we please update to 6.2.4+dfsg-8 on jammy? qt6webengine is DEAD without the upstream fix in qt6base 6.2.4+dfsg-8. 6.2.4+dfsg-8 has already been imported and was used for a time on kinetic. There is a ppa (https://launchpad.net/~tsteven4/+archive/ubuntu/qt6-backports) with a very simple recipe (https://code.launchpad.net/~tsteven4/+recipe/qt6-base-daily) from existing source.

This same bug also causes other issues, e.g. applications will have difficulty finding qt6-translations-l10n. A symptom of this is button labels appearing in english instead of the local language.

thanks

[gzotti (georg-zotti)]

+1.
I don't know how to express my support for this.

[tom stevens (tsteven4)]

related bug report, with mitigation instructions using ppa:
https://github.com/GPSBabel/gpsbabel/issues/931

[Daniel Wagenaar (dawagenaar)]

+1.
I am the project leader of NotedELN. This problem is preventing me from porting NotedELN from Qt5 to Qt6.

[Dan Bryant (justdan96)]

@racb I will paste the SRU contents here:

[Impact]

 * The current version of qtbase6 in Ubuntu Jammy is not working correctly due to bug https://bugs.launchpad.net/ubuntu/+source/qt6-base/+bug/1970057. This issue has been resolved in Kinetic package qt6-base_6.2.4+dfsg-10.

[Test Plan]

 * As Tom Stevens copied into the bug report:

 1. Check the output of qtdiag from qt6-tools-dev-tools, and verify the paths exist.
 a) Run "qtdiag --no-vulkan --no-rhi --no-gl | grep Path"
 b) verify that the paths actually exist, in particular DataPath: /share/qt6 does not exist on jammy.

 2. Build the Qt supplied simplebrowser example, and verify it launches without error
 a) apt-get source libqt6webenginecore6
 b) cd qt6-webengine-6.2.4+dfsg/examples/webenginewidgets/simplebrowser
 c) mkdir bld
 d) cd bld
 e) cmake -DCMAKE_BUILD_TYPE=Release ..
 f) cmake --build .
 g) ./simplebrowser
 An example of output from a failure is attached as faillog.txt. In particular note the failure to find resources at /share:
   Qt WebEngine resources not found at /share/qt6/resources. Trying parent directory...
   Qt WebEngine resources not found at /share/qt6. Trying application directory...
and the resulting segmentation fault:
  Segmentation fault (core dumped)
   a) Check the output of qtdiag from qt6-tools-dev-tools, and verify the paths exist.
   b) Run "qtdiag --no-vulkan --no-rhi --no-gl | grep Path"
   c) verify that the paths actually exist, in particular DataPath: /share/qt6 does not exist on jammy.

[Scope]

 * Use package from Ubuntu 22.10 (Kinetic) qt6-base_6.2.4+dfsg-10, or use the patch here: https://bugs.launchpad.net/ubuntu/+source/qt6-base/+bug/1970057/+attachment/5584085/+files/1-qt6-base_6.2.4+dfsg-2ubuntu2.dsc.debdiff

[Where problems could occur]

* As Tom Stevens copied into the bug report:

* The change in 6.2.4+dfsg-8 restores the configuration settings that were used to build qtbase-opensource-src (5.14.2+dfsg-3) through qtbase-opensource-src (5.15.3+dfsg-2). The setting was lost on the transition from the Qt5 qmake build to the Qt6 cmake build.

* The change causes QLibraryInfo to use paths known at build time instead of trying to dynamically establish the paths. This will fail if the package is relocated to a different location from that which it is build for. Since the package isn't relocatable this shouldn't be a problem.

[Other Info]

 * I have already backported the package with this fix in my PPA, it is published here: https://launchpad.net/~justdan96/+archive/ubuntu/qt6-backports-minimal/+packages?field.name_filter=&field.status_filter=published&field.series_filter=jammy
 * Tom Stevens has a similar PPA: https://launchpad.net/~tsteven4/+archive/ubuntu/qt6-backports
 * As mentioned on IRC the SRU was raised separately as a bug, this has been marked as a duplicate.

[Dmitry Shachnev (mitya57)]

I have uploaded the debdiff from comment #7 to jammy, it is currently waiting for the SRU team approval in the queue:

https://launchpad.net/ubuntu/jammy/+queue?queue_state=1&queue_text=qt6-base

[Brian Murray (brian-murray)]

Hello tom, or anyone else affected,

Accepted qt6-base into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qt6-base/6.2.4+dfsg-2ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[tom stevens (tsteven4)]

I executed the test plan from the bug description successfully on jammy. The following packages from jammy-proposed where installed:

libqt6concurrent6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6core6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6dbus6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6gui6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6network6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6opengl6-dev/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed]
libqt6opengl6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6openglwidgets6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6printsupport6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6sql6-sqlite/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6sql6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6test6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6widgets6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
libqt6xml6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
qmake6-bin/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
qmake6/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
qt6-base-dev-tools/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
qt6-base-dev/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed]
qt6-gtk-platformtheme/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]
qt6-qpa-plugins/jammy-proposed,now 6.2.4+dfsg-2ubuntu1.1 amd64 [installed,automatic]

In part one all paths exist except:
  ExamplesPath: /usr/lib/x86_64-linux-gnu/qt6/examples (The path to examples upon install.)
  TestsPath: /usr/tests (The path to installed Qt testcases.)
Ubuntu doesn't have a package that installs the Qt6 examples, so the ExamplesPath is not expected to exist.
Likewise, I am not aware of any package containing any Qt testcases. The non-existence of this path is expected.
All other paths exist, including the DataPath which is necessary for Qt WebEngine.

I executed part two as well, with the exception that I used the following command instead of 2e:
cmake -G Ninja -DCMAKE_PREFIX_PATH=/usr/lib/x86_64-linux-gnu/cmake/Qt6 ..
The simplebrowser executable worked as expected.

[tom stevens (tsteven4)]

I'm not sure how to "change the tag from verification-needed-jammy to verification-done-jammy."

tags:added: verification-done-jammy
     removed: verification-needed-jammy

[Launchpad Janitor (janitor)]

This bug was fixed in the package qt6-base - 6.2.4+dfsg-2ubuntu1.1

---------------
qt6-base (6.2.4+dfsg-2ubuntu1.1) jammy; urgency=medium

  * Make QLibraryInfo tolerant of symbolic links in loader search path
    (LP: #1970057).

 -- Steven Trabert <email address hidden> Mon, 02 May 2022 13:21:10 -0600

[Robie Basak (racb)]

The verification of the Stable Release Update for qt6-base has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.