[SRU] Upgrade rapid-photo-downloader to version 0.9.23a1

[Damon Lynch]

[Bug Description Edited June 2, 2020]

[Impact]

Rapid Photo Downloader 0.9.22 contains a bug that means for some users, it causes a segfault during startup every time it is launched. (Other users are unaffected. For example, I have never been able to replicate the bug on any of my systems.) There is no workaround for the bug except to upgrade it to a newer release.

This is a report from a user who encountered the problem (left in a review in Ubuntu Software):

"When I reinstalled my OS to upgrade to 20.04 (Studio), this app came preinstalled, but it wouldn't open. I tried uninstalling and reinstalling from Snap Store, but issue persisted. Installed using script from main website (Version 0.9.24) and works fine now."

The backstory: Rapid Photo Downloader is written using Qt5. Earlier this year I improved the program to function properly on HighDPI screens. A side effect of this fix was that for a handful of users on displays with no screen scaling, a bug somewhere in the X window / Qt rendering chain made the program UI utterly broken. For a screenshot, see https://discuss.pixls.us/t/graphical-issues-on-xubuntu-18-04/17405

My solution to this problem was to disable HighDPI support on systems where the screen scaling factor was set to 1.0. Why enable it when it's not needed, right? But this requires querying the screen scaling before initializing Rapid Photo Downloader's UI, because Qt5 requires HighDPI be set before instantiating QApplication. Unfortunately when running under X, Gtk and Qt/KDE use incompatible methods to enable and report screen scaling.

So in sum, Rapid Photo Downloader needs to query the system's screen scaling before initializing its own application window, and how it does that depends on whether it is running in a Gtk or Qt based desktop.

The bug in Rapid Photo Downloader 0.9.22 is caused by calling the Gtk/GDK API without first calling gtk_init() during application startup. Upon becoming aware of the bug, I (the application developer) discussed the problem with Emmanuele Bassi and Matthias Clasen, and they strong urged me to query screen scaling using XSETTINGS, and not the GTK/GDK API. They were correct. Querying XSETTINGS has fixed the bug.

[Test Case]

Because the bug in 0.9.22 only occurs on some systems and not others, it will either occur on a test system every time the program is started, or it will not. There appears to be no in-between.

When the bug does occur, a segfault always results.

If the program UI renders normally upon program startup an upgraded system version in a system in which 0.9.22 crashes, the problem is fixed.

[Regression Potential]

If there are to be any regression, it would be in querying XSETTINGS.

The actual code to query XSETTINGS is copied directly from Ubuntu and Debian developer Dmitry Shachnev's ReText program (which like Rapid Photo Downloader, is coded using Python and Qt5). Dmitry's code has been in use for five years, so it appears stable. This is the original source:

https://github.com/retext-project/retext/blob/master/ReText/xsettings.py

The call to query XSETTINGS in wrapped in try / except Python block that catches all exceptions, so the only way a problem would manifest in this code block would be a segfault. Knowing what might cause a segfault in such circumstances is well beyond my understanding of how X works.

The great news is that after the bug fix, I have had no bug reports relating to HighDPI screen scaling or application startup. Screen scaling is working for those who use HighDPI screens, and for those who are not, the program UI renders as expected. Just as importantly, no user has reported a segfault during program startup in 0.9.23 or 0.9.24 (or after startup either).

I have tested versions 0.9.23 and 0.9.24 on as many desktop environments as I have been able to, both popular and obscure, using both X and where possible Wayland, toggling screen scaling on systems that allow it. The program UI renders properly in all the environments I tested it.

Version 0.9.23 was released on 2020-04-16, and 0.9.24 on 2020-05-03. Both versions contain the bug fix for the segfault startup problem, but they also contain other bug fixes and translation updates. Given the potential of other regressions, and assuming either 0.9.23 or 0.9.24 will be used for the SRU, I will discuss them briefly here too.

I recommend 0.9.24 for the SRU. Compared to 0.9.22, 0.9.24 contains 9 other bug fixes. Every fix is localized to the specific problem itself, and does not affect any other areas of the program. For all nine fixes, the problem was resolved with no follow-up issues reported. In all honesty, while I assume there are possibilities for regressions caused by these nine fixes, I am unaware of what they are! As far as I'm concerned, the problems are fixed, and no other problems have been caused because of them.

There is a new feature in 0.9.24 which is a more realistic candidate for regressions: the feature to provide the use the option of automatically creating a tar file of the program configuration file and log files when submitting a bug report. See this documentation for an illustration:

https://damonlynch.net/rapid/documentation/#reportproblem

I tested the code extensively, and exceptions are caught when creating the tar (e.g. in case of problems such as no space left on the file system). But nonetheless, given the dialog automatically appears in case of any uncaught program exception, it's possible that an unstable program condition might somehow cause problems in the generation of the tar file. To mitigate this possibility, no compression of the files included in the tar is performed, and the tar itself is not compressed. So while a bug report tar file not being created properly is possible, I judge it unlikely, and if it does occur, any side effects should be neglible.

No user has reported a problem using this new feature (having said that, I've had only one bug report in which it was used).

[Other Info]

The full changelog for all versions is available here:

https://launchpad.net/rapid/pyqt/0.9.24/+download/CHANGES.rst

Thank you.

[Zobeid Zuma (zobeid)]

zobeid@zobeid-Thelio:~$ rapid-photo-downloader

/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: cannot register existing type 'GdkWindow'

app = QGuiApplication(sys.argv)

/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: g_param_spec_object: assertion 'g_type_is_a (object_type, G_TYPE_OBJECT)' failed

app = QGuiApplication(sys.argv)

/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: validate_pspec_to_install: assertion 'G_IS_PARAM_SPEC (pspec)' failed

app = QGuiApplication(sys.argv)

/usr/lib/python3/dist-packages/raphodo/rapid.py:5843: Warning: cannot register existing type 'GdkScreen'

super().__init__(*argv)

/usr/lib/python3/dist-packages/raphodo/rapid.py:5843: Warning: g_once_init_leave: assertion 'result != 0' failed

super().__init__(*argv)

(rapid-photo-downloader:3867): Gdk-CRITICAL **: 21:16:08.327: IA__gdk_screen_get_default_colormap: assertion 'GDK_IS_SCREEN (screen)' failed

(rapid-photo-downloader:3867): Gdk-CRITICAL **: 21:16:08.327: IA__gdk_colormap_get_visual: assertion 'GDK_IS_COLORMAP (colormap)' failed

[Damon Lynch (dlynch3)]

To clarify, Zobeid Zuma has hit the startup bug with version 0.9.22 (as packaged in Ubuntu 20.04), which was fixed in 0.9.23.

Since this bug report was filed, there has been another Rapid Photo Downloader release:

https://launchpad.net/rapid/pyqt/0.9.24/+download/rapid-photo-downloader-0.9.24.tar.gz

As the developer I'm happy to report that the changes in 0.9.24 are highly unlikely to cause any other regressions compared to 0.9.22.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Erich Eickmeyer (eeickmeyer)]

Damon,

If you could update the description of the bug report to include this information, that would be great:

https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

And remember "none" for Regression Potential is not a good answer, there is always *some* regression potential.

Once you have this, I might be able to sponsor this. This is a package which is normally a sync from upstream Debian, so this will be an exception to that.

Thanks!

[Damon Lynch (dlynch3)]

Hi Erich, I have revamped the bug report description using the guide you suggested. Apologies if it is now too long!

[Erich Eickmeyer (eeickmeyer)]

Hi Damon, thanks for that.

>There is a new feature in 0.9.24 which is a more realistic candidate for regressions: the feature
>to provide the use the option of automatically creating a tar file of the program configuration
>file and log files when submitting a bug report. See this documentation for an illustration

So, this is an immediate red flag for SRU consideration. Any new features pretty much automatically disqualifies it from consideration. We can try, but I can't guarantee it would be allowed so much as 0.9.23 would be a more likely candidate to be allowed, especially since it's in upstream Debian already.

If you can somehow justify that the new feature is a bugfix, then that would be more likely allowed. However, at this time, I'm leaning heavily on the conservative approach of using 0.9.23.

[Damon Lynch (dlynch3)]

The new feature is a usability fix that makes it more likely the user will create a bug report with the information I require to fix the problem being reported. I guess that might be a bit of a stretch? Please use your best judgement. I have zero experience with SRUs prior to this.

If you are going to reuse the Debian package, please be aware that Debian does not package gir1.2-unity-5.0. This package is necessary for Rapid Photo Downloader to display app emblems and a progress bar in Ubuntu Gnome and in KDE, which is a great feature that I enjoy a lot. Unfortunately the Debian packager did not update the rules section as I suggested in this bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=954747 , so it needs to be added manually.

It's also a shame many developers mistakenly assume that libunity is tied only to the Unity desktop, when in fact it's not!

[Brian Murray (brian-murray)]

I'm rejecting this upload at this point in time as it is essentially a major update of the package which does not qualify as a Stable Release Update. It is possible for a package to apply for a microrelease exception and you can see examples of that on the Stable Release Updates wiki page.

I appreciate the need to fix this bug and it seems to me updating the code just what was in version 0.9.23a1 would do that.

If there are other bugs that are worth fixing in Focal they should be individual bug reports for those issues with Test Cases and Regression Potentials for each one.

[Damon Lynch (dlynch3)]

Brian if you may, allow me a minute to put to one side the (important) micro features of this issue and look at the bigger picture.

You are a professional whose job it is to manage bugs, always keeping in mind the reality that fixes can have unintended consequences. You have systems and processes in place that keep you sane and the operation transparent. That's all good. It makes sense.

I develop the code, manage the project, maintain its website, write its documentation, ingest bug reports, manage translations, attempt to make its distribution better in whatever ways I can, etc. No one pays me (although I occasionally receive a donation). I'm a volunteer whose time is truly very limited. I have other things going on in my life, like my unfinished doctoral dissertation.

(I imagine the packager for the program in Debian is also a volunteer facing similar time constraints.)

The fact is, the Rapid Photo Downloader release in Ubuntu 20.04 is a snapshot in time of the Rapid Photo Downloader project. That's it. There is literally nothing special about it. Let's not be overly devoted to it. It has a serious problem that I fixed before Ubuntu 20.04 was released, but not quickly enough for it to make past the release freeze. It also contains fixes for issues that are truly meaningful to a set of users that were fixed before the 20.04 release, but again didn't make it past the freeze. Version 0.9.24 of Rapid Photo Downloader is the most stable release of the program I've made in the 13 years I've been developing it. I know that empirically (reports from users) and I know it from my experience from running the project.

So when I'm confronted with the prospect of justifying a bug fix release in what looks like excruciating detail. . . . I hope you can see what it looks like from my end of the process. It's all a bit much really. It's like I'm being crushed by the process. I can't do it all. I see the rationale for the process, but I can't cope with it. At this moment, it's a bridge too far.

The thing is, the Rapid Photo Downloader release in Ubuntu 20.04 is a really big deal because Ubuntu 20.04 is going to be used by so many users for so long, and I know I'm going to get bug reports for the Rapid Photo Downloader package in it for years to come.

And that's painful to think about. It leads to questions as to whether the processes in place are alienating to other developers in my position. It leads me to question whether I should be scaling back my ambitions for the project.

In the bigger picture, I don't think Canonical wants this to happen to this project or any other volunteer driven FOSS project.

[Gustavo L (gustavo-lapido)]

I confirm the bug.
Upgrading to 0.9.4 did the trick, and it is now running as expected.

[Gustavo L (gustavo-lapido)]

I mean "upgrading to 0.9.24 did the trick"

[Colin Morrison (colmoni)]

I can confirm the same, 0.9.22 on Ubuntu 20.04 had a game-stopping fault at startup that was solved by upgrading to 0.9.24 with no regressions.

[Simon (simon-w)]

Also experiencing the issue mentioned:

#############:~$ rapid-photo-downloader
/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: cannot register existing type 'GdkWindow'
  app = QGuiApplication(sys.argv)
/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: g_param_spec_object: assertion 'g_type_is_a (object_type, G_TYPE_OBJECT)' failed
  app = QGuiApplication(sys.argv)
/usr/lib/python3/dist-packages/raphodo/viewutils.py:409: Warning: validate_pspec_to_install: assertion 'G_IS_PARAM_SPEC (pspec)' failed
  app = QGuiApplication(sys.argv)
Gtk-Message: 11:37:48.396: Failed to load module "topmenu-gtk-module"
QApplication: invalid style override passed, ignoring it.
    Available styles: gtk2, Windows, Fusion
/usr/lib/python3/dist-packages/raphodo/rapid.py:5843: Warning: cannot register existing type 'GdkScreen'
  super().__init__(*argv)
/usr/lib/python3/dist-packages/raphodo/rapid.py:5843: Warning: g_once_init_leave: assertion 'result != 0' failed
  super().__init__(*argv)

(rapid-photo-downloader:27140): Gdk-CRITICAL **: 11:37:48.451: IA__gdk_screen_get_default_colormap: assertion 'GDK_IS_SCREEN (screen)' failed

(rapid-photo-downloader:27140): Gdk-CRITICAL **: 11:37:48.451: IA__gdk_colormap_get_visual: assertion 'GDK_IS_COLORMAP (colormap)' failed

[Erich Eickmeyer (eeickmeyer)]

New upload (0.9.23a1) has been submitted, awaiting approval from SRU team.

[Brian Murray (brian-murray)]

Hello Damon, or anyone else affected,

Accepted rapid-photo-downloader into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/rapid-photo-downloader/0.9.23a1-0ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Erich Eickmeyer (eeickmeyer)]

This was a tricky one to verify as I had to install 18.04, launch rapid-photo-downloader, then upgrade to 20.04 to duplicate the issue. As it's intermittent, it took a couple of tries to get the crash to happen, but it did and somewhat reproducibly thereafter.

I then upgraded to the 0.9.23a1 package that I had uploaded, after which I could no longer reproduce the error. Marking as verification-done.

[Launchpad Janitor (janitor)]

This bug was fixed in the package rapid-photo-downloader - 0.9.23a1-0ubuntu0.20.04.1

---------------
rapid-photo-downloader (0.9.23a1-0ubuntu0.20.04.1) focal; urgency=medium

  * Fix for crash on startup (LP: #1873944)
  * Add gir-1.2-unity-5.0 to build dep per upstream request

 -- Erich Eickmeyer <email address hidden> Wed, 09 Sep 2020 09:18:10 -0700

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for rapid-photo-downloader has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.