Regression: packagekit crashes updating itself to a new version

[Rik Mills]

[Impact]
Bionic: 18.04
Version: 1.1.9-1ubuntu2 upgrading itself to 1.1.9-1ubuntu2.18.04.1

Updating with pkcon upgrade or plasma-discover crashes packagekit mid transaction, requiring user intervention on the command line.

In plasma-discover the gui reports the crash briefly, but then stalls in apparent mid update, requiring the user to force close it and again resolve the issue on the command line.

Example transactions:

$ pkcon update
Getting updates [=========================]
Finished [=========================]
Loading cache [=========================]
Testing changes [=========================]
Finished [ ] (0%)
The following packages have to be updated:
 gir1.2-packagekitglib-1.0-1.1.9-1ubuntu2.18.04.1.amd64 GObject introspection data for the PackageKit GLib library
 libpackagekit-glib2-18-1.1.9-1ubuntu2.18.04.1.amd64 Library for accessing PackageKit using GLib
 packagekit-1.1.9-1ubuntu2.18.04.1.amd64 Provides a package management service
 packagekit-tools-1.1.9-1ubuntu2.18.04.1.amd64 Provides PackageKit command-line tools
Proceed with changes? [N/y] y

                              [=========================]
Updating packages [=========================]
Waiting for authentication [=========================]
Loading cache [=========================]
Running [=========================]
Installing packages [==================== ] (80%) The daemon crashed mid-transaction!

$ sudo apt-get upgrade
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.

and obviously after that doing anything meaningful with packagekit like installing a package fails

$pkcon install kaffeine
Resolving [=========================]
Testing changes [=========================]
Finished [ ] (0%)
The following packages have to be installed:
 kaffeine-2.0.14-1.amd64 versatile media player for KDE
Proceed with changes? [N/y] y

                              [=========================]
Installing [=========================]
Waiting for authentication [=========================]
Waiting for package manager lock[=========================]
Finished [=========================]
Fatal error: E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem.

[Test case]
Upgrade packagekit using pkcon, make sure it does not "crash".

[Regression potential]
It could only lead to old packagekitd processes not restarting. packagekit will still be told to restart itself like before, the restart being added was unintended.

---
ProblemType: Bug
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
CurrentDesktop: KDE
DistroRelease: Ubuntu 18.04
InstallationDate: Installed on 2018-06-26 (69 days ago)
InstallationMedia: Kubuntu 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)
Package: packagekit 1.1.9-1ubuntu2.18.04.1
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.15.0-33.36-generic 4.15.18
Tags: bionic
Uname: Linux 4.15.0-33-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True

[Julian Andres Klode (juliank)]

We need a crash dump to be able to investigate this. Apport can probably help you with apport-collect 1790613 or something. Without a crash dump we don't see where it crashed and thus can't do much.

[Rik Mills (rikmills)]

apport information

[Rik Mills (rikmills)]

apport information

[Rik Mills (rikmills)]

apport information

[Brian Murray (brian-murray)]

I was able to recreate this using 'pkcon upgrade' as mentioned in the description of the bug report. I did not find a crash report in /var/crash/ though.

[Julian Andres Klode (juliank)]

Hmm odd maybe something's in the journal

[Julian Andres Klode (juliank)]

packagekit postinst is restarting packagekit:

deb-systemd-invoke try-restart 'packagekit-offline-update.service' 'packagekit.service'

so yes, that's what's causing the issue. I don't think I agree with the importance High, though. PackageKit is not normally used for upgrading, only for installing and removing packages. You have to manually force this by using pkcon, hence it should be Medium I think.

[Rik Mills (rikmills)]

On 07/09/18 11:11, Julian Andres Klode wrote:
> packagekit postinst is restarting packagekit:
>
> deb-systemd-invoke try-restart 'packagekit-offline-update.service'
> 'packagekit.service'
>
> so yes, that's what's causing the issue. I don't think I agree with the
> importance High, though. PackageKit is not normally used for upgrading,
> only for installing and removing packages.

Not true. packagekit is the default update mechanism used in Kubuntu via
GUI, and update system tray notifications.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Rik Mills (rikmills)]

Bug report on previous post release update.

https://bugs.launchpad.net/ubuntu/+source/packagekit/+bug/1766932

[Rik Mills (rikmills)]

On 07/09/18 11:11, Julian Andres Klode wrote:
> PackageKit is not normally used for upgrading,
> only for installing and removing packages.

Kubuntu updated/Discover with crashing packagekit daemon testing update for LP: #1790671

[Julian Andres Klode (juliank)]

I don't think there is a solution for this problem. We can stop restarting packagekit to fix the crash, but then you end up with old packagekitd's running. So if there is a security update in packagekit or one of its libraries, it would not take effect. Not sure how to fix this properly.

[Gordon Lack (gordon-lack)]

>> I don't think there is a solution for this problem.

So why has it only started to happen recently?

What about treating it like a kernel update - don't restart the process but schedule/request a reboot?

[Matthias Klumpp (ximion)]

We don't actively kill PackageKit when updating it, we just tell the daemon to quit itself as soon as possible if it can do so.
Also, a packagekitd times out automatically when inactive.
So this crash really shouldn't happen...

[Julian Andres Klode (juliank)]

Of course we do:

if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
        if [ -d /run/systemd/system ]; then
                systemctl --system daemon-reload >/dev/null || true
                if [ -n "$2" ]; then
                        deb-systemd-invoke try-restart 'packagekit-offline-update.service' 'packagekit.service' >/dev/null || true
                fi
        fi
fi

[Julian Andres Klode (juliank)]

The default changed to restart in debhelper 10. I think we need (one of?) --no-restart-on-upgrade --no-restart-after-upgrade passed to dh_installsystemd.

[Matthias Klumpp (ximion)]

Yes, I completely missed that change, this needs to be changed.
I need to prepare a new upload for Debian anyway, maybe I can change this behavior on the weekend at Debian.

[Julian Andres Klode (juliank)]

Uploaded, both in unapproved

[Julian Andres Klode (juliank)]

FWIW, the option to pass is --no-restart-after-upgrade

[Launchpad Janitor (janitor)]

This bug was fixed in the package packagekit - 1.1.10-1ubuntu7

---------------
packagekit (1.1.10-1ubuntu7) cosmic; urgency=medium

  * Pass --no-restart-after-upgrade to dh_installsystemd to avoid PackageKit
    restarting while upgrading under PackageKit (LP: #1790613)

 -- Julian Andres Klode <email address hidden> Mon, 15 Oct 2018 15:37:44 +0200

[Brian Murray (brian-murray)]

Hello Rik, or anyone else affected,

Accepted packagekit into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/packagekit/1.1.9-1ubuntu2.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Julian Andres Klode (juliank)]

From release to updates:

# pkcon install packagekit
Resolving [=========================]
Testing changes [=========================]
Finished [ ] (0%)
The following packages have to be updated:
 packagekit-1.1.9-1ubuntu2.18.04.1.amd64 Provides a package management service
 packagekit-tools-1.1.9-1ubuntu2.18.04.1.amd64 Provides PackageKit command-line tools
Proceed with changes? [N/y] y

                              [=========================]
Installing [=========================]
Loading cache [=========================]
Downloading packages [=========================]
Running [=========================]
Installing packages [=============== ] (63%) The daemon crashed mid-transaction!

To proposed (-...18.04.3):

# pkcon install packagekit
Resolving [=========================]
Testing changes [=========================]
Finished [ ] (0%)
The following packages have to be updated:
 apt-1.6.6.amd64 commandline package manager
 apt-utils-1.6.6.amd64 package management related utility programs
 libapt-inst2.0-1.6.6.amd64 deb package format runtime library
 libapt-pkg5.0-1.6.6.amd64 package management runtime library
 packagekit-1.1.9-1ubuntu2.18.04.3.amd64 Provides a package management service
 packagekit-tools-1.1.9-1ubuntu2.18.04.3.amd64 Provides PackageKit command-line tools
Proceed with changes? [N/y] y

                              [=========================]
Installing [=========================]
Loading cache [=========================]
Downloading packages [=========================]
Running [=========================]
Installing packages [=========================]
Finished [=========================]

=> no crash, verified

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for packagekit has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package packagekit - 1.1.9-1ubuntu2.18.04.3

---------------
packagekit (1.1.9-1ubuntu2.18.04.3) bionic; urgency=medium

  * Pass --no-restart-after-upgrade to dh_installsystemd to avoid PackageKit
    restarting while upgrading under PackageKit (LP: #1790613)

packagekit (1.1.9-1ubuntu2.18.04.2) bionic; urgency=medium

  * debian/patches/frontend-locking.diff:
    Implement frontend locking in a simple way. Will need some more
    work to upstream, and possibly some error checking. (LP: #1795614)
  * Bump libapt-pkg-dev build-dep to >= 1.6.5~ for frontend locking
  * debian/patches/aptcc-Fix-invalid-version-dereference-in-AptInf-prov.patch,
    aptcc-removing-duplicate-delete-call.patch:
    Fix invalid dereference and delete wrong (duplicate) "delete"
    statement in providesCodec (LP: #1790671)

 -- Julian Andres Klode <email address hidden> Mon, 15 Oct 2018 15:46:47 +0200

[Gordon Lack (gordon-lack)]

Has it been fixed in/checked for 18.10 (Cosmic)?

[Rik Mills (rikmills)]

Already in Cosmic

https://launchpad.net/ubuntu/+source/packagekit/1.1.10-1ubuntu7