packagekit frontend locking

[Julian Andres Klode]

[SRU Notes]

[rbasak] Requires apt 1.2.28 (Xenial) and 1.6.5 (Bionic), both of which are currently in proposed. Do not release to updates without releasing the newer apt first.

[Impact]
PackageKit needs an adjustment for frontend locking, so it does not release the frontend lock during dpkg invocations, but only the normal dpkg lock.

Frontend locking prevents race conditions between multiple dpkg frontends, which can cause other frontends to be interrupted while they are installing stuff - the frontend loses the dpkg lock between dpkg runs and the system ends up in a partially configured state.

See bug 1781169 for more details on frontend locking.

[Test case]
1. Install a package
2. Modify prerm to sleep
3. Remove package via pkcon and check that packagekitd process holds lock-frontend and dpkg holds lock (we release lock by closing the lock files, so just check if the files are open).

[Regression potential]
Changing the code to call UnLockInner() vs UnLock() makes it do less steps and only release "lock" as before, and not "lock-frontend". That should not be causing any regressions.

The patch also adds a call to LockInner() after the dpkg execution to make it reacquire "lock", this could fail. It should not have much impact however, as it only affects a single transaction AFAICT. It does reduce the risk of some frontend not implementing frontend locking from racing while we still hold the frontend lock, though.

[Other info]
This is part of a wider series of SRUs for frontend locking

- dpkg (bug 1796081)
- apt (bug 1781169)
- python-apt (bug 1795407)
- packagekit (bug 1795614)
- unattended-upgrades (bug 1789637)
- aptdaemon (no bug filed yet)

Further details about frontend locking can be found in https://lists.debian.org/debian-dpkg/2017/01/msg00044.html

[Robie Basak (racb)]

The updated description explains why frontend locking is necessary, but how does this particular bug impact users? What's wrong for users as a consequence of this bug? Eg. what user story is broken? I need to understand this in order to judge whether an SRU is warranted, which I'm supposed to consider as part of my SRU review.

[Julian Andres Klode (juliank)]

Added some more details, but I'm really at a loss what else you could want to know than it screwing up installations.

[Balint Reczey (rbalint)]

@rbasak If unattended-upgrades runs at the same time as software updater is updating apt caches then unattended upgrades crashes.

[Robie Basak (racb)]

Hello Julian, or anyone else affected,

Accepted packagekit into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/packagekit/1.1.9-1ubuntu2.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Hello Julian, or anyone else affected,

Accepted packagekit into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/packagekit/0.8.17-4ubuntu6~gcc5.4ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Julian, or anyone else affected,

Accepted packagekit into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/packagekit/1.1.9-1ubuntu2.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Julian Andres Klode (juliank)]

I verified by creating a prerm script for packagekit-tools with sleep 20 in it, and then looked at the locks while that was sleeping:

# lslocks
COMMAND PID TYPE SIZE MODE M START END PATH
cron 415 FLOCK 4B WRITE 0 0 0 /run/crond.pid
packagekitd 3374 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock-frontend
packagekitd 3374 POSIX 0B WRITE 0 0 0 /var/cache/apt/archives/lock
atd 406 POSIX 4B WRITE 0 0 0 /run/atd.pid
dpkg 3500 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock

We can see the packagekitd holds the frontend lock, and dpkg holds the lock.

[Julian Andres Klode (juliank)]

(that was 1.1.9-1ubuntu2.18.04.3 in bionic)

[Julian Andres Klode (juliank)]

Same on xenial (0.8.17-4ubuntu6~gcc5.4ubuntu1.4):

root@xxx:~# lslocks
COMMAND PID TYPE SIZE MODE M START END PATH
iscsid 396 POSIX 4B WRITE 0 0 0 /run/iscsid.pid
atd 390 POSIX 4B WRITE 0 0 0 /run/atd.pid
cron 392 FLOCK 4B WRITE 0 0 0 /run/crond.pid
dpkg 2729 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock
packagekitd 2653 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock-frontend
packagekitd 2653 POSIX 0B WRITE 0 0 0 /var/cache/apt/archives/lock

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for packagekit has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package packagekit - 1.1.9-1ubuntu2.18.04.3

---------------
packagekit (1.1.9-1ubuntu2.18.04.3) bionic; urgency=medium

  * Pass --no-restart-after-upgrade to dh_installsystemd to avoid PackageKit
    restarting while upgrading under PackageKit (LP: #1790613)

packagekit (1.1.9-1ubuntu2.18.04.2) bionic; urgency=medium

  * debian/patches/frontend-locking.diff:
    Implement frontend locking in a simple way. Will need some more
    work to upstream, and possibly some error checking. (LP: #1795614)
  * Bump libapt-pkg-dev build-dep to >= 1.6.5~ for frontend locking
  * debian/patches/aptcc-Fix-invalid-version-dereference-in-AptInf-prov.patch,
    aptcc-removing-duplicate-delete-call.patch:
    Fix invalid dereference and delete wrong (duplicate) "delete"
    statement in providesCodec (LP: #1790671)

 -- Julian Andres Klode <email address hidden> Mon, 15 Oct 2018 15:46:47 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package packagekit - 0.8.17-4ubuntu6~gcc5.4ubuntu1.4

---------------
packagekit (0.8.17-4ubuntu6~gcc5.4ubuntu1.4) xenial; urgency=medium

   * debian/patches/frontend-locking.diff:
     Implement frontend locking in a simple way. Will need some more
     work to upstream, and possibly some error checking. (LP: #1795614)
   * Bump libapt-pkg-dev build-dep to >= 1.2.28~ for frontend locking

 -- Julian Andres Klode <email address hidden> Tue, 02 Oct 2018 16:13:24 +0200