Proper support for frontend lock

Bug #1789637 reported by Balint Reczey on 2018-08-29
64
This bug affects 8 people
Affects Status Importance Assigned to Milestone
unattended-upgrades (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned

Bug Description

[Impact]

Apt and dpkg implemented the Frontend Locking API and unattended-upgrades needs to adopt it to not leave the packaging system unlocked while passing control to python-apt and dpkg to perform package installations and removals. Leaving the packaging system unlocked caused many crashes of u-u when other tools took the lock and in the worse case let other package management tools operate on dpkg's database breaking systems.

The change takes advantage of python-apt's new API and keeps the frontend lock during the run of u-u and unlocks only the inner locks for committing changes.

[Test case]

Run strace unattended-upgrades to upgrade several packages and check that lock-frontend is acquired at the beginning and not released until the end (not reacquired repeatedly).

Unfixed u-u's output is like this:
# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 93
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
...

Fixed u-u's output is like that:
# strace unattended-upgrade --dry-run --verbose 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 8
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 57
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 69
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 70
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 10
...

[Regression potential]

Incorrect lock API usage can make unattended-upgrades to crash, but it is unlikely to hang since the lock handling operations are non-blocking. Failing to reserve the locks can also cause problems by allowing other package management tools to install packages in parallel, but this problem existed in the past, too.

[Additional Info]
This is part of a wider series of changes for frontend locking

- dpkg (bug 1796081)
- apt (bug 1781169)
- python-apt (bug 1795407)
- packagekit (bug 1795614)
- unattended-upgrades (bug 1789637)
- aptdaemon (no bug filed yet)

Further details about frontend locking can be found in https://lists.debian.org/debian-dpkg/2017/01/msg00044.html

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in unattended-upgrades (Ubuntu):
status: New → Confirmed
Balint Reczey (rbalint) on 2018-10-02
description: updated
description: updated
Changed in unattended-upgrades (Ubuntu Bionic):
status: New → In Progress
Changed in unattended-upgrades (Ubuntu Xenial):
status: New → Confirmed
Brian Murray (brian-murray) wrote :

It wasn't clear from this bug report whether or not this is fixed in Cosmic but I did some detective work and yes it is, setting it to Fix Released.

Changed in unattended-upgrades (Ubuntu):
status: Confirmed → Fix Released
Changed in unattended-upgrades (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Balint Reczey (rbalint) wrote :

Tested with 1.1ubuntu1.18.04.6:

root@uu-frontend-lock:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 37
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
root@uu-frontend-lock:~# apt-get install una
unaccent unace-nonfree unagi unagi-dev unar
unace unadf unagi-dbg unalz unattended-upgrades
root@uu-frontend-lock:~# apt-get install unattended-upgrades
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  bsd-mailx default-mta | mail-transport-agent needrestart
The following packages will be upgraded:
  unattended-upgrades
1 upgraded, 0 newly installed, 0 to remove and 23 not upgraded.
Need to get 37.7 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 unattended-upgrades all 1.1ubuntu1.18.04.6 [37.7 kB]
Fetched 37.7 kB in 0s (186 kB/s)
Preconfiguring packages ...
(Reading database ... 28996 files and directories currently installed.)
Preparing to unpack .../unattended-upgrades_1.1ubuntu1.18.04.6_all.deb ...
Unpacking unattended-upgrades (1.1ubuntu1.18.04.6) over (1.1ubuntu1.18.04.5) ...
Processing triggers for ureadahead (0.100.0-20) ...
Processing triggers for systemd (237-3ubuntu10.3) ...
Setting up unattended-upgrades (1.1ubuntu1.18.04.6) ...
Processing triggers for man-db (2.8.3-2) ...
root@uu-frontend-lock:~# strace unattended-upgrade --verbose --dry-run 2>&1 | grep lock
openat(AT_FDCWD, "/var/run/unattended-upgrades.lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 4
openat(AT_FDCWD, "/var/lib/dpkg/lock-frontend", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 5
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
openat(AT_FDCWD, "/var/cache/apt/archives/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 37
openat(AT_FDCWD, "/var/lib/dpkg/lock", O_RDWR|O_CREAT|O_NOFOLLOW, 0640) = 6
root@uu-frontend-lock:~#

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic

The verification of the Stable Release Update for unattended-upgrades has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.6

---------------
unattended-upgrades (1.1ubuntu1.18.04.6) bionic; urgency=medium

  * Unlock for dpkg operations with apt_pkg.pkgsystem_unlock_inner() when it is
    available. Also stop running when reacquiring the lock fails.
    Thanks to Julian Andres Klode for original partial patch (LP: #1789637)
  * Skip rebuilding python-apt in upgrade autopkgtests.
    Python-apt has a new build dependency making the rebuilding as is failing
    and the reference handling issue is worked around in unattended-upgrades
    already. (LP: #1781586)
  * Stop trying when no adjustment could be made and adjust package candidates
    only to lower versions (LP: #1785093)
  * Skip already adjusted packages from being checked for readjusting.
    This makes it clearer that the recursion ends and can also be a bit quicker.
    (LP: #1785093)

 -- Balint Reczey <email address hidden> Tue, 02 Oct 2018 19:18:02 +0200

Changed in unattended-upgrades (Ubuntu Bionic):
status: Fix Committed → Fix Released

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.0 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Łukasz Zemczak (sil2100) wrote :

Hello Balint, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers