Unattended-Upgrades crashes with a dpkg error

Bug #1260041 reported by avius on 2013-12-11
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
unattended-upgrades (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned

Bug Description

[Impact]

 * Unattended-upgrades fails upgrading packages in the middle of a dpkg/apt call due to a parallel process acquiring apt or dpkg lock while u-u is running, but u-u running apt/dpkg anyway without successfully re-acquiring the lock.
 * The fix is stopping processing updates when the lock is detected to be lost.

[Test Case]

 * Set up a system with packages (> 30) to be upgraded:
$ lxc launch ubuntu:b36ec647e374 uu-lock-bail

 * Set up two shells to run commands in parallel
 * In shell "A" run sudo apt update && sudo unattended-upgrade --dry-run --verbose --debug
 * After u-u started installing the updates run the following command in shell "B":
$ while sleep 0.1; do python3 -c 'import apt, apt_pkg, time; print(apt_pkg.pkgsystem_lock()); time.sleep(2)' ; done

* With not fixed u-u observe the following exception repeated in shell "B", then True being printed when the lock is successfully stolen from u-u:
...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
True
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
True
True
True
True
...

Buggy u-u continues operation even after the lock is stolen.

Fixed u-u stops operation with a printout similar to the following when it detects the lock to be stolen:
...
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
/usr/bin/dpkg --status-fd 9 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/distro-info-data_0.37ubuntu0.1_all.deb
/usr/bin/dpkg --status-fd 9 --configure --pending
Exception happened during upgrade.
Traceback (most recent call last):
  File "./unattended-upgrade", line 539, in cache_commit
    res = cache.commit(install_progress=iprogress)
  File "./unattended-upgrade", line 360, in __exit__
    apt_pkg.pkgsystem_lock()
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
Exception happened during upgrade.
NoneType: None
Installing the upgrades failed!
error message: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
dpkg returned a error! See /var/log/unattended-upgrades/unattended-upgrades-dpkg.log for details
...

[Regression Potential]

 * As a consequence of the fix unattended-upgrade stops installing security-updates when an other dpkg frontend's parallel activity is detected. This is intentional to prevent parallel execution of dpkg and potentially leaving the system in an inconsistent state. The full fix will be implemented in LP: #1781169 and changing u-u to use the new locking mechanism, and then u-u will never lose the lock.

[Other Info]

I believe parallel dpkg calls cause many breakages reported against u-u and other packages listing breaking maintainer scripts.

[Original Bug Text]

As per https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1052360/comments/17, I am filing a new bug with the specific error I am running into. This has been going on for quite a while now.

Also, my log files still show up as I described in https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1052360/comments/14.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: unattended-upgrades 0.79.3ubuntu8
ProcVersionSignature: Ubuntu 3.11.0-14.21-generic 3.11.7
Uname: Linux 3.11.0-14-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 2.12.5-0ubuntu2.1
Architecture: amd64
Date: Wed Dec 11 11:07:09 2013
MarkForUpload: True
PackageArchitecture: all
SourcePackage: unattended-upgrades
UpgradeStatus: Upgraded to saucy on 2013-10-18 (54 days ago)

avius (avi142) wrote :
Balint Reczey (rbalint) on 2018-07-19
description: updated
Balint Reczey (rbalint) on 2018-07-19
Changed in unattended-upgrades (Ubuntu):
status: New → Fix Released
Balint Reczey (rbalint) wrote :

In Cosmic 1.4ubuntu1 fixed the issue:

 * Unlock for dpkg operations with apt_pkg.pkgsystem_unlock_inner() when it
    is available, also stop running when reacquiring the lock fails.
    Thanks to to Julian Andres Klode for original partial patch

Hello avius, or anyone else affected,

Accepted unattended-upgrades into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in unattended-upgrades (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
Balint Reczey (rbalint) wrote :
Download full text (3.6 KiB)

Verified the fix with 1.1ubuntu1.18.04.5:

root@uu-lock-bail:~# ./unattended-upgrade --dry-run --verbose
Initial blacklisted packages:
Initial whitelisted packages:
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
Option --dry-run given, *not* performing real actions
Packages that will be upgraded: apport bind9-host curl dirmngr distro-info-data dnsutils file git git-man gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv libbind9-160 libcurl3-gnutls libcurl4 libdns-export1100 libdns1100 libgcrypt20 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 liblwres160 libmagic-mgc libmagic1 libperl5.26 libpng16-16 libpolkit-agent-1-0 libpolkit-backend-1-0 libpolkit-gobject-1-0 libprocps6 libssl1.0.0 libssl1.1 openssl perl perl-base perl-modules-5.26 policykit-1 procps python3-apport python3-problem-report wget
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
/usr/bin/dpkg --status-fd 9 --no-triggers --unpack --auto-deconfigure /var/cache/apt/archives/distro-info-data_0.37ubuntu0.1_all.deb
/usr/bin/dpkg --status-fd 9 --configure --pending
Exception happened during upgrade.
Traceback (most recent call last):
  File "./unattended-upgrade", line 539, in cache_commit
    res = cache.commit(install_progress=iprogress)
  File "./unattended-upgrade", line 360, in __exit__
    apt_pkg.pkgsystem_lock()
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
Exception happened during upgrade.
NoneType: None
Installing the upgrades failed!
error message: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
dpkg returned a error! See /var/log/unattended-upgrades/unattended-upgrades-dpkg.log for details
root@uu-lock-bail:~# dpkg -l unattended-upgrades | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==================-============-===========================================

# while sleep 0.1; do python3 -c 'import apt, apt_pkg, time; print(apt_pkg.pkgsystem_lock()); time.sleep(2)' ; done
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable), E:Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
apt_pkg.Error: E:Could not ...

Read more...

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.1ubuntu1.18.04.5

---------------
unattended-upgrades (1.1ubuntu1.18.04.5) bionic; urgency=medium

  * Stop updating the system when reacquiring the dpkg system lock fails.
    (LP: #1260041)

 -- Balint Reczey <email address hidden> Wed, 18 Jul 2018 13:22:36 +0200

Changed in unattended-upgrades (Ubuntu Bionic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for unattended-upgrades has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Hello avius, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.0 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in unattended-upgrades (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Łukasz Zemczak (sil2100) wrote :

Hello avius, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers