cache.commit() doesn't release the archives lock

Bug #1826870 reported by Julian Andres Klode on 2019-04-29
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-apt (Debian)
Fix Released
Unknown
python-apt (Ubuntu)
High
Unassigned
Xenial
High
Unassigned
Bionic
High
Unassigned
Cosmic
High
Unassigned

Bug Description

[Impact]
cache.commit() does not release all the locks it acquires as part of committing. This is a regression of locking fixes in bug 1795407.

[Test case]
This script should work:

#!/usr/bin/env python
import apt
import subprocess

cache = apt.Cache()
pkg = cache["hello"]
pkg.mark_install()
cache.commit()
subprocess.check_call(["apt", "remove", "--yes", "hello"])

[Regression potential]
Other new locking bugs could pop up

Julian Andres Klode (juliank) wrote :

This was fixed in disco, in 1.8.4. It also affects cosmic and bionic; and the proposed SRU for bug 1795407 in xenial.

Changed in python-apt (Ubuntu):
status: New → Fix Released
status: Fix Released → New
status: New → Fix Released
Changed in python-apt (Ubuntu Xenial):
status: New → Triaged
Changed in python-apt (Ubuntu Bionic):
status: New → Triaged
Changed in python-apt (Ubuntu Cosmic):
status: New → Triaged
description: updated
Changed in python-apt (Debian):
status: Unknown → Fix Released
Changed in python-apt (Ubuntu):
importance: Undecided → High
Changed in python-apt (Ubuntu Xenial):
importance: Undecided → High
Changed in python-apt (Ubuntu Bionic):
importance: Undecided → High
Changed in python-apt (Ubuntu Cosmic):
importance: Undecided → High
tags: added: regression-update

Hello Julian, or anyone else affected,

Accepted python-apt into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-apt/1.7.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in python-apt (Ubuntu Cosmic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Łukasz Zemczak (sil2100) wrote :

Hello Julian, or anyone else affected,

Accepted python-apt into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-apt/1.6.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in python-apt (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic
Łukasz Zemczak (sil2100) wrote :

Hello Julian, or anyone else affected,

Accepted python-apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-apt/1.1.0~beta1ubuntu0.16.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in python-apt (Ubuntu Xenial):
status: Triaged → Fix Committed
tags: added: verification-needed-xenial
Julian Andres Klode (juliank) wrote :

I installed the upgrades, and ran the script in the bug description and see if it got fixed.

xenial python3-apt (1.1.0~beta1ubuntu0.16.04.4) verified - note that the one in -updates is fine too, it only broke for the previous upload in proposed.
bionic python3-apt (1.6.4) verified
cosmic python3-apt (1.7.1) verified

Full testing script for inside a lxd container:

while ! grep cloud-init /etc/apt/sources.list; do
 sleep 1
done
while ! ping -c1 google.com; do
 sleep 1
done
echo "deb http://archive.ubuntu.com/ubuntu $(lsb_release -cs)-proposed main" >> /etc/apt/sources.list
apt update
python3 - << EOF
#!/usr/bin/env python
import apt
import subprocess

cache = apt.Cache()
pkg = cache["hello"]
pkg.mark_install()
cache.commit()
subprocess.check_call(["apt", "remove", "--yes", "hello"])
EOF
apt install python3-apt
python3 - << EOF
#!/usr/bin/env python
import apt
import subprocess

cache = apt.Cache()
pkg = cache["hello"]
pkg.mark_install()
cache.commit()
subprocess.check_call(["apt", "remove", "--yes", "hello"])
EOF

Run like this:

( i=cosmic; lxc launch -e ubuntu:$i test-$i && lxc exec test-$i -- bash -x - < ~/a.sh ; lxc stop test-$i )

tags: added: verification-done verification-done-bionic verification-done-cosmic verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-cosmic verification-needed-xenial
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 1.7.1

---------------
python-apt (1.7.1) cosmic; urgency=medium

  * apt.Cache: Fix (un)locking of archives (Closes: #922416) (LP: #1826870)
  * apt.Cache: Use explicit, more safe locking in update()
  * Update mirror lists

 -- Julian Andres Klode <email address hidden> Mon, 29 Apr 2019 13:44:16 +0200

Changed in python-apt (Ubuntu Cosmic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for python-apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Łukasz Zemczak (sil2100) wrote :

Both the bionic and xenial uploads seems to have a few ADT regressions - could you take at those and see if they're not regressions? Thanks!

Julian Andres Klode (juliank) wrote :

xenial:
- ubuntu-make not a regression, same bug as before
- apport retried
- snapcraft not a regression, same issues as before

Julian Andres Klode (juliank) wrote :

xenial: None of the failures are regressions; the apport one we can also see just with apport as trigger.

Julian Andres Klode (juliank) wrote :

bionic: Only apt-clone and snapcraft left after retries. Retried them both with themselves as triggers, and they're still failing, so no regressions either.

Brian Murray (brian-murray) wrote :

I don't see apt-clone having been tried with itself as a trigger for amd64.

http://autopkgtest.ubuntu.com/packages/a/apt-clone/bionic/amd64

Julian Andres Klode (juliank) wrote :

Well, no, I'm not going to waste resources retrying all the archs, I only retried ppc64el.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 1.6.4

---------------
python-apt (1.6.4) bionic; urgency=medium

  * apt.Cache: Fix (un)locking of archives (Closes: #922416) (LP: #1826870)
  * apt.Cache: Use explicit, more safe locking in update()
  * Update mirror lists

 -- Julian Andres Klode <email address hidden> Mon, 29 Apr 2019 13:49:15 +0200

Changed in python-apt (Ubuntu Bionic):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 1.1.0~beta1ubuntu0.16.04.4

---------------
python-apt (1.1.0~beta1ubuntu0.16.04.4) xenial; urgency=medium

  * apt.Cache: Fix (un)locking of archives (Closes: #922416) (LP: #1826870)
  * apt.Cache: Use explicit, more safe locking in update()
  * Update mirror lists

python-apt (1.1.0~beta1ubuntu0.16.04.3) xenial; urgency=medium

  * Frontend locking and related locking improvements (LP: #1795407)
    - apt.Cache: Keep / Re-establish the system lock in commit()
    - apt.Cache: Keep archive locked during commit()/ in fetch_archives()
    - apt.Cache: Reinstate locks in a finally / run dpkg inside try
    - Introduce frontend locking
    - Convert apt.Cache.commit and apt_pkg.DepCache.commit to FE lock
  * Other changes to make that work:
    - travis CI: enable PPA for new apt versions
  * As always, updated mirror lists

 -- Julian Andres Klode <email address hidden> Mon, 29 Apr 2019 14:11:32 +0200

Changed in python-apt (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.