Object of different cache passed as argument to apt_pkg.DepCache method

Bug #1773316 reported by errors.ubuntu.com bug bridge on 2018-05-25
88
This bug affects 8 people
Affects Status Importance Assigned to Milestone
python-apt (Ubuntu)
Status tracked in Cosmic
Bionic
Undecided
Unassigned
Cosmic
Undecided
Unassigned
update-manager (Ubuntu)
Undecided
Unassigned
Trusty
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Cosmic
Undecided
Unassigned

Bug Description

[Impact]
python-apt 1.6 raises an exception when objects of an old cache are passed to a apt_pkg.DepCache methods for a different cache. Prior to that, those would either segfault, succeed, or silently operate on a different object, as they use package/version ids, and e.g. two different packages in the old and new cache might have the same id.

With 1.6.1, we introduce a remapping algorithm that remaps objects of apt.Cache() when calling apt.Cache.open(), allowing old objects to be used after reopening, as long as they exist in the new cache. If they don't exist in the new cache, apt_pkg.CacheMismatchError will be raised from the apt_pkg layer.

[Test case]
import apt
c=apt.Cache()
p=c["apt"]
c.open()
p.mark_install()

[Regression potential]
Could be remapping to wrong items which would cause us to install a wrong version, for example. Compared to pre-bionic, bionic is a regression already, though, and any regression caused here is less important than
what we have now.

[Original bug report]
The Ubuntu Error Tracker has been receiving reports about a problem regarding update-manager. This problem was most recently seen with package version 1:18.04.11, the problem page at https://errors.ubuntu.com/problem/e6ff7b5c385c512b7933497ad895c8a19ed063b2 contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker and are a software developer, you can request it at http://forms.canonical.com/reports/.

Related branches

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-apt (Ubuntu Bionic):
status: New → Confirmed
Changed in python-apt (Ubuntu):
status: New → Confirmed
Changed in update-manager (Ubuntu Bionic):
status: New → Confirmed
Changed in update-manager (Ubuntu):
status: New → Confirmed
summary: - /usr/bin/update-
- manager:ValueError:foreach_cb:packages_are_selected:is_selected:marked_install
+ Object of different cache passed as argument to apt_pkg.DepCache method
description: updated
Changed in update-manager (Ubuntu Bionic):
status: Confirmed → Triaged
status: Triaged → Won't Fix
Changed in update-manager (Ubuntu Cosmic):
status: Confirmed → Won't Fix
Changed in python-apt (Ubuntu Cosmic):
status: Confirmed → Fix Committed
Changed in python-apt (Ubuntu Bionic):
status: Confirmed → Triaged
Changed in python-apt (Ubuntu Cosmic):
status: Fix Committed → Confirmed
status: Confirmed → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 1.7.0~alpha0~ubuntu3

---------------
python-apt (1.7.0~alpha0~ubuntu3) cosmic; urgency=medium

  * apt.Cache: Fix error in apt.Package caching
  * apt: Fix typing errors and use mypy --strict
  * Correctly handle missing candidate in Package.get_changelog
  * apt.Cache: Remap objects when reopening cache (LP: #1773316)
  * apt_pkg.DepCache: Raise CacheMismatchError if argument belongs to diff. cache
  * apt.debfile: Avoid exception in replaces_real_pkg for no-candidate packages
  * apt.debfile.DebPackage: Handle no-candidate package in _maybe_append_multiarch_suffix
  * utils/get_debian_mirrors.py: Get data from salsa

 -- Julian Andres Klode <email address hidden> Wed, 06 Jun 2018 16:09:32 -0700

Changed in python-apt (Ubuntu Cosmic):
status: Fix Committed → Fix Released

Hello errors.ubuntu.com, or anyone else affected,

Accepted python-apt into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-apt/1.6.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-apt (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Changed in update-manager (Ubuntu):
status: Confirmed → Won't Fix
Julian Andres Klode (juliank) wrote :

Fix works fine:

root@bionic:~# python3 -c "import apt; c=apt.Cache(); p=c['aptitude']; c.open(); p.mark_install()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/apt/package.py", line 1469, in mark_install
    self._pcache._depcache.mark_install(self._pkg, auto_inst, from_user)
ValueError: Object of different cache passed as argument to apt_pkg.DepCache method
root@bionic:~# dpkg -i python3-apt_1.6.1_amd64.deb
(Reading database ... 29845 files and directories currently installed.)
Preparing to unpack python3-apt_1.6.1_amd64.deb ...
Unpacking python3-apt (1.6.1) over (1.6.0) ...
Setting up python3-apt (1.6.1) ...
root@bionic:~# python3 -c "import apt; c=apt.Cache(); p=c['aptitude']; c.open(); p.mark_install()"

another check:

root@bionic:~# python3 -c "import apt; c=apt.Cache(); p=c['aptitude']; c.open(); p.mark_delete(); print(c.get_changes()); c.commit()"
[<Package: name:'aptitude' architecture='amd64' id:257>]
(Reading database ... 29852 files and directories currently installed.)
Removing aptitude (0.8.10-6ubuntu1) ...

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Julian Andres Klode (juliank) wrote :

I also ran unattended-upgrades and aptdcon --system-upgrade, both of which performed normally.

The verification of the Stable Release Update for python-apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-apt - 1.6.1

---------------
python-apt (1.6.1) unstable; urgency=medium

  * apt: Fix typing errors to get CI pass again
  * apt.Cache: Remap objects when reopening cache (LP: #1773316)
  * apt_pkg.DepCache: Raise CacheMismatchError if argument belongs to diff. cache
  * Set branch to 1.6.y and adjust travis CI to run against stable releases
  * travis CI / Dockerfile: Pin mypy to 0.600 to prevent future changes breaking CI
  * utils/get_debian_mirrors.py: Get data from salsa

 -- Julian Andres Klode <email address hidden> Wed, 06 Jun 2018 15:14:59 -0700

Changed in python-apt (Ubuntu Bionic):
status: Fix Committed → Fix Released
Julian Andres Klode (juliank) wrote :

This update causes a regression that is tracked in bug 1780099 - notably the __hash__ of an apt.Package can change after opening, so if used in a set, or as a dict key, the package won't be found anymore. The regression is less worse than the previous state, but will be fixed shortly.

Julian Andres Klode (juliank) wrote :
description: updated
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-apt (Ubuntu Trusty):
status: New → Confirmed
Changed in python-apt (Ubuntu Xenial):
status: New → Confirmed
Changed in update-manager (Ubuntu Trusty):
status: New → Confirmed
Changed in update-manager (Ubuntu Xenial):
status: New → Confirmed
description: updated
no longer affects: python-apt (Ubuntu Xenial)
no longer affects: python-apt (Ubuntu Trusty)
Julian Andres Klode (juliank) wrote :

JFTR, in case anyone is looking at this:

The fix for xenial and trusty has been merged into the fix for 1737441, and was fixed there, as the regression that manifested in this bug due to the fix for 1737441 never manifested in those releases, as there was no upload with just that fix.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers