Do not consider two versions with differing SHA256 to be the same

[Julian Andres Klode]

[Impact]
APT sometimes deduplicates two debs into the same version object even if they have different SHA256 field values, causing download to fail later if one the sources also defines SHA512 (or MD5 or SHA1).

This is a problem for example, if you rebuild in a PPA because PPAs do not have SHA512 enabled but the priamary archive does.

Repositories are not required to have SHA256, so this does nothing if we do not have SHA256 for both .deb.

[Test plan]
An automated test is included in apt's extensive autopkgtest regression test suite. Successful pass of autopkgtest is the goal.

[Where problems could occur]
In terms of regressions it seems unlikely, because we compare the SHA256 only if we previously would have considered them the same version to reject them if they differ.

But of course there could be the usual unsafe memory bugs.

In a future this will bite us when we migrated to SHA3 and want to drop SHA256, just like we cannot seem to drop MD5 now.

[Julian Andres Klode (juliank)]

Merge request: https://salsa.debian.org/apt-team/apt/-/merge_requests/305/diffs

[Julian Andres Klode (juliank)]

This has landed in 2.7.3 which will sync in the next day or so.

[Julian Andres Klode (juliank)]

Generally this is a problem that's been there forever, the solution is not complete (some repositories may not have SHA256 but only SHA512), and 20.04 does not cherry-pick cleanly, so sticking with 22.04+ if there's not a lot of demands for this in 20.04.

[Andreas Hasenack (ahasenack)]

I checked previous SRUs, and we do rely on the automated tests for such changes, as it can be difficult to fabricate something "in real life" that would remain valid for the time we need it to be. I'm glad tests are being added.

This being a native package, and imported into the archive without rich history, it's a bit hard to follow what was described in d/changelog and the actual changes for a package I'm unfamiliar with, but I checked with the upstream branch in salsa, and it looks ok.

The po/* changes, and doc changes, are just about the version, so should be no issue for translators.

The same set of changes is applied to jammy and lunar, and in devel.

[Andreas Hasenack (ahasenack)]

Hello Julian, or anyone else affected,

Accepted apt into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.6.0ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Julian, or anyone else affected,

Accepted apt into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.4.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apt (2.6.0ubuntu0.1) for lunar have finished running.
The following regressions have been reported in tests triggered by the package:

auto-apt-proxy/14 (amd64)
devscripts/2.23.4ubuntu1 (amd64, arm64, armhf, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/lunar/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.7.3

---------------
apt (2.7.3) unstable; urgency=medium

  [ Tianon Gravi ]
  * Add "apt-patterns" reference to "apt list" description in apt(8)

  [ Frans Spiesschaert ]
  * Dutch manpages translation update (Closes: #1033904)
  * Dutch program translation update (Closes: #1033909)

  [ Mert Dirik ]
  * Turkish program translation update

  [ Remus-Gabriel Chelu ]
  * Romanian program translation update (Closes: #1040644)

  [ David Kalnischkies ]
  * Add apt-patterns(7) to apt{,-cache,-get} SEE ALSO sections

  [ Julian Andres Klode ]
  * Compare SHA256 to check if versions are really the same (Closes: #931175)
    (LP: #2029268)

 -- Julian Andres Klode <email address hidden> Wed, 02 Aug 2023 14:30:47 +0200

[Julian Andres Klode (juliank)]

The integration tests on autopkgtest.ubuntu.com have passed for the apt/2.4.10 and apt/2.6.0ubuntu0.1 uploads.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.6.0ubuntu0.1

---------------
apt (2.6.0ubuntu0.1) lunar; urgency=medium

  * dist-upgrade: Revert phased updates using keeps only (LP: #2025462)
  * Do not mark updates for install that are still phasing
  * Compare SHA256 to check if versions are really the same (Closes: #931175)
    (LP: #2029268)
  * Branch CI and metadata for lunar, do not require test user to have UID
    1000, the lunar image has added an ubuntu user with that uid.

 -- Julian Andres Klode <email address hidden> Wed, 02 Aug 2023 16:05:30 +0200

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for apt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.4.10

---------------
apt (2.4.10) jammy; urgency=medium

  * dist-upgrade: Revert phased updates using keeps only (LP: #2025462)
  * Do not mark updates for install that are still phasing (same bug)
  * Compare SHA256 to check if versions are really the same (Closes: #931175)
    (LP: #2029268)

 -- Julian Andres Klode <email address hidden> Wed, 02 Aug 2023 15:15:58 +0200