apt ignoring pin/block/hold files in preferences.d for snapd

[rec9140]

[Impact]
Negative pins are overridden by a pin of 1 if the package is phasing and "not for us", which suddenly makes packages installable that were effectively blocklisted by the negative pin, overriding user settings

[Test plan]
Integration tests covers the fix for this. The fix is not enabled by default as the implementation was changed for bug 1979244, however the new implementation also tests that.

These tests are run during autopkgtest.

To test with the new implementation, create an update that pulls in a phasing package that is pinned -1 and make sure it's not being installed. I did not add a test for that case as the new implementation only works by keeping back packages, so it by definition can't suddenly cause a package to be allowed.

[Where problems could occur]
This specific bit is not enabled anymore, but for the sake of it, it is implemented as a ceiling for the pin, so any other pin will be limited to 1 if the package is considered a "not-for-us" phasing package. So problems could occur there.

For the new phasing implementation, see bug 1979244.

[Original bug report]
Did some upgrades on a new box on 22.04, and had previously removed snapd and BLOCKED via a file in /etc/apt/preferences.d/

And this upgrade cycle REINSTALLED snapd! and the stupid FF snap! Had to repurge it again!

I had done this previously, and it appears that apt or something is IGNORING any pin/holds of snapd

I use preferences.d files as using:
sudo apt-mark hold snapd

This has never worked on any package, ever...

I have:
/etc/apt/preferences.d$ cat snapd

Package: snapd
Pin: origin *
Pin-Priority: -1

And that previously resulted in an error on apt in any attempt to install snapd, including using -s... NOW it will still attempt to install snapd!

I've tried several variants of this as well, which other 22.04 and 20.04 boxes have, same on 22.04, it will allow snapd install!

Did sudo apt-get update, apt update several times, rebooted several times, had various levels of the Pin-Priority from -1 to -9999, still will attempt to install snapd, versus the expected error

sudo apt-get -s install snapd

Expected error: Reading package lists... Done Building dependency tree
Reading state information... Done Package snapd is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source

E: Package 'snapd' has no installation candidate

20.04 boxes still seem to fail correctly with this pin file,.....

Checking my BASE VM IMAGE of 22.04 and this still works there, as its not been touched, this pin file blocks snapd from installing as expected... this is apt 2.3.15, updated one is 2.4.5....

If I pick ANY OTHER RANDOM PACKAGE out and use the same pin/block file, and change the name to that package, it blocks it from installing! Anything but snapd this works for!

1)$ lsb_release -rd
Description: Ubuntu 22.04 LTS
Release: 22.04

2) sudo apt-cache policy apt
apt:
  Installed: 2.4.5
  Candidate: 2.4.5
  Version table:
 *** 2.4.5 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

What I expect to happen? HONOR MY BLOCK on snapd! It works for any random package chosen, EXCEPT snadp!

[rec9140 (rec9140)]

Seems to be others as well from

https://askubuntu.com/questions/1404886/apt-pin-for-snapd-is-ignored-in-22-04

[Julian Andres Klode (juliank)]

There is no specific handling in apt for this, so where are you seeing this? Is it possible you see this in a graphical update tool like update-manager?

Your apt-get log looks correct.

What does apt-cache policy snapd say?

[rec9140 (rec9140)]

>There is no specific handling in apt for this, so where are you seeing this?

Open konsole, and do an

sudo apt-get -s install snapd

SHOULD RESULT in
$ sudo apt-get install snapd

Reading package lists... Done
Building dependency tree
Reading state information... Done
Package snapd is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'snapd' has no installation candidate

It DOES for my 20.04 boxes, and the VM IMAGE for 22.04 which has not been touched with any updates since created... (we update that once in a VM) and it WAS PREVIOUSLY WORKING on this box, as I had just rebuilt it for 22.04 specifically.. went to do some testing with another program.. and had to cleanse the system before I could do the testing....

ON 22.04 it WILL INSTALL SNAPD!

I do NOT use "update manager"

>Is it possible you see this in a graphical update tool like update-manager?

This is *** CLI **** I do all updates via CLI for things... I do not do any auto updating ever.

I will occasionally use synaptic for GUI interface, when I need to search for something.... mostly to get package names, and the dump to the CLI to install.

>What does apt-cache policy snapd say?

FOR 20.04 and it correctly blocked I get:

$ sudo apt-cache policy snapd
snapd:
  Installed: (none)
  Candidate: (none)
  Version table:
     2.54.3+20.04.1ubuntu0.3 -10
        500 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     2.54.3+20.04.1ubuntu0.2 -10
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages

ON *** 22.04 *** which has been updated yesterday:
$ sudo apt-cache policy snapd

snapd:
  Installed: (none)
  Candidate: 2.55.5+22.04
  Version table:
     2.55.5+22.04 1 (phased 50%)
        500 http://us.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.55.3+22.04 -10
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages

     2.44.3+20.04 -10
        500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages

I just tried this in my VM that I was testing with last night, and which was powered up, checked, powered down, and NO UPDATES were applied, and NOW IT IS PERMITTING snapd to be installed! Thankfully I didn't run with permission to complete (-s option!)

Testing with a 22.10 daily build ISO and it CORRECTLY BLOCKS things...

[Julian Andres Klode (juliank)]

Thanks. I think this is a corner case I did not handle correctly in the phased updates support.

https://discourse.ubuntu.com/t/phased-updates-in-apt-in-21-04/20345

I think if you set

APT::Get::Never-Include-Phased-Updates

In the meantime, this should workaround the issue.

[Julian Andres Klode (juliank)]

What happens here is that that version of snapd is not phased for you, and it hence pins it down to 1. But you already had it pinned to -1 and that got overridden, sorry. It should only downgrade pin priority to 1 if it's phased as not for you.

[rec9140 (rec9140)]

Ok... this looks to resolve what ever was changed.... THANK YOU! I will make a note to add this to our config changes for our base images.... I take it that 22.10 didn't get this yet??? So this will be needed going forward or 22.10 solves this glitch???

This is way above my pay grade... I'd just like to get some clarity and understanding on part of this.....

My question on this is:

My BASE 22.04 VM image what we use to create a new 22.04 VM, is cloned/copied... to a VM.. then updated then. setup....

So in that cycle testing it power up, check that it still honors the pin/hold/reject file... OK, works, power down... come back to it a day later, rinse repeat, and IT WAS NOT HONORING THE PIN FILE! NO UPDATES were performed, ie: sudo apt-get update, upgrade, dist-upgrade, nothing. Just sudo apt-get -s install snapd to test.....auto-upgrades are blocked in the various config files... so what triggers this fails as expected initially, then exhibits the same behavior... that tells me something is updating stealthy that should not. Which I need to disable. We don't want anything updating till I tell it to via sudo apt-get update, upgrade|dist-upgrade etc...

Thank you again, for the solution, and I look forward to the educational reply to understand this more.. Thank you.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.5.1

---------------
apt (2.5.1) unstable; urgency=medium

  [ Américo Monteiro ]
  * Portuguese manpages translation update (Closes: #1011315)

  [ Ronan Desplanques ]
  * Fix integer underflow in flExtension

  [ Roberto C. Sánchez ]
  * Some minor tweaks of spelling/grammar for better readability.

  [ Tianon Gravi ]
  * Switch from "security.d.o" to "deb.d.o" (matching bullseye release notes)

  [ Julian Andres Klode ]
  * (Temporarily) Rewrite phased updates using a keep-back approach
    (LP: #1979244)
  * policy: Do not override negative pins with 1 due to phasing (LP: #1978125)

 -- Julian Andres Klode <email address hidden> Thu, 30 Jun 2022 13:27:30 +0200

[Łukasz Zemczak (sil2100)]

Hello rec9140, or anyone else affected,

Accepted apt into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.4.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello rec9140, or anyone else affected,

Accepted apt into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.3.9ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apt (2.3.9ubuntu0.2) for impish have finished running.
The following regressions have been reported in tests triggered by the package:

reprotest/0.7.16 (amd64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/impish/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apt (2.4.6) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu82.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Julian Andres Klode (juliank)]

Verified. I hacked around the Packages files locally to simulate the situation:

0. Pinned snapd to -1 and removed it
1. Modified packages file to add Depends: snapd to an update in proposed (netplan.io), and set
   Phased-Update-Percentage: 0 on snapd

Before:

root@jammy:~# apt policy snapd
snapd:
  Installed: (none)
  Candidate: 2.55.5+22.04
  Version table:
     2.55.5+22.04 1 (phased 0%)
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
     2.55.3+22.04 -1
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
root@jammy:~# apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  snapd
The following packages will be upgraded:
  apt libapt-pkg6.0 libgstreamer1.0-0 libicu70 libnetplan0 libnss3 netplan.io python3-gi
8 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
1 standard security update
Need to get 34.8 MB/37.1 MB of archives.
After this operation, 89.3 MB of additional disk space will be used.
Do you want to continue? [Y/n] ^C
root@jammy:~# apt install snapd -s
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  zenity | kdialog
The following NEW packages will be installed:
  snapd
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Inst snapd (2.55.5+22.04 Ubuntu:22.04/jammy-updates [amd64])
Conf snapd (2.55.5+22.04 Ubuntu:22.04/jammy-updates [amd64])

After upgradi...

[Julian Andres Klode (juliank)]

For impish, please remove the update, it is not necessary to release an SRU 3 days before EOL that improves the situation for further SRUs.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 2.4.6

---------------
apt (2.4.6) jammy; urgency=medium

  * (Temporarily) Rewrite phased updates using a keep-back approach
    (LP: #1979244)
  * policy: Do not override negative pins with 1 due to phasing (LP: #1978125)
  * Point branch to 2.4.y and use jammy in gitlab-ci

 -- Julian Andres Klode <email address hidden> Thu, 30 Jun 2022 15:33:22 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for apt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.