APT::AutoRemove::SuggestsImportant "false" should be the default

[Steve Langasek]

After an upgrade to 17.10, I took a look at how much cruft I had accumulated on my system, and started marking various packages 'auto' which I know I don't care about keeping installed.

apt autoremove didn't remove nearly as much stuff as I expected, and as I dug down into some of them I found that a number of them were being kept because other packages on the system have Suggests: referencing them.

This is asymmetric and wrong. If Suggested packages are not automatically installed by default, then a Suggests should also not prevent a package from being automatically removed.

After a web search led me to 'https://askubuntu.com/questions/351085/how-to-remove-recommended-and-suggested-dependencies-of-uninstalled-packages', I set 'APT::AutoRemove::SuggestsImportant "false"' in my apt config; apt autoremove now wants to remove 365MiB of packages from my system. That is a LOT of cruft that has accumulated over the years of upgrades, none of which I have ever asked to be installed and all of which were universe or no-longer-available packages.

[Julian Andres Klode (juliank)]

I package A suggests package B and you install B because C depends on it, and now you remove C a year later, removing B might or might not be a good idea. I think it's better to be on the safe side here and not remove stuff the user might have come to rely on in autoremove.

[Julian Andres Klode (juliank)]

Because let's face it: If A suggests B, B provides or modifies functionality for A, so removing B might just break your use case of A.

[Steve Langasek (vorlon)]

On Sun, Oct 22, 2017 at 12:04:01AM -0000, Julian Andres Klode wrote:
> Because let's face it: If A suggests B, B provides or modifies
> functionality for A, so removing B might just break your use case of A.

That is absolutely not apt's responsibility to handle, when the admin has
never indicated to the package manager that they want to keep B; and is
nothing when weighed against the asymmetry that causes package cruft to be
kept around on every user's systems after upgrade.

A user who discovers after autoremovals that she wants B installed can
install B from the archive. But a user who just wants no-longer-needed
packages to be autoremoved has no reasonable way to get this - because
that's what 'autoremove' is supposed to imply already, but it doesn't
actually deliver unless you track down this non-obvious apt setting and
tweak your config.

If 'apt install A=1; apt install A=2; apt autoremove --purge' gives
different results than 'apt install A=2; apt autoremove --purge', then that
is a bug. Sometimes it's a bug in a package. In this case, it's a bug in
the package manager.

The current behavior is not a sensible default.

[Julian Andres Klode (juliank)]

Users have in the past "accidentally" removed entire desktops ("I installed this package, now my desktop is gone and I can't use my PC anymore"). They often just hit enter and don't really read what apt is going to do.

aptitude does autoremoval by default in any install/remove/upgrade/etc operation

SuggestsImportant was set to true in 2011 with 0.8.15.3. It's been that way for 6 years, 3 LTS releases. I feel like it's a bit late to change that back now.

Also, just setting SuggestsImportant to false will not really help - you might still get a different result in your example. Any dependency on a virtual package or an or group will keep _every_ installed package satisfying that dependency installed. So install A depending on B | C | D, install packages needing C and D, remove the latter packages and C and D will still be kept around.

So my personal autoremoval script (which based on pseudo boolean optimization to mathematically ensure the smallest possible system based on manually installed + depends + recommends) would currently remove 42 packages, autoremove only 24.

In my opinion, for careful people like us, I do agree that the Suggests handling is bad (after all, my script also does not treat Suggests as important). But I don't fully trust normal users with that, given what we've seen in the past with some not reading what apt is going to do and then complain afterwards that their entire desktop was gone.

A reasonable compromise here would be to easily give the user the information that they can remove even more packages:

* Keep autoremove SuggestsImportant
* Add a new option to autoremove and co like --autoremove-more that sets SuggestsImportant to false
* When running autoremove, show "The following packages could also be removed, but might enhance other installed packages. Use --autoremove-more to remove them" and a list of packages that are only kept by Suggests (and preferably list which package(s) are suggesting them).

Now the enhances might be confusing in the wording, but we eventually want to also have the autoremover respect Enhances, but Enhances is more complicated since it's the other way around. But it does explain the situation.

Finally, I'd like to hear what DonKult has to say, given that he turned that on in the first place.

[David Kalnischkies (donkult)]

While that sounds reasonable at first in simple situations, if I follow that argument, I can find no reason why we are doing complex metapackage handling, keeping many providers and a lot of other things, so we should get right of all those, too, should we?

In reality we have to deal with many many users who only very casually check the output of apt and generally trust it to do the right thing™. And that is kinda reasonable if we don't want to teach every user packaging practices. Most users will just not make the connection between having run autoremove a couple of days ago and suddenly not being able to push audio files from their favorite player&manager directly to their phones causing users (and supporters alike) to be frustrated. Having "too many" packages installed rarely causes that level of frustration in comparison.

autoremove is just not an "undo". It is supposed to remove things which are clearly no longer needed by anything. Like old kernels, old libraries and co. In fact, ideally we should end up in a situation in which autoremove can be called automatically so that old kernels are really gone, the system is cleaner after an upgrades and such… (but for various reasons that isn't really possible/advisable at the moment).

Perhaps we should implement an "undo" – just named differently as that is too confusing as we can't really perform an undo, but we have the history.log from which we can extract which packages were newly installed in an "apt install A" and offer to remove them as well (maybe with a question ala: those other packages make use of B initially installed with A, should we keep it?).

(And, while we are at it also a way for a repository to say: I don't support package A anymore with options among a) you can safely remove it as something else takes care of it now, b) here are potential alternatives [we tend to have a list in RM requests, but nothing a user can look at easily] c) it is dead and nothing compares d) look elsewhere for it e) … – which should deal with a lot of packages which autoremove should be removing, but is too scared ATM as it has not enough information to make a safe call)

[No, I haven't implemented either. I haven't even really thought about them. It just sounds for me like those could be potential ways to resolve the situation]

[Balint Reczey (rbalint)]

@vorlon

> After an upgrade to 17.10, I took a look at how much cruft I had accumulated on my system, and
> started marking various packages 'auto' which I know I don't care about keeping installed.
>
> apt autoremove didn't remove nearly as much stuff as I expected, and as I dug down into some of
> them I found that a number of them were being kept because other packages on the system have
>Suggests: referencing them.
>
> This is asymmetric and wrong. If Suggested packages are not automatically installed by default,
> then a Suggests should also not prevent a package from being automatically removed.

IMO this is asymmetric and right because the cost of keeping the packages on the system is much lower than the expected benefits they provide for the user. Why else would be suggested?

> After a web search led me to 'https://askubuntu.com/questions/351085/how-to-remove-recommended-and-suggested-dependencies-of-uninstalled-packages',
> I set 'APT::AutoRemove::SuggestsImportant "false"' in my apt config; apt autoremove now wants
> to remove 365MiB of packages from my system. That is a LOT of cruft that has accumulated over
> the years of upgrades, none of which I have ever asked to be installed and all of which were
> universe or no-longer-available packages.

IMO 365MiB over _years_ is not a lot at all the age of streaming 4K videos every day. Did you experience any improvement in the system's performance after removing the packages?
I just checked the size of apt's package list cache on an Artful system as a comparison:

 ~$ du -sh /var/lib/apt/lists
 367M /var/lib/apt/lists

IMO saving a tiny amount of space and surprising people by removing more packages than before would hardly make the Ubuntu experience better.

[Balint Reczey (rbalint)]

Reopening as incomplete for the further discussion, if needed.

[Steve Langasek (vorlon)]

Balint, if you think this is about disk space, I'm afraid you've missed my point. These packages are individually tiny. The point is that 365MiB of packages is a *lot* of packages - all of which are unsupported, many of which are no longer available in Ubuntu at all anymore, each of which represents a change in behavior between an upgraded system and a newly installed system that causes a combinatoric explosion of possible configurations in Ubuntu's "supported" upgrade path - "supported" in quotes, because in practice, any time a user stumbles because of such a difference, they will categorically told by the developers to remove the unsupported package.

The default behavior of Ubuntu should give users systems which, on upgrade, are as well-supported and supportable as new installs. The current behavior, in its asymmetry, does not give us that. The tools are wrong.

[Julian Andres Klode (juliank)]

Unsupported and no longer available packages can be dealt with better by extending u-r-u, for example. They are not a valid argument for autoremoving packages still suggested by other packages.

As I mentioned before, it seems reasonable to me that u-r-u would autoremove more than apt by setting SuggestsImportant to false. That solves (well, it improves, it's not an optimal solution) the problem of accumulated cruft during dist upgrades while at the same time not affecting operation on a running system.

[Steve Langasek (vorlon)]

On Fri, Feb 16, 2018 at 10:04:10PM -0000, Julian Andres Klode wrote:
> Unsupported and no longer available packages can be dealt with better by
> extending u-r-u, for example. They are not a valid argument for
> autoremoving packages still suggested by other packages.

> As I mentioned before, it seems reasonable to me that u-r-u would
> autoremove more than apt by setting SuggestsImportant to false. That
> solves (well, it improves, it's not an optimal solution) the problem of
> accumulated cruft during dist upgrades while at the same time not
> affecting operation on a running system.

Sure, I'm fine with the notion of this being handled in u-r-u instead of in
apt itself; but the general problem of cruft left behind across upgrades
should not be ignored.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:18.04.13

---------------
ubuntu-release-upgrader (1:18.04.13) bionic; urgency=medium

  * Do not consider Suggests important, so automatically installed packages
    that are not recommended or dependend on anymore are removed across a
    dist upgrade (LP: #1725861)

 -- Julian Andres Klode <email address hidden> Thu, 29 Mar 2018 15:30:19 +0200