Is do-release-upgrade too aggressively removing transitionals?

[Christian Ehrhardt ]

It seems do-release-upgrade on long term multiple upgrades might unintentionally remove things a bit too aggressive if they have changed to be transitionals.

Example Scenario:
- LTS = src:foo v1 creates package bin:bar v1
- LTS+1 = src:foo v2 creates transitional:bar v2 depending on bin:newbar v2
- LTS+2 = src:foo v3 no more creates the transitional:bar

Expectation:
- LTS -> LTS+1 switches to transitional:bar v2 which pulls in bin:newbar
- LTS+1 -> LTS+2 keeps transitional:bar v2 around pulling in bin:newbar still
- apt dist-upgrade (even with apt autoremove) behaves this way

What happens:
- LTS -> LTS+1 switches to transitional:bar v2 which pulls in bin:newbar v2
- LTS+1 -> LTS+2 suggests to remove transitional:bar v2 in the section "Remove (was auto installed)"
- If "y" is given, then the former functionality stops working

To be fair, all of this is behind a question, with a default not to do so, like:
```
39 packages are going to be removed.
...
 Continue [yN]
```

Question to update-release-manager:
- Is this intentionally more aggressive than dist-upgrade + autoremove?
- If so, what should packages or admins do to avoid that (other than saying "n" at the "Remove (was auto installed)" question?

----------

###

This was initially reported and checked for libvirt (below) but should be a generic behavior

###

# History

In debian/1.2.10-1 7ca6a8a libvirt-bin was changed to be a transitional.

This will need to be retained until an LTS change happens so for Ubuntu this was kept and up until Xenial libvirt-bin was a package with content.
Then it followed Debian

Later in yakkety yak and later it got changed to be a transitional in 1.3.1-2
Since we'd need to wait for an Ubuntu LTS 1.3.3-2ubuntu1 kept libvirt-bin (the transitional) around this stayed another while.

Then post Bionic we dropped even that and libvirt-bin was no more (no package, no transitional, no nothing).

# Problem

This is kind of how transitions work, but now we've got a report that if people installed e.g. on Xenial and just `apt install libvirt-bin` to get virsh and other things back then. And upgrade through to e.g. focal or even later - they got the replacement packages like `libvirt-clients` removed (presumably as nothing depended on it anymore).

TODO:

This history is a bit convoluted as the timing was so different due to waiting until after LTSes.
But we need to:
1. Install xenial, install libvirt-bin, do-release-upgrade through to Jammy, check if libvirt-clients is still around or not.
2. if not, then this is a bug and we need to find how to avoid.

Potentially this is needed in all >Bionic and need to retain the libvirt-bin transitional forever? This feels so worng, with a deeper look I hope we can find what is actually going on and find a better solution.

P.S. original reporter is Mmike on #ubuntu-server in IRC

[Christian Ehrhardt  (paelzer)]

This is the focal after upgrade of Mmike: https://pastebin.com/4JFf3a0h

<Mmike> cpaelzer, so, there packages are removed during final stage of do-release-upgrade from 20.04 to 22.04: libvirt-clients libvirt-daemon libvirt-daemon-config-network libvirt-daemon-config-nwfilter libvirt-daemon-driver-qemu libvirt-daemon-system libvirt-daemon-system-systemd libvirt0
<Mmike> They're listed under `Remove (was auto installed)` 'section'.
<Mmike> (There is around 200 of them, but I only noticed issues on machines where I have libvirt installed while those were fresh xenial boxes)

[Christian Ehrhardt  (paelzer)]

Expectation:
- post bionic the transitional package has no new version anymore, the old bionic version should stay around
- unless manually removed it should continue to stay around and keep depending on e.g. libvirt-clients

Case
1. Deploy Xenial
2. Install libvirt-bin
3. Upgrade -> Bionic -> Focal
4. check availability of e.g. virsh that formerly was in there

Variants:
A) modify sources + apt dist-upgrade
B) do-release-upgrade

I ran these tests, logs to be attached soon.

[Christian Ehrhardt  (paelzer)]

Here from /var/log/dpkg.log on the system that got it removed by do-release-upgrade:

2024-01-09 12:09:13 status installed libvirt-bin:amd64 4.0.0-1ubuntu8.21
2024-01-09 12:09:13 remove libvirt-bin:amd64 4.0.0-1ubuntu8.21 <none>
2024-01-09 12:09:13 status half-configured libvirt-bin:amd64 4.0.0-1ubuntu8.21
2024-01-09 12:09:13 status half-installed libvirt-bin:amd64 4.0.0-1ubuntu8.21
2024-01-09 12:09:14 status config-files libvirt-bin:amd64 4.0.0-1ubuntu8.21
...
2024-01-09 12:09:28 purge libvirt-bin:amd64 4.0.0-1ubuntu8.21 <none>
2024-01-09 12:09:28 status config-files libvirt-bin:amd64 4.0.0-1ubuntu8.21
2024-01-09 12:09:28 status triggers-pending systemd:amd64 245.4-4ubuntu3.22
2024-01-09 12:09:28 status not-installed libvirt-bin:amd64 <none>

I can only assume that it removed packages no more in the new repository?
Maybe under some condition like being in oldlibs or any such?

[Christian Ehrhardt  (paelzer)]

I can now show that the behavior with just apt dist-upgrade behaves as I'd have expected.
libvirt-bin no more being in focal stays on 4.0.0-1ubuntu8.21 of bionic and keeps pulling things in

But indeed as reported when upgrading with do-release-upgrade the transitional package goes away and any following auto-remove would eliminate it.

In fact further upgrading to jammy in the test above does just that.
After another do-release-upgrade to in jammy it is gone.

ubuntu@x-lvb:~$ apt-cache policy libvirt-bin libvirt-clients; dpkg -S virsh; virsh -v
libvirt-bin:
  Installed: (none)
  Candidate: (none)
  Version table:
libvirt-clients:
  Installed: (none)
  Candidate: 8.0.0-1ubuntu7.7
  Version table:
     8.0.0-1ubuntu7.7 500
        500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
     8.0.0-1ubuntu7.5 500
        500 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
     8.0.0-1ubuntu7 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
sosreport: /usr/lib/python3/dist-packages/sos/report/plugins/virsh.py
Command 'virsh' not found, but can be installed with:
sudo apt install libvirt-clients

But also, I looked through logs how things happened and I think it is not that weird.

It is like:

Remove (was auto installed)
<list>
[yN]

And to be clear, that only happened because I wanted the "yes to all experience".
I said I'm ok for it to remove those, so it is my fault I guess?

I'll need to do a run with just "enter enter enter" to make sure and see how it would behave there.

[Christian Ehrhardt  (paelzer)]

Sadly the do-release-upgrade is in a screen session not directly seen in the logs.
But I'll run a "all default enter only" test whenever I come by this virtual workspace.

Until then I'm already extending that bug to do-release-upgrade for expertise on what to expect.
(Hence setting to incomplete there too)

I'm not saying do-release-upgrade does something wrong, but rather the opposite "What would be expected to be done by a package to not be removed by accident?"

Or is everything actually all right?

Furthermore, while this seems totally reproducible - this particular example is in the field since almost 4 years and comes up now. And it should affect more than just libvirt, I'd assume any transitional would be handled this way? Hence for now priority low until we know more.

[Christian Ehrhardt  (paelzer)]

Here is what happens when taking the default except where just hitting enter would abort the upgrade.
Logging all decisions (except the do you really want to do that inside ssh)

...

Xenial -> Bionic

3 installed packages are no longer supported by Canonical. You can
still get support from the community.

4 packages are going to be removed. 227 new packages are going to be
installed. 436 packages are going to be upgraded.

You have to download a total of 350 M. This download will take about
1 minute with your connection.

Installing the upgrade can take several hours. Once the download has
finished, the process cannot be canceled.

 Continue [yN] Details [d]
y

39 packages are going to be removed.

 Continue [yN] Details [d]d

...

Remove: initscripts insserv libbind9-140 libcryptsetup4
  libdns-export162 libdns162 libevent-2.0-5 libgdbm3 libicu55
  libisc-export160 libisc160 libisccc140 libisccfg140 libjson-c2
  liblwres141 libmpfr4 libpng12-0 libprocps4 libpython3.5-minimal
  libpython3.5-stdlib libreadline6 libxtables11 python3.5
  python3.5-minimal sysv-rc

Remove (was auto installed) cgmanager libboost-iostreams1.58.0
  libboost-random1.58.0 libboost-system1.58.0 libboost-thread1.58.0
  libcgmanager0 libnih-dbus1 libpython3.5 libx86-1 libxen-4.6
  linux-headers-4.4.0-210 linux-headers-4.4.0-210-generic pm-utils
  vbetool

Continue [yN] Details [d]
N

Restart required
(has to be done to be able to enter the next upgrade)
Continue [yN]
Y

---

# Now Bionic to Focal

16 installed packages are no longer supported by Canonical. You can
still get support from the community.

3 packages are going to be removed. 195 new packages are going to be
installed. 603 packages are going to be upgraded.

You have to download a total of 453 M. This download will take about
1 minute with your connection.

Installing the upgrade can take several hours. Once the download has
finished, the process cannot be canceled.

 Continue [yN] Details [d]

Remove: btrfs-tools command-not-found-data gcc-5-base gcc-6-base
  libffi6 libhogweed4 liblvm2app2.2 liblvm2cmd2.02 libnettle6 libnih1
  libplymouth4 libvirt-bin ureadahead

Remove (was auto installed) augeas-lenses bridge-utils cgmanager
  ebtables efibootmgr gcc-8-base libargon2-0 libaugeas0 libbind9-160
  libbluetooth3 libboost-iostreams1.58.0 libboost-random1.58.0
  libboost-system1.58.0 libboost-thread1.58.0 libbrlapi0.6
  libcgmanager0 libdns-export1100 libdns1100 libevent-2.1-6 libgdbm5
  libicu60 libip4tc0 libip6tc0 libiptc0 libirs160 libisc-export169
  libisc169 libisccc160 libisccfg160 libjson-c3 liblwres160 libnetcf1
  libnih-dbus1 libntfs-3g88 libperl5.26 libprocps6 libpython3.5
  libpython3.6 libpython3.6-minimal libpython3.6-stdlib libreadline7
  libsdl1.2debian libvpx5 libx86-1 libxen-4.6 libxen-4.9
  libxenstore3.0 libxentoolcore1 linux-headers-4.15.0-213
  linux-headers-4.15.0-213-generic linux-headers-4.4.0-210
  linux-headers-4.4.0-210-generic linux-image-4.4.0-210-generic
  linux-modules-4.4.0-210-generic nplan perl-modules-5.26 pm-utils
  python3-asn1crypto python3-pam python3.6 python3.6-minimal vbetool
  xdelta3

Continue [yN] Details [...

[Christian Ehrhardt  (paelzer)]

On one hand I might slightly criticize that there are a bunch of [yN] that the user needs to "yes" to get going at all.

And then there is the one about the removals in between.
Due to formerly having to "y" over the "[yN]" I instinctively kept going with "y" before.
But that isn't a bug, it is me not reading.

--

But on the other hand I'm still slightly wondering, why the package I once explicitly installed now got into the "Remove (was auto installed)" category.

And that is the bit that I'd ask to rethink if there is anything we'd need to fix (or not).
I'm also adapting the title and initial description to reflect that ...

[Christian Ehrhardt  (paelzer)]

FYI: Updated the description and bug states as it is now a question mostly to the intention/design of ubuntu-release-upgrader.