APT::Default-Release breaks distribution upgrade

[Matthijs Kooijman]

I just tried to upgrade my Ubuntu system from cosmic to disco, but it seems the presence of the Default-Release configuration value breaks the upgrade.

I have some Debian sources in my sources.list so I can occasionally cherry-pick packages from there. To make sure these are not by default, I created a /etc/apt/apt.conf.d/10default-release file containing:

    APT::Default-Release "cosmic";

When doing the actual upgrade (through `update-manager -c`, but that probably started something else in the background), I get the following output in the terminal:

```
Checking for a new Ubuntu release [25/498]
/usr/lib/python3/dist-packages/DistUpgrade/DistUpgradeFetcher.py:23: PyGIWarning: Gtk was imported without specifying a version first. Use gi.require_version('
Gtk', '3.0') before import to ensure that the right version gets loaded.
  from gi.repository import Gtk, Gdk
/usr/lib/python3/dist-packages/DistUpgrade/ReleaseNotesViewerWebkit.py:33: PyGIWarning: WebKit2 was imported without specifying a version first. Use gi.require
_version('WebKit2', '4.0') before import to ensure that the right version gets loaded.
  from gi.repository import WebKit2 as WebKit
authenticate 'disco.tar.gz' against 'disco.tar.gz.gpg'
extracting 'disco.tar.gz'

The AppStream system cache was updated, but some errors were detected, which might lead to missing metadata. Refer to the verbose log for more information.
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/disco", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 2086, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 1947, in fullUpgrade
    self.openCache(restore_sources_list_on_fail=True)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 215, in openCache
    return self._openCache(lock)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 243, in _openCache
    lock)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeCache.py", line 140, in __init__
    apt.Cache.__init__(self, progress)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 138, in __init__
    self.open(progress)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 198, in open
    self._cache = apt_pkg.Cache(progress)
apt_pkg.Error: E:The value 'cosmic' is invalid for APT::Default-Release as such a release is not available in the sources
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/problem_report.py", line 497, in add_to_existing
    self.write(f)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 450, in write
    block = f.read(1048576)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Original exception was:
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/disco", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 2086, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 1947, in fullUpgrade
    self.openCache(restore_sources_list_on_fail=True)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 215, in openCache
    return self._openCache(lock)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeController.py", line 243, in _openCache
    lock)
  File "/tmp/ubuntu-release-upgrader-3aipsq2r/DistUpgrade/DistUpgradeCache.py", line 140, in __init__
    apt.Cache.__init__(self, progress)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 138, in __init__
    self.open(progress)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 198, in open
    self._cache = apt_pkg.Cache(progress)
apt_pkg.Error: E:The value 'cosmic' is invalid for APT::Default-Release as such a release is not available in the sources
```

In the GUI, I got a a fairly non-descript "there was a problem" popup, which offered to report a bug / send info to developers, which I did (but this just happened in the background, no clue where the info went).

Ideally, this Apt config value would be updated during the update (at the same time when the sources.list is updated). Alternatively, they could perhaps be disabled (just like it already disables unsupported sources such as my debian sources.list entries)?

[Matthijs Kooijman (matthijskooijman)]

After this error, removing the Default-Release config and starting `update-manager -c` again shows a popup "Failed to load the package list". Clicking "ok" shows a spinner that takes some time and then offers a partial upgrade, which looks reasonable (but has not finished yet).

[Matthijs Kooijman (matthijskooijman)]

This is still an issue: Just started the disco -> eoan upgrade through the GUI prompt I got, which broke on, I think, the second step (setting up software sources IIRC). A systray icon showed up saying there was a problem with the package manager (saying 'disco', which I have now configured as the default-release, is not a valid distribution).

What seems to have happened is the sources.list was updated, replacing disco with eoan sources, breaking because the default-release is no longer available. I tried updating the default-release afterwards, so my system now shows a lot of pending package updates, but I cannot retry the release upgrade since there are pending updates.

Instead, I reverted the changes to sources.list (replacing eoan with disco again), removed my default-release, which allowed starting the release upgrade process again and now actually started the upgrade.

[Matthijs Kooijman (matthijskooijman)]

Just finished the upgrade without problems, so just disabling the default-release temporarily seems to be sufficient. Would be nicer if the upgrade somehow handled this, even if just detecting that it was set and bailing out before rather than during the upgrade and allow the user to unset it manually.

[Matthijs Kooijman (matthijskooijman)]

This is still a problem: I wanted to upgrade from eoan to focal and forgot about this bug, so the upgrade failed again (I think it actually crashed this time, the GUI window disappeared, but I accidentally dismissed the "send report" prompt, so I'm not sure how exactly).

It seems the pattern is still the same: The source.list is rewritten from eoan to focal, Default-Release is unchanged, so subsequent apt operations fail. If I then remove the Default-Release and retry to retry the upgrade with `update-manager -c`, that seems to block indefinitely at some step, even before it offers to upgrade to focal.

Reverting the sources.list changes from focal to eoan *and* removing Default-Release does allow the upgrade to be started again (like the last time).

I just noticed that when you have insufficient disk space, the upgrade procedure correctly reverts the changes made to sources.list. If this would also happen with a Default-Release-based failure, then at least recovery would be a lot easier (so this is actually a second, separate problem).

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.