do-release-upgrade should block release upgrades in some circumstances

[Brian Murray]

Impact
------
do-release-upgrade will allow people to upgrade to a new release of Ubuntu even if they have outstanding updates to install or need to reboot from a libc6 upgrade. This behavior is different than that of update-manager which forces you to install all updates or reboot before upgrading. Additionally, this specifically allows users to upgrade without receiving fixes that would improve the upgrade process e.g. bug 1780996.

Test Case for missing updates
-----------------------------
1) Install Ubuntu 18.04 and don't install updates.
2) Modify /etc/update-manager/release-upgrades so Prompt=normal
3) Run 'do-release-upgrade -d'

With the current version of ubuntu-release-upgrader-core you'll be able to upgrade to Ubuntu 18.10, with the version from -proposed you'll receive a message saying "Please install all available updates for your release before upgrading."

Test Case for libc6 update
--------------------------
1) Install Ubuntu 18.04
2) Modify /etc/update-manager/release-upgrades so Prompt=normal
3) Run 'sudo apt-get install --reinstall libc6'
4) Run 'do-release-upgrade -d'

With the current version of ubuntu-release-upgrader-core you'll be able to upgrade to Ubuntu 18.10, with the version from -proposed you'll receive a message saying "You have not rebooted after a previous libc6 update. Please reboot before upgrading."

Regression Potential
--------------------
The strings are not translated but that seems better than people having a bad outcome when upgrading. It's also worth noting that
/usr/lib/ubuntu-release-upgrader/release-upgrade-motd calls /usr/lib/ubuntu-release-upgrader/check-new-release to write a stamp file which contains information about the availability of a release upgrade. The behavior of check-new-release should be the same i.e. it should say a new release is available and not mention the need to install updates or reboot. This can be tested by running '/usr/lib/ubuntu-release-upgrader/check-new-release -qcd'. The output should contain "New release ... available ... use do-release-upgrade".

[Iain Lane (laney)]

I saw this in the unapproved queue and have the following questions:

What's the reason for singling out the libc6 reboot-required reason rather than asking for *any* reboot-required?

If you do strongly want to keep it limited to libc6 only, wouldn't opening the file in the python code instead of opening a grep subprocess be faster?

I'm semi-convinced by your reasoning on untranslated strings, but if we could make some effort towards getting as many as possible I'd be happier, such as mailing the translator list so it gets in the next langpack refresh in bionic. (And it seems to me like if the string could avoid mentioning "libc6" that would be more friendly to any future tweaks in this area.)

Is it right to say that this could equally be a 0-day SRU and so this upload isn't important for the release itself - there's no need to rush it in for final freeze?

I'll reject from the queue for now, but someone else could find it in the rejected queue and accept if that is the decision.

[Brian Murray (brian-murray)]

On Thu, Oct 11, 2018 at 09:41:42AM -0000, Iain Lane wrote:
> I saw this in the unapproved queue and have the following questions:
>
> What's the reason for singling out the libc6 reboot-required reason
> rather than asking for *any* reboot-required?

At the end of the release upgrade you'll be required to reboot anyway so
it seemed redundant to have people reboot because evolution-data-server
or the kernel were upgraded when they'll have to reboot after the
upgrade.

> If you do strongly want to keep it limited to libc6 only, wouldn't
> opening the file in the python code instead of opening a grep subprocess
> be faster?

Ah yeah, I'll do that.

> I'm semi-convinced by your reasoning on untranslated strings, but if we
> could make some effort towards getting as many as possible I'd be
> happier, such as mailing the translator list so it gets in the next
> langpack refresh in bionic. (And it seems to me like if the string could
> avoid mentioning "libc6" that would be more friendly to any future
> tweaks in this area.)

Okay, I'll make the error message more generic.

> Is it right to say that this could equally be a 0-day SRU and so this
> upload isn't important for the release itself - there's no need to rush
> it in for final freeze?

Yes, that's true but we really need to fix it in Ubuntu 18.04 as
do-release-upgrade is run from the package on the system being upgraded
and not from the dist-upgrader tarball. So even if its not accepted
right now I think the SRU team would appreciate a we'll accept this as a
0-day SRU comment.

--
Brian Murray

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:18.10.11

---------------
ubuntu-release-upgrader (1:18.10.11) cosmic; urgency=medium

  * do-release-upgrade: do not run the release upgrade if either not all
    updates are installed or a reboot is required due to a libc6 upgrade.
    (LP: #1797209)
  * debian/control: change ubuntu-release-upgrader-core to depend on
    ca-certificates (LP: #1796940)
  * Update of demotions and mirrors data.

 -- Brian Murray <email address hidden> Wed, 10 Oct 2018 14:10:29 -0700

[Łukasz Zemczak (sil2100)]

Hello Brian, or anyone else affected,

Accepted ubuntu-release-upgrader into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:18.04.27 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Testing not having all your updates installed.

bdmurray@clean-bionic-amd64:/tmp$ apt-cache policy ubuntu-release-upgrader-core
ubuntu-release-upgrader-core:
  Installed: 1:18.04.27
  Candidate: 1:18.04.27

bdmurray@clean-bionic-amd64:/tmp$ do-release-upgrade -d
Checking for a new Ubuntu release
Please install all available updates for your release before upgrading.

So that bit passed the verification test.

[Brian Murray (brian-murray)]

If libc6 is not in reboot-required.pkgs you can upgrade to the next release.

bdmurray@clean-bionic-amd64:/tmp$ cat /run/reboot-required.pkgs
evolution-data-server
linux-base
bdmurray@clean-bionic-amd64:/tmp$ do-release-upgrade -d
Checking for a new Ubuntu release
Get:1 Upgrade tool signature [819 B]
Get:2 Upgrade tool [1,263 kB]
Fetched 1,264 kB in 0s (0 B/s)
authenticate 'cosmic.tar.gz' against 'cosmic.tar.gz.gpg'
extracting 'cosmic.tar.gz'

If libc6 is in it though you cannot upgrade to the next release.

bdmurray@clean-bionic-amd64:/tmp$ cat /run/reboot-required.pkgs
evolution-data-server
linux-base
libc6
bdmurray@clean-bionic-amd64:/tmp$ do-release-upgrade -d
Checking for a new Ubuntu release
You have not rebooted after updating a package which requires a reboot. Please reboot before upgrading.

[Brian Murray (brian-murray)]

And testing check-new-release also passed.

bdmurray@clean-bionic-amd64:/tmp$ /usr/lib/ubuntu-release-upgrader/check-new-release -qcd
New release '18.10' available.
Run 'do-release-upgrade' to upgrade to it.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:18.04.27

---------------
ubuntu-release-upgrader (1:18.04.27) bionic; urgency=medium

  * do-release-upgrade: do not run the release upgrade if either not all
    updates are installed or a reboot is required due to a libc6 upgrade.
    (LP: #1797209)
  * debian/control: change ubuntu-release-upgrader-core to depend on
    ca-certificates (LP: #1796940)
  * Update mirrors data.

 -- Brian Murray <email address hidden> Fri, 12 Oct 2018 11:27:19 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Brian Murray (brian-murray)]

Hello Brian, or anyone else affected,

Accepted ubuntu-release-upgrader into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:16.04.26 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

The verification regarding missing updates completed successfully on Ubuntu 16.04.

[Brian Murray (brian-murray)]

The verification regarding libc6 and rebooting also completed successfully on Ubuntu 16.04.

[Robie Basak (racb)]

13:12 <rbasak> bdmurray: do you know where https://git.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/diff/DistUpgrade/apt_btrfs_snapshot.py?h=ubuntu/xenial-proposed&id=7c0245c13bdeb92fb005aa7073c50bd78db99693 came from in your 1:16.04.26 upload of ubuntu-release-upgrader? Was that intentional?

[Robie Basak (racb)]

15:03 <bdmurray> rbasak: it's updated as a part of the pre-build.sh script so it was somewhat intentional.

I looked into pre-build.sh and understand why that happened now - thanks.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:16.04.26

---------------
ubuntu-release-upgrader (1:16.04.26) xenial; urgency=medium

  * do-release-upgrade: do not run the release upgrade if either not all
    fully phased updates are installed or a reboot is required due to a libc6
    upgrade. (LP: #1797209)

 -- Brian Murray <email address hidden> Tue, 06 Nov 2018 15:55:10 -0800

[Tom Womack (tom-womack)]

I am not happy with the state of this in 16.04: I do 'sudo do-release-upgrade' in the anticipation of getting 18.04.1, and I get instead

'Checking for a new Ubuntu release
Please install all available updates for your release before upgrading.'

even if I have just run sudo apt-get update; sudo apt-get upgrade.

I am *not* in the habit of rebooting my machine after updates:

oak@oak:~$ uptime
 13:01:09 up 260 days, 18:06, 12 users, load average: 20.53, 20.91, 21.40

but the message does *not* suggest that the reason it won't let me do-release-upgrade is that there is a reboot pending.

Indeed, even after a reboot and sudo apt-get update; sudo apt-get upgrade I get the same message.

oak@oak:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"

[Brian Murray (brian-murray)]

On Sat, Dec 15, 2018 at 01:04:54PM -0000, Tom Womack wrote:
> I am not happy with the state of this in 16.04: I do 'sudo do-release-
> upgrade' in the anticipation of getting 18.04.1, and I get instead
>
> 'Checking for a new Ubuntu release
> Please install all available updates for your release before upgrading.'
>
> even if I have just run sudo apt-get update; sudo apt-get upgrade.
>
> I am *not* in the habit of rebooting my machine after updates:
>
> oak@oak:~$ uptime
> 13:01:09 up 260 days, 18:06, 12 users, load average: 20.53, 20.91, 21.40
>
> but the message does *not* suggest that the reason it won't let me do-
> release-upgrade is that there is a reboot pending.
>
> Indeed, even after a reboot and sudo apt-get update; sudo apt-get
> upgrade I get the same message.

If you run 'sudo apt-get dist-upgrade' are there any packages available
for upgrading? Its possible the error message should have said
'dist-upgrade' instead of 'upgrade'.

Thanks!
--
Brian Murray

[Belgarath (belgarath)]

Is there any way to force update regardless of that message if I know what I'm doing(or not but I don't care)?

[Olivier Philip (olivier.philip)]

Third party packet (like cuda-drivers) also blocks a release upgrade process if they are not up-to-date.

They may not show in an 'apt update' command result but should be seen with 'apt list --upgradable'