Cannot update to 19.04 ubuntu-minimal could not be located

[Jonathan H]

This happens on two different machines running VPS on DigitalOcan, both trying to update, one from 18.04, the other from 18.10

ProblemType: Bug
DistroRelease: Ubuntu 18.10
Package: ubuntu-release-upgrader-core 1:18.10.11.6
ProcVersionSignature: Ubuntu 4.18.0-17.18-generic 4.18.20
Uname: Linux 4.18.0-17-generic x86_64
ApportVersion: 2.20.10-0ubuntu13.2
Architecture: amd64
CrashDB: ubuntu
Date: Tue Apr 23 07:22:43 2019
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: Upgraded to cosmic on 2019-04-23 (0 days ago)
VarLogDistupgradeAptlog:
 Log time: 2019-04-23 07:06:30.943188
 Log time: 2019-04-23 07:11:57.621194

[Brian Murray (brian-murray)]

The log files attached to this bug report aren't particularly helpful for some reason. Could you include the terminal output when the upgrade attempt fails? Thanks in advance.

[Jonathan H (talkingnews)]

Note that in the error below, it complains about not being able to find ubuntu-minimal, however, if I try and install it, it says:

"ubuntu-minimal is already the newest version (1.425.1)"

Also verified here: https://www.digitalocean.com/community/questions/cannot-update-to-19-04-the-essential-package-ubuntu-minimal-could-not-be-located-2

So might be an issue with Digitalocean VPS setup. Anway...

Here's a summary of the output - most of the stuff about "No valid mirror found" seems fairly standard on each update and hasn't been an issue before.

do-release-upgrade

Updating repository information

No valid mirror found

While scanning your repository information no mirror entry for the
upgrade was found. This can happen if you run an internal mirror or
if the mirror information is out of date.

Do you want to rewrite your 'sources.list' file anyway? If you choose
'Yes' here it will update all 'cosmic' to 'disco' entries.
If you select 'No' the upgrade will cancel.

Get:1 http://security.ubuntu.com/ubuntu disco-security InRelease [79.7 kB]
Hit https://repos.sonar.digitalocean.com/apt main InRelease
Get:2 https://deb.nodesource.com/node_8.x disco InRelease [4594 B]
Get:3 http://security.ubuntu.com/ubuntu disco-security/main Sources [712 B]
Get:4 http://security.ubuntu.com/ubuntu disco-security/main amd64 Packages [664 B]
Get:5 http://security.ubuntu.com/ubuntu disco-security/main Translation-en [380 B]
Get:6 http://security.ubuntu.com/ubuntu disco-security/main amd64 c-n-f Metadata [108 B]
Get:7 http://security.ubuntu.com/ubuntu disco-security/restricted amd64 c-n-f Metadata [116 B]
Get:8 http://security.ubuntu.com/ubuntu disco-security/universe amd64 c-n-f Metadata [112 B]
Get:9 http://security.ubuntu.com/ubuntu disco-security/multiverse amd64 c-n-f Metadata [116 B]
Get:10 https://deb.nodesource.com/node_8.x disco/main amd64 Packages [765 B]
Fetched 87.3 kB in 0s (0 B/s)

Checking package manager
Reading package lists... Done
Building dependency tree
Reading state information... Done

Invalid package information

After updating your package information, the essential package
'ubuntu-minimal' could not be located. This may be because you have
no official mirrors listed in your software sources, or because of
excessive load on the mirror you are using. See /etc/apt/sources.list
for the current list of configured software sources.
In the case of an overloaded mirror, you may want to try the upgrade
again later.

Restoring original system state

[Brian Murray (brian-murray)]

I went ahead and switched my mirror of the Ubuntu archive to mirrors.digitalocean.com and set RELEASE_UPGRADER_ALLOW_THIRD_PARTY=1, which just prevents the unknown mirror question from appearing, and ran into the following issue:

2019-04-23 11:34:19,521 WARNING mirror check skipped, *overriden* via config
2019-04-23 11:34:19,521 DEBUG examining: 'deb http://mirrors.digitalocean.com/ubuntu cosmic main restricted universe multiverse'
2019-04-23 11:34:19,522 DEBUG verifySourcesListEntry: deb http://mirrors.digitalocean.com/ubuntu disco main restricted universe multiverse
2019-04-23 11:34:19,522 DEBUG url_downloadable: http://mirrors.digitalocean.com/ubuntu/dists/disco/Release
2019-04-23 11:34:19,522 DEBUG s='http' n='mirrors.digitalocean.com' p='/ubuntu/dists/disco/Release' q='' f=''
2019-04-23 11:34:19,567 DEBUG error from httplib: 'HTTP Error 403: Forbidden'
2019-04-23 11:34:19,567 DEBUG entry '# deb http://mirrors.digitalocean.com/ubuntu cosmic main restricted universe multiverse' was disabled (no Release file)

The disco release file is accessible by other means and given that the release upgrader is just using urlopen to retrieve the Release file I suggest you take this up with digitalocean.

[Brian Murray (brian-murray)]

Perhaps they are blocking by user agents.

bdmurray@clean-cosmic-amd64:/tmp/ubuntu-release-upgrader-rdj71yjw$ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import utils
>>> utils.url_downloadable('http://mirrors.digitalocean.com/ubuntu/dists/disco/Release', print)
url_downloadable: http://mirrors.digitalocean.com/ubuntu/dists/disco/Release
s='http' n='mirrors.digitalocean.com' p='/ubuntu/dists/disco/Release' q='' f=''
send: b'HEAD /ubuntu/dists/disco/Release HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: mirrors.digitalocean.com\r\nUser-Agent: Python-urllib/3.6\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Date: Tue, 23 Apr 2019 19:02:25 GMT
header: Content-Type: text/html; charset=UTF-8
header: Connection: close
header: Set-Cookie: __cfduid=d5e9b6d1be64106c99e8a26d9faf281c41556046145; expires=Wed, 22-Apr-20 19:02:25 GMT; path=/; domain=.digitalocean.com; HttpOnly
header: Cache-Control: max-age=10
header: Expires: Tue, 23 Apr 2019 19:02:35 GMT
header: X-Frame-Options: SAMEORIGIN
header: Vary: Accept-Encoding
header: Server: cloudflare
header: CF-RAY: 4cc21776da5d29fb-SEA
error from httplib: 'HTTP Error 403: Forbidden'
False

[Jonathan H (talkingnews)]

Many thanks - that's very helpful! They even suggested that the problem was because "The do-release-upgrade will only upgrade an LTS version to the next LTS release by default".

I've forwarded the above info. Thank you.

[Brian Murray (brian-murray)]

On Tue, Apr 23, 2019 at 08:20:47PM -0000, Jonathan H wrote:
> Many thanks - that's very helpful! They even suggested that the problem
> was because "The do-release-upgrade will only upgrade an LTS version to
> the next LTS release by default".

That's definitely not the issue given that you also tried upgrading from
18.10 to 19.04 and that failed.

--
Brian Murray

[Vivien Milat (d-ubnntu-t)]

This issue affected me too, and I think I figured out what the issue is.

When the do-release upgrade script gives this prompt,
While scanning your repository information no mirror entry for the
upgrade was found. This can happen if you run an internal mirror or
if the mirror information is out of date.

Do you want to rewrite your 'sources.list' file anyway? If you choose
'Yes' here it will update all 'cosmic' to 'disco' entries.

and you say yes, it doesn't actually update all the 'cosmic' entries to 'disco'. Instead, it comments out the cosmic entries to mirrors.digitalocean.com, leaving only a few security.ubuntu.com entries uncommented. (I presume the ubuntu-minimal package is not available from those sources)

I confirmed this by opening another SSH session at this prompt:
 Third party sources disabled

Some third party entries in your sources.list were disabled. You can
re-enable them after the upgrade with the 'software-properties' tool
or your package manager.

To continue please press [ENTER]
... and then I went to look at /etc/apt/sources.list and found most of the entries commented. I uncommented all the entries for mirrors.digitalocean.com, changed cosmic to disco, then went back to the original SSH session, hit enter, and suddenly the upgrade proceeded.

[Launchpad Janitor (janitor)]

[Expired for ubuntu-release-upgrader (Ubuntu) because there has been no activity for 60 days.]