Unknown bad source brings up during 'zesty' to 'artful' upgrade and It break the process

[ZdravkoG]

[Impact]
Upgrades from unsupported releases do not work if they have some third-party repositories in their sources.list, for example a repository with "stable" as the suite. u-r-u generates a new sources.list with an Ubuntu entry for "stable".

[Test case]
Run the provided autopkgtest tests against 17.10.9 and check that it fails. Run it again 17.10.10 and check that it works.

[Regression Potential]
This affects release upgrades, and if broken, might prevent some release updates. It seems unlikely though: The change just checks that the new distribution of a sources.list entry is valid before writing it out.

[Original bug report]

During update after release 'zesty' 'end of life' strange error appear:

E:The repository 'http://archive.ubuntu.com/ubuntu stable Release' does not have a Release file.

This error prevent upgrade process. There is no entry like 'http://archive.ubuntu.com/ubuntu stable Release' in my 'sources.list'. I have no idea where this entry comes from. During regular packages update try, only appears error messages directly related to 'end of life' release state. No error like above. I am not sure, but it seems like 'http://archive.ubuntu.com/ubuntu stable Release' is wrongly formatted and have no idea where this comes from as all. Here is my sources list (as is now). As could be seen from the sources list my install is from 2014. Over the years I make upgrade after 'end of life' release few times without any problems. Now is the first time I have such issue.
Thanks in advance!

ProblemType: Bug
DistroRelease: Ubuntu 17.04
Package: ubuntu-release-upgrader-core 1:17.04.10
ProcVersionSignature: Ubuntu 4.10.0-42.46-generic 4.10.17
Uname: Linux 4.10.0-42-generic x86_64
NonfreeKernelModules: wl
ApportVersion: 2.20.4-0ubuntu4.10
Architecture: amd64
CrashDB: ubuntu
CurrentDesktop: KDE
Date: Mon Jan 22 15:56:24 2018
EcryptfsInUse: Yes
InstallationDate: Installed on 2014-07-12 (1289 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
PackageArchitecture: all
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: Upgraded to zesty on 2018-01-20 (2 days ago)
VarLogDistupgradeAptlog:
 Log time: 2018-01-20 03:20:51.814404
 Log time: 2018-01-20 03:20:56.946483
 Log time: 2018-01-20 03:21:25.049488

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jean-Baptiste Lallement (jibel)]

Thanks for the logs. I've been able to reproduce with your sources files. I'll investigate further.

[ZdravkoG (zdravko-g)]

Thanks

[Otto (otto11235)]

IMHO this is another result of the circular reference in the archive server

http://archive.ubuntu.com/ubuntu

Click on 'ubuntu' and see...

[ZdravkoG (zdravko-g)]

@Otto usually this can't lead to such issue. I'm not sure but probably this is some back compatibility fix or something like.

[Jean-Baptiste Lallement (jibel)]

The problem is that the release upgrader does not find any candidate for any base meta package because the release has been removed from archive.ubuntu.com. Then, when it disables third party repositories (ie non official Ubuntu repositories or whitelisted third party), it tries to be kind and adds invalid entries for main (in DistUpgradeController.py lines 730-748) These entries are invalid because it takes them from the entry that has been disabled.

for example in your case this line has been added:
deb http://archive.ubuntu.com/ubuntu stable main # auto generated by ubuntu-release-upgrader

[ZdravkoG (zdravko-g)]

I am not very familiar with the Python language (prefer C/C++). It seems file versions we are using are little bit different (lines I suppose You mean, are shifted little up). That's why I attach my version here.
I suppose the wrong value sits in 'entry.dist' field. With method "self.sources.add(entry.type, uri, entry.dist, entry.comps, comment)", erroneous value goes to take effect. Is it possible to insert one line just before noted method call and there to set correct value (just temporary as workaround)? Something like 'entry.dist = "artful"'?
Can I rely that if this don't help me, at least will not disturb current installation?

[Jean-Baptiste Lallement (jibel)]

I proposed a patch that adds an entry only if the suite in the entry is a valid suite (ie the name of the release, for example 'zesty' and not 'stable')

[ZdravkoG (zdravko-g)]

@Jean-Baptiste Lallement I just a moment ago made a try upgrade but no any difference. Probably it's need more time, Your 'fix' to bring up on the web. Actual buggy code do not persist locally, but is downloaded from "http://archive.ubuntu.com/ubuntu/dists/artful-updates/main/dist-upgrader-all/current/artful.tar.gz". Actual buggy code is not the local one, but is downloaded from "http://archive.ubuntu.com/ubuntu/dists/artful-updates/main/dist-upgrader-all/current/artful.tar.gz".
What time You think will be need?

[ZdravkoG (zdravko-g)]

In context of my previous post, sorry for wrong attachment I made early (about DistUpgradeController.py). It is the local one and is irrelevant to the case.

[ZdravkoG (zdravko-g)]

Finally I can confirm, that the @Jean-Baptiste Lallement's idea work... 😎😉
I have download the buggy artful upgrader from "http://archive.ubuntu.com/ubuntu/dists/artful-updates/main/dist-upgrader-all/current/artful.tar.gz", manually extract its content and apply proposed path to "DistUpgradeController.py". After that everything is done and I already have Ubuntu Artful (17.10) upgraded and working.
At time of this post writing, upgrader version (timed 2017-10-31 17:48) is still the buggy one. It's time the path to be applied in "artful.tar.gz", so automatic upgrade to be possible through update manager (not by hand, like me, at least for other people).
@Jean-Baptiste Lallement THANKS FOR YOUR WORK!

[Jean-Baptiste Lallement (jibel)]

@ZdravkoG, to answer your question in comment #10. The fix has to be reviewed and eventually approved, then uploaded to the development release (bionic), then to artful-proposed, verified and accepted to finally be in artful updates and publish a new release upgrader tarball on archive.u.c. It takes time.

Thanks for your help on this bug and the valuable information you provided.

[Julian Andres Klode (juliank)]

Uploaded a fix, also includes an autopkgtest test to make sure it does not happen again.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:18.04.7

---------------
ubuntu-release-upgrader (1:18.04.7) bionic; urgency=medium

  [ Julian Andres Klode ]
  * DistUpgradeController.py: Improve wording for the EFI System
    Partition messages as suggested by Steve Langasek in LP 1695666.
  * DistUpgradeFetcherKDE.py: Drop unneeded pyqtSlot, QUrl, breaks
    pyflakes
  * DistUpgradeController.py: Only add a new entry for previously
    unknown mirrors if the new entries distro is a valid release.
    (LP: #1744722)

  [ Po-Hsu Lin ]
  * DistUpgrade/DistUpgradeController.py:
    - use dictionary to simplify the mutiple if check for directories.
    - the use of multiple %s make this string difficult to translate
      for some languages, as the sentence might need to be rearranged.
      Use format() and named-variables to fix this issue.

 -- Julian Andres Klode <email address hidden> Mon, 29 Jan 2018 14:23:23 +0100

[Syniurge (syniurge)]

Thank you for finally fixing this. The faster it reaches the artful upgrader the better, not being able to install most software on 17.04 for several weeks has been painful.

[Stephan van Ingen (stephanvaningen)]

@juliank: how can I get this fix in my ubuntu 17.04 in order to be able to upgrade to 17.10?

[ZdravkoG (zdravko-g)]

Hi @Stephan van Ingen and all other impatient to wait for convenience (like me :). As I already noted above (comment #12) You can download the available buggy version (noted in same comment) and after that to correct and run it by hand. It needs a little change only in file 'DistUpgradeController.py'. One possibility is to prepend lines from 733 to 748 (inclusive) with 4 spaces for every one. I used this approach and can confirm - it works. Another possibility is to insert proposed by @Julian Andres Klode 2 lines just before line 733. I suppose this also work, but I have not checked it. Third possibility is to download fixed version of the archived tarball which I provide here. That is exact files I used for the upgrade. I use in the proposed tarball the first correction variant (You don't have to fix anything).
Does not matter what variant You choose, finally You should have a directory (created earlier with name of Your choice) where the unpacked and fixed files sits. It is time to open a terminal window.
Change Your current working directory with that (created early) where sits prepared files.
  cd place/of/prepared/files
!!!Note - It is important to be in exactly this directory! Otherwise incorrect library will be used.
There is file named 'artful'. This file represent actual upgrade application. You should run it in super user context with correct parameters. Something like:
  sudo ./artful --mode=server --frontend=DistUpgradeViewText
After entering Your password (need for 'sudo'), actual upgrade process will start (hopefully without issues). During upgrade, additional questions, related to upgrade process, can be set by application (in fact same like in graphical interface). Just answer, as appropriate, according to Your opinion.
That's all. I hope I have helped.

[Brian Murray (brian-murray)]

Hello ZdravkoG, or anyone else affected,

Accepted ubuntu-release-upgrader into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:17.10.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[ZdravkoG (zdravko-g)]

Hi @Brian Murray,

As I already noted in comment #12, my release is successfully upgraded and currently I can not reproduce same conditions. Sorry, but it seems that I can not help in such a way. 😞...
I'm sure that there are more people that are still waiting for "convenient way" and they can help.

I am not sure, but here could be some test issue! Are You sure that in case of 'end of life' release, switching to 'proposed' could be consistent? ('end of life' release repositories are unavailable)
The important thing is the archived tarball, so any such change could be irrelevant to the actual test target! So, probably preliminary beta testing is meaningless here. Actual test will happen in the final release, where is the only case with same test condition. (any other is unreliable for particular bug, I think)

[Brian Murray (brian-murray)]

On Mon, Feb 05, 2018 at 07:20:46PM -0000, ZdravkoG wrote:
> Hi @Brian Murray,
>
> As I already noted in comment #12, my release is successfully upgraded and currently I can not reproduce same conditions. Sorry, but it seems that I can not help in such a way. 😞...
> I'm sure that there are more people that are still waiting for "convenient way" and they can help.
>
> I am not sure, but here could be some test issue! Are You sure that in case of 'end of life' release, switching to 'proposed' could be consistent? ('end of life' release repositories are unavailable)
> The important thing is the archived tarball, so any such change could be irrelevant to the actual test target! So, probably preliminary beta testing is meaningless here. Actual test will happen in the final release, where is the only case with same test condition. (any other is unreliable for particular bug, I think)

do-release-upgrade downloads the release upgrader tarball from the
release to which you are upgrading so the fact that zesty is End of Life
won't matter. The test case in the bug description is correct the text
in comment #19 isn't great, i.e. paragraph 2 is wrong, for testing
ubuntu-release-upgrader.

--
Brian Murray

[Julian Andres Klode (juliank)]

ZdravkoG, don't worry, it's just a standard template the tool uses. I can do the verification myself, you don't have to get involved :)

[ZdravkoG (zdravko-g)]

Hi @Julian Andres Klode,

It seems I didn't read carefully Your additions to the bug description. Especially the line:

"Now run do-release-upgrade --proposed and check that it does not do that."

Thanks to @Brian Murray for the note.
So, if there is someone still running Ubuntu 17.04 and want to upgrade to 17.10, can open a terminal window and run the command:

  do-release-upgrade --proposed

Just to be clear. Thanks.

[Julian Andres Klode (juliank)]

While I cannot seem to reproduce thsi with the provided sources.list, my test case works. I had to delete the DistUpgrade package from the source tree to get autopkgtest to pick up the installed version (or autopkgtest here apparently test the source tree, which is wrong).

With 17.10.9:

======================================================================
FAIL: test sources.list rewrite of an obsolete mirror
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/tmp/autopkgtest.qMDThe/tree/tests/test_sources_list.py", line 169, in test_sources_list_inactive_mirror
    """)
  File "/tmp/autopkgtest.qMDThe/tree/tests/test_sources_list.py", line 584, in _verifySources2Way
    (l, sources_list))
AssertionError: False is not true : unexpected entry 'deb http://archive.ubuntu.com/ubuntu stable main # auto generated by ubuntu-release-upgrader' in sources.list. got:

With 17.10.10 (--apt-pocket=proposed):

autopkgtest [11:01:33]: test nose-tests: [-----------------------
...
** (nosetests3:6104): WARNING **: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files

(nosetests3:6104): Gtk-CRITICAL **: gtk_main_quit: assertion 'main_loops != NULL' failed
.....................................äää....
----------------------------------------------------------------------
Ran 44 tests in 11.451s

OK
autopkgtest [11:01:45]: test nose-tests: -----------------------]

[Jean-Baptiste Lallement (jibel)]

SRU verification for Artful:
I have reproduced the problem with ubuntu-release-upgrader 1:17.10.9 in artful-updates using sources.list files from the original reporter and have verified that the version of ubuntu-release-upgrader 1:17.10.10 in -proposed fixes the issue.

[David Grossberg (davidgro)]

I can confirm that

do-release-upgrade --proposed

successfully works around this issue on the user end. I am doing the upgrade now. (I also added -f DistUpgradeViewGtk3 to it because I assume the KDE UI still has broken autoscrolling in the terminal)

If anything can be done to get the fix to the main repo faster, I hope this confirmation helps.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubuntu-release-upgrader - 1:17.10.10

---------------
ubuntu-release-upgrader (1:17.10.10) artful; urgency=medium

  * DistUpgradeController.py: Only add a new entry for previously
    unknown mirrors if the new entries distro is a valid release.
    (LP: #1744722)

 -- Julian Andres Klode <email address hidden> Mon, 05 Feb 2018 10:21:00 +0100

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.