do-release-upgrade should make disabling third party repositories optional

[Loye Young]

Binary package hint: update-manager-core

Currently, do-release-upgrade will automatically disable third party repositories that it finds in /etc/apt/sources.list.

A better option would be to add a config option to /etc/update-manager/release-upgrades (like DisableThirdPartyRepos) to determine whether or not to disable third party repositories.

Use case: I use apt-mirror to maintain a local mirror of Ubuntu repositories, in order to minimize my own bandwidth usage and to relieve strain on Ubuntu servers. Current behavior does not allow me to use the local network mirror, but instead sends me to archive.ubuntu.com.

Use case 2: We maintain a local apt repo with site-specific packages that we don't want to get disabled when there's a release upgrade.

Reasons:
1. As Ubuntu grows, demands on its servers will grow; local admins should have the option to point to other mirrors if they are confident of the currency of the mirror.
2. It is much, much faster to upgrade over a local network than to retrieve all the files over the network -- on the order of 10 to 15 times faster using 100 Mb Ethernet and 100 times faster on Gb Ethernet. If the admin has a non-trivial number of computers to upgrade, the admin will definitely want to use a local mirror.
3. Enterprise customers will likely want to fine-tune their upgrade paths to meet organizational requirements. The customer should have that option.

[Loye Young (loyeyoung)]

Reason 4 to fix this behavior:

4 Upgrade fails on machines that are intentionally kept from internet access (e.g., backend database servers or reverse proxy servers) because they can't see ubuntu.com. (This just happened to me.)

[Loye Young (loyeyoung)]

This bug was in the Feisty to Gutsy upgrade, and still exists in Gutsy to Hardy.

Instead of taking 20 minutes to upgrade from the local mirror over the LAN, it is taking 4 to 6 hours downloading from the archive pool.

I have a complete mirror of the entire Ubuntu distribution on my LAN, but the program disables the local repository and downloads from archive.ubuntu.com instead.

This doesn't just affect my company's computers, either. I've included the IYCC mirror in the sources.list in all the computers I've sold, in order to reduce the demand on the rest of the Ubuntu repository pool, and so that my customers can take advantage of the much faster speeds available from my server.

[uvi (vladimir-umnov)]

Seems ubuntu developers too busy and fliyng in the clouds. Many only ubuntu scripts works only in perfectly world, many often fail and there is no way to continue task.
This must be done prior next release. This is major task. Also it's easy to implement.

[David Fraser (davidf)]

I also needed this to use a local mirror on my ISP.
Here's my method I just applied to get this working (excuse my wiki syntax :-)) - it's definitely a hacky workaround as you can see, but it saves much time in the long run:

Upgrading to ''Hardy Heron'' following [https://help.ubuntu.com/community/HardyUpgrades HardyUpgrades]:
 * `apt-get update && apt-get upgrade` (this updated `update-manager-core`)
 * Patched `upgrade-manager-core` to allow me to hijack the mirrors list during the upgrade
{{{
--- /usr/lib/python2.5/site-packages/UpdateManager/Core/DistUpgradeFetcherCore.py.orig 2007-09-28 16:25:58.000000000 +0200
+++ /usr/lib/python2.5/site-packages/UpdateManager/Core/DistUpgradeFetcherCore.py 2008-05-08 15:05:17.000000000 +0200
@@ -143,6 +143,8 @@
             return error(_("Could not run the upgrade tool"),
                          _("This is most likely a bug in the upgrade tool. "
                           "Please report it as a bug"))
+ print "MAKE YOUR ADJUSTMENTS NOW!!! YOU HAVE 60 SECONDS!"
+ import time ; time.sleep(60)
         return True

     def _expandUri(self, uri):
}}}
 * During the wait period suspended the `do-release-upgrade` script, and edited the list of mirrors in `/tmp/tmpoKPvWP/mirrors.cfg`:
{{{
--- mirrors.cfg.orig 2008-04-17 17:58:31.000000000 +0200
+++ mirrors.cfg 2008-05-08 15:08:58.000000000 +0200
@@ -219,6 +219,8 @@
 ftp://debian.nsu.ru/ubuntu/(i386andamd64)

 #===SouthAfrica===
+http://mirror.frogfoot.net/ubuntu/
+http://za.archive.ubuntu.com/ubuntu/
 ftp://ftp.is.co.za/ubuntu/
 ftp://ftp.leg.uct.ac.za/pub/linux/ubuntu/
 ftp://ftp.sun.ac.za/ftp/ubuntu/
}}}
 * Allowed the upgrade to continue and voila it works from the Frogfoot mirrors, and the ZA ones for `backports`...
 * Revert the change to the Python file above... (obviously I saved it...)

[Kelly Collard (kjcollard)]

I have found another workaround for this problem. I didn't really want to patch all of the servers i manage so I could use our local ubuntu mirror. I used apt-mirror to mirror several ubuntu distros.

I then set up 2 additional zones in our dns server so that "archive.ubuntu.com" and "security.ubuntu.com" would actually go to our local mirror. That way when do-release-upgrade disables the third party mirror it will actually still be using our local mirror. The added benefit of doing this is that i don't even have to touch /etc/apt/sources.list in all of our servers because the default settings will reach the local mirror.

Also I had to add the the real IP addresses for those 2 zones to /etc/hosts on the mirror server so it wont do a dns lookup and loop back to itself for apt-mirror updates.

One other problem i found was that the do-release upgrade looks for several files that apt mirror isn't getting.

If you try this method you'll get this error.

do-release-upgrade
Checking for a new ubuntu release
Failed Upgrade tool signature
Done Upgrade tool
Done downloading
extracting '/tmp/tmpA-amtI/gutsy.tar.gz'
extraction failed

you need to get this directory and all the files inside and put it on your mirror as well.

http://archive.ubuntu.com/ubuntu/dists/hardy-proposed/main/dist-upgrader-all/

The above is for hardy but the same is true for gusty and feisty.

After i did that i was able to use the do-release-upgrade without patching and without changing the configs on all my servers.
If you only have a few servers you can just modify your /etc/hosts file and add those two zones and use the local ip address for your mirror and the same effect will happen. /etc/hosts will be used before the dns lookup.

[Jonathan Thomas (echidnaman)]

Adept merely displays the release announcement and launches the dist-upgrade app, update-manager-kde. update-manager-kde lives in the update-manager package, which is already marked as being affected here.
Status -> Invalid for Adept.

[Max Barry (max-maxbarry)]

Workaround in comment 4 worked great for me, thanks.

I was upgrading a remote machine on a bandwidth-capped connection; this allowed me to use a local unmetered mirror.

[Michael Vogt (mvo)]

I milestoned this for jaunty. Please ignore the "declined for jaunty" line, I can not undo it (LP oopses in this case).

[Michael Vogt (mvo)]

This is fixed in bzr now.

The way it works is that you drop a file in /etc/update-manager/release-upgrades.d/
with the extension .cfg (e.g. /etc/update-manager/release-upgrades.d/my-config.cfg) that contains:

[Sources]
AllowThirdParty=yes

as the only lines. This will override the internal default.

[Max Barry (max-maxbarry)]

Terrific! Thanks Michael.

[Geir Helland (1aunchpad-nut)]

Has this fix been back-ported to Hardy? I'm looking at a massive upgrade Hardy->Lucid now, and need to use a local mirror.