debmirror fails on focal because of legacy-images

[Olaf Seibert]

If you use debmirror to mirror the focal repository, and use `--di-dist focal --di-arch=arches`, then the mirroring process fails, because it expects to fetch files from http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/images/ but that does not exist. Instead there is http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/legacy-images/.

$ /usr/bin/debmirror --host archive.ubuntu.com --method http --rsync-extra=none --diff=none --root ubuntu --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg --arch amd64 --dist focal --section main,restricted,main/debian-installer,restricted/debian-installer,universe/debian-installer,multiverse/debian-installer --di-dist focal --di-arch=arches --progress --verbose --debug /home/syseleven/dm
...
/usr/bin/debmirror: Created directory: .temp/d-i/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/focal/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/focal/main/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/focal/main/installer-amd64/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/focal/main/installer-amd64/current/
/usr/bin/debmirror: Created directory: .temp/d-i/dists/focal/main/installer-amd64/current/images/
http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/images/MD5SUMS => [ 3%] Getting: dists/focal/main/installer-amd64/current/images/MD5SUMS... #** GET http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/images/MD5SUMS ==> 404 Not Found
404 Not Found
failed 404 Not Found
Failed to download dists/focal/main/installer-amd64/current/images/MD5SUMS; skipping.
Errors:
 Download of dists/focal/main/installer-amd64/current/images/MD5SUMS failed: 404 Not Found
Failed to download some Package, Sources or Release files!
WARNING: releasing 1 pending lock...

Also, it is quite annoying that you can't specify `--dist bionic,focal-updates,focal-security,focal-backports`. It will fail because these distributions don't exist yet. However, at some point they wull probably exist, but you won't mirror them.

[Olaf Seibert (oseibert-sys11)]

This should patch it:

--- /usr/bin/debmirror 2017-07-22 01:09:37.000000000 +0000
+++ ./debmirror 2020-05-19 07:49:07.409643328 +0000
@@ -2679,7 +2679,7 @@
   if ($use_cache) {
     say(": using cache.");
     foreach my $file (sort keys %files) {
- next if (@di_dists && $file =~ m:installer-\w(-|\w)*/current/images/:);
+ next if (@di_dists && $file =~ m:installer-\w(-|\w)*/current/[a-z-]*images/:);
       if ($files{$file} == 2 && -f $file) {
         say("deleting $file") if ($verbose);
         if (! $do_dry_run) {
@@ -2696,7 +2696,7 @@
     foreach my $file (`find . -type f 2>/dev/null`) {
       chomp $file;
       $file=~s:^\./::;
- next if (@di_dists && $file =~ m:installer-\w(-|\w)*/current/images/:);
+ next if (@di_dists && $file =~ m:installer-\w(-|\w)*/current/[a-z-]*images/:);
       unless ((exists $files{$file} &&
                $files{$file} != -1 && $files{$file} != 2) or
               (defined($ignore) && $file=~/$ignore/o)) {
@@ -2764,9 +2764,18 @@

       my $image_dir = "dists/$dist/main/installer-$arch/current/images";
       make_dir ("$tdir/$image_dir");
+ my @old_errlog = @errlog;
+ my $old_num_errors = $num_errors;
       if (!remote_get("$image_dir/MD5SUMS", $tdir)) {
- say("Failed to download $image_dir/MD5SUMS; skipping.");
- return;
+ say("Failed to download $image_dir/MD5SUMS; trying legacy-images.");
+ $image_dir = "dists/$dist/main/installer-$arch/current/legacy-images";
+ make_dir ("$tdir/$image_dir");
+ if (!remote_get("$image_dir/MD5SUMS", $tdir)) {
+ say("Failed to download $image_dir/MD5SUMS; skipping.");
+ return;
+ }
+ @errlog = @old_errlog;
+ $num_errors = $old_num_errors;
       }
       if (-f "$tdir/$image_dir/MD5SUMS") {
         $bytes_to_get += -s _; # As we did not have the size earlier
@@ -2841,7 +2850,7 @@
 sub di_cleanup {
   # Clean up obsolete files
   foreach my $image_dir (`find dists/ -type d -name images 2>/dev/null`) {
- next unless $image_dir =~ m:/installer-\w(-|\w)*/current/images$:;
+ next unless $image_dir =~ m:/installer-\w(-|\w)*/current/[a-z-]*images$:;
     chomp $image_dir;
     chdir("$image_dir") or die "unable to chdir($image_dir): $!\n";
     foreach my $file (`find . -type f 2>/dev/null`) {

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "debmirror patch for focal fossa installer directory" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.