apt-mirror does not reflect dep11/Components-* and dep11/icons-*

[fabianbur]

Ubuntu 16.04 recently added dep-11 metadata to their main archive
mirrors: dep11/Components-* and dep11/icons-*,

But these are not reflected in the mirror created using apt-mirror

When upgrading from the local mirror created using apt-mirror, customers get this error that makes it impossible to update this way:

Can't open /ubuntu/dists/xenial/main/dep11/icons-64x64.tar: No such file or directory
Can't open /ubuntu/dists/xenial/main/dep11/Components-amd64.yml

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: apt-mirror 0.5.1-1
ProcVersionSignature: Ubuntu 4.4.0-8.23-generic 4.4.2
Uname: Linux 4.4.0-8-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CurrentDesktop: Unity
Date: Sat Feb 27 23:07:43 2016
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=es_EC:es
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=es_EC.UTF-8
 SHELL=/bin/bash
SourcePackage: apt-mirror
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile..etc.apt.mirror.list: 2016-02-27T19:52:02.368127

[Mikko Pesari (mpesari)]

I've created a pull request to github/apt-mirror:

https://github.com/apt-mirror/apt-mirror/pull/56

[Timo Aaltonen (tjaalton)]

Thanks for the patches, I'll test this on my local mirror first before uploading.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt-mirror - 0.5.1-1ubuntu1

---------------
apt-mirror (0.5.1-1ubuntu1) xenial; urgency=medium

  * download-dep11-files.diff: Add support for DEP-11. (LP: #1550852)

 -- Timo Aaltonen <email address hidden> Fri, 18 Mar 2016 21:40:32 +0200

[André Bauer (monotek)]

Will there be a backport to 14.04?

[Jeff Sereno (jsereno)]

I installed the Xenial package manually (dpkg -i apt-mirror_0.5.1-1ubuntu1_all.deb downloaded directly from http://au.archive.ubuntu.com/ubuntu/pool/universe/a/apt-mirror/apt-mirror_0.5.1-1ubuntu1_all.deb) into my Trusty server and Apt-Mirror is working quite happily again for me. I can confirm that the translation and metadata files for my Xenial mirror are downloading successfully.

[Boris Pérez (polux27)]

I installed both ways, apt-get install way and dpkg -i way, but i found the same bug...apt-mirror doesn´t download dep11 and i18n folders/files...the i´d check /usr/bin/apt-mirror and modified lines 502 and 648...
Line 502 is now:
if ( $filename =~ m{^$component/i18n/(Translation-[^./]*\.gz|Translation-[^./]*\.bz2$)} ) ####Adding .gz translation files
and Line 648 is now:
if ( $filename =~ m{^$component/dep11/(Components-${arch}\.yml|Components-${arch}\.yml.gz|icons-[^./]+\.tar)\.gz$} ) ######Adding Components .yml.gz files
And now all is OK

[Eric Menard (eric-menard)]

I observed the same behavior than Boris Pérez (polux27), dep11 and yml have not be downloaded by apt-mirror version 0.5.1-ubuntu1.
I tried Boris Pérez (polux27) solution. Fix for line 502 make sense however the line 648 fix does not seem the be located at the right place. Could you confirm it or post the full content of the file.

Thanks

[Mikko Pesari (mpesari)]

Can you post your mirror.list or are you using the default?

[Eric Menard (eric-menard)]

here is my mirror.list file

[Taylor Braun-Jones (nocnokneo)]

I think this fix should be back ported to trusty. Re-open?

[Brandon (genokan)]

Having this issue when running apt-mirror from an Ubuntu 14.04 box:

dpkg -l | grep apt-mirror ii apt-mirror 0.5.1-1 all APT sources mirroring tool

I attempted to add dep11 to my mirror.list in this way: (please correct me if I did this wrong)

deb-dep11 http://us.archive.ubuntu.com/ubuntu xenial main multiverse restricted universe

Getting this error when running apt-mirror (with the above entry in mirror.list):

apt-mirror: can't open index in process_index_gz at /usr/bin/apt-mirror line 678.
sh: us.archive.ubuntu.com/ubuntu//dists/xenial-security/universe/binary-dep11/Packages.gz: No such file or directory

Before attempting to add this entry, the dep11 folder simply did not exist, resulting in errors on clients attempting to pull from my mirror.

Please advise.

[Zach Welch (zwelch)]

Jeff Sereno's comment (#6) allowed me to install the Xenial package on my Trusty mirror server. Conveniently, apt-mirror had already downloaded the necessary file, so I simply had to locate it on the local filesystem.

This fix really should be backported to Trusty. Mirrors are run on stable machines, and that means LTS. Trusty is the current LTS release, until 16.04.1 is released later this year. There should not even be debate about that course of action, as this change probably broke a huge percentage of the mirrors out there. That's not something that I expect from either Debian or Ubuntu.

[Fabien (w-launchpad-net-coelho-net)]

This extension also breaks the simple use case where someone has a mirror run on a Debian (ie not Ubuntu) box... Currently a mirror can only work with a Xenial server, how short sighted.

Breaking compatibility with Debian was not a good move: it is a "solution" which solves no problem I had, but creates some instead. I'm not impressed.

Is there a plan to backport up-stream these stupid extensions to Debian?

[Viniciusferrao (viniciusferrao)]

+1 for fixing this on the upstream.

I've a server running Debian Jessie 8 with apt-mirror running for Debian and Ubuntu. Things got broken with those Dep11 problems.

It's starting to be easier to just rsync a mirror than use the apt-mirror package.

[Aaron Thomas (athomas-work)]

Broken. I'm running a mirror for many different oses on ubuntu 12.04. Upgrading the server is a chore as I'm serving scientific linux, debian, and ubuntu mirrors as well as private repos for debian software build by myself. This broke our ability to run mirrors for xenial hosts. Bad ubuntu. What were you thinking?

[Stuart Page (sdpagent)]

@viniciusferrao can you give me the rsync command that you use? I too try to mirror debian 8 and ubuntu 14.04 and now trying to mirror 16.04.

[laulau (olaulau)]

this still hasn't been backported to trusty ... this is an LTS and un very annoying bug. please fix it !

[Adam Holland (sharpzq4300)]

I run a local Ubuntu repository on trusty. I wanted to add xenial to it, and ran into this bug.

I have tried, and can report that the fixed version of apt-mirror can be force-installed with dpkg. It is able to at least continue to work with the trusty repo while working with xenial and the dep11 subfolder.

2 years and 7 months after the fix, I have no clue why this wasn't simply added to the trusty-backports repo list! It is a bit worrisome, actually.