"Size mismatch" caused by writing wrong data

[Willem de Groot]

5-15% of packages downloaded through apt-cacher-ng are corrupt with a "Size mismatch" or "Hash Sum mismatch" error, reported by the client. Apparently, the corrupted files on the cache host in /var/cache/apt-cacher-ng are not corrupt but contain a different package than the filename suggests. For example, python-apt_0.8.3ubuntu7.1_amd64.deb is in fact xz-lzma_5.1.1alpha+20110809-3

Not sure how to debug this further.

I use apt-cacher-ng on various hosts on various isolated environments. Some apt-cacher-ng installs yield a higher "size mismatch" frequency than others, which might indicate a race condition. The ones with lower error frequency have higher uplink speeds and higher cpu speeds.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: apt-cacher-ng 0.7.18-1
ProcVersionSignature: Ubuntu 3.11.0-9.16-generic 3.11.2
Uname: Linux 3.11.0-9-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu1
Architecture: amd64
Date: Thu Oct 3 22:44:42 2013
EcryptfsInUse: Yes
InstallationDate: Installed on 2013-09-03 (30 days ago)
InstallationMedia: Xubuntu 13.10 "Saucy Salamander" - Alpha amd64 (20130723.2)
MarkForUpload: True
SourcePackage: apt-cacher-ng
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.apt.cacher.ng.backends.ubuntu: [modified]
modified.conffile..etc.apt.cacher.ng.security.conf: [inaccessible: [Errno 13] Permission denied: '/etc/apt-cacher-ng/security.conf']
mtime.conffile..etc.apt.cacher.ng.backends.ubuntu: 2013-09-03T16:38:18.300177

[Eduard Bloch (edi-gmx)]

Well... could you tell me (exactly) which version of acng is in use there?

[Willem de Groot (gwillem)]

Hi Eduard, all installs use apt-cacher-ng 0.7.18-1. Will run it with debug
mode on when I find a spare hour. Cheers!

On Saturday, October 5, 2013, Eduard Bloch wrote:

> Well... could you tell me (exactly) which version of acng is in use
> there?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1234946
>
> Title:
> "Size mismatch" caused by writing wrong data
>
> Status in “apt-cacher-ng” package in Ubuntu:
> New
>
> Bug description:
> 5-15% of packages downloaded through apt-cacher-ng are corrupt with a
> "Size mismatch" or "Hash Sum mismatch" error, reported by the client.
> Apparently, the corrupted files on the cache host in /var/cache/apt-
> cacher-ng are not corrupt but contain a different package than the
> filename suggests. For example, python-apt_0.8.3ubuntu7.1_amd64.deb is
> in fact xz-lzma_5.1.1alpha+20110809-3
>
> Not sure how to debug this further.
>
> I use apt-cacher-ng on various hosts on various isolated environments.
> Some apt-cacher-ng installs yield a higher "size mismatch" frequency
> than others, which might indicate a race condition. The ones with
> lower error frequency have higher uplink speeds and higher cpu speeds.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 13.10
> Package: apt-cacher-ng 0.7.18-1
> ProcVersionSignature: Ubuntu 3.11.0-9.16-generic 3.11.2
> Uname: Linux 3.11.0-9-generic x86_64
> NonfreeKernelModules: nvidia
> ApportVersion: 2.12.5-0ubuntu1
> Architecture: amd64
> Date: Thu Oct 3 22:44:42 2013
> EcryptfsInUse: Yes
> InstallationDate: Installed on 2013-09-03 (30 days ago)
> InstallationMedia: Xubuntu 13.10 "Saucy Salamander" - Alpha amd64
> (20130723.2)
> MarkForUpload: True
> SourcePackage: apt-cacher-ng
> UpgradeStatus: No upgrade log present (probably fresh install)
> modified.conffile..etc.apt.cacher.ng.backends.ubuntu: [modified]
> modified.conffile..etc.apt.cacher.ng.security.conf: [inaccessible:
> [Errno 13] Permission denied: '/etc/apt-cacher-ng/security.conf']
> mtime.conffile..etc.apt.cacher.ng.backends.ubuntu:
> 2013-09-03T16:38:18.300177
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/apt-cacher-ng/+bug/1234946/+subscriptions
>

[Willem de Groot (gwillem)]

Set debug to 10, it doesn't give more output. I guess I need a recompile with debugging enabled.

apt-cacher-ng is pretty much unusable right now. Sometimes, the failure rate is 100%. In that case, every cached deb files is stored with the filename of the previously requested package.

Will switch over to squid-deb-proxy as it does pretty much the same (http://www.garron.me/en/blog/ubuntu-deb-proxy-cache.html)

Cheers!

[Seth Arnold (seth-arnold)]

I have the same issue with apt-cacher-ng version 0.7.18-1 as packaged in Ubuntu 13.10.

I've attached pieces of today's apt-cache.err and apt-cache.log files. I hope they are useful.

(Willem, thanks so much for filing this bug report. I've spent two hours going around in circles trying to discover this problem. I've seen dozens, if not hundreds, of size mismatches while trying to build some software today and none of my debugging attempts would fix the issue for good.)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Seth Arnold (seth-arnold)]

This is how apt inside a chroot saw the failure to download and install packages.

[James Kyle (jkyle)]

Seeing the same error patterns here on 0.7.11-1~ubuntu12.04.1

[Michael Tibben (michael-tibben)]

Seeing the same issues on Trusty with 0.7.18-1

[Michael Tibben (michael-tibben)]

Ah sorry I'm actually on Saucy.

[Rolf Leggewie (r0lf)]

does this ever occur in trusty or later?

[Rolf Leggewie (r0lf)]

I believe I'm seeing this with acng on a jessie host. And indeed that host has patchy and slow internet connectivity.