Size mismatch remains after much fiddling

[John Botha]

Binary package hint: apt-cacher-ng

Hi,

A few days ago I started receiving size mismatches upon updating. I thought I'd give it a day or so, given the festive season and potential slower fixes. The problem remains, however, and I've tried many things (as per below), all to no avail.

As you will see, I tried a lot, and would appreciate some pointers. I intended to upgrade 4 machines in addition to my cache one in the next few days. Three are on the same (very limited, very expensive) broadband, and the other is only on dial-up, so I really need this to work!

First, system info:

#######

# lsb_release -rd
Description: Ubuntu 8.10
Release: 8.10

#######

# apt-cache policy apt-cacher-ng
apt-cacher-ng:
  Installed: 0.3.3-1
  Candidate: 0.3.3-1
  Version table:
 *** 0.3.3-1 0
        100 /var/lib/dpkg/status
     0.2.1-1 0
        500 http://localhost intrepid/universe Packages

#######

# grep -v '^$\|^#' /etc/apt/sources.list
deb http://localhost:3142/packages.medibuntu.org/ intrepid free non-free
deb http://localhost:3142/archive.ubuntu.com/ubuntu intrepid main universe multiverse restricted
deb-src http://localhost:3142/archive.ubuntu.com/ubuntu intrepid universe main multiverse restricted
deb http://localhost:3142/archive.ubuntu.com/ubuntu intrepid-updates universe main multiverse restricted
deb-src http://localhost:3142/archive.ubuntu.com/ubuntu intrepid-updates restricted main multiverse universe
deb http://localhost:3142/archive.canonical.com/ubuntu intrepid partner
deb-src http://archive.canonical.com/ubuntu intrepid partner

#######

# uname -a
Linux [name removed] 2.6.27-9-generic #1 SMP Thu Nov 20 21:57:00 UTC 2008 i686 GNU/Linux

#######

I tried the following before logging this (apart from various attempts of "apt-get update" and "apt-get --fix-missing upgrade". Nothing made any apparent difference.

1. Removed all the non-intrepid directories under /var/cache/apt-cacher-ng/

2. Upgraded manually to the latest version of apt-cacher-ng, given the older version in the intrepid repositories.

3. Did an expiration via the web page. Options were:

  "Stop cleanup on errors during index update step"

    and

  "Force the download of index files (even fresh ones)"

4. Edited out the country-specific parts of sources.list

5. Moved /var/cache/apt-cacher-ng/_expending_dat elsewhere and tried apt-get update and upgrade again.

Thanks in advance. This program is the reason I finally switched to a Debian-based distro, and has served me well for some time, now.

~8-)
John

[John Botha (v-launchpad-yellowacorn-com)]

I forgot one comment: what really worries me is the following at the end of the expiration result: "1639 package file(s) marked for removal in few days."

If that's going to clear my cache (or lots of it), I'll use up an entire month of broadband to get up-to-date again.

[John Botha (v-launchpad-yellowacorn-com)]

This is probably as far as I can go now. The following steps addressed the symptom and allowed me to install the packages. I still don't know what went wrong in the first place, though, but hopefully this can help others. Do all as root.

I took the first package which it complains about, and wanted to find where on my system it is:

# locate cups_1.3.9-2ubuntu4_i386.deb | xargs ls -l
-rw-r--r-- 1 root root 1161864 2008-11-26 18:05 /var/cache/apt/archives/partial/cups_1.3.9-2ubuntu4_i386.deb
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng 1161864 2008-12-19 09:08 /var/cache/apt-cacher-ng/uburep/pool/main/c/cups/cups_1.3.9-2ubuntu4_i386.deb
-rw-r--r-- 1 apt-cacher-ng apt-cacher-ng 248 2008-12-19 09:08 /var/cache/apt-cacher-ng/uburep/pool/main/c/cups/cups_1.3.9-2ubuntu4_i386.deb.head

The first two have the same size, and diff showed them to be identical. The third is a control file with the following contents:

# cat /var/cache/apt-cacher-ng/uburep/pool/main/c/cups/cups_1.3.9-2ubuntu4_i386.deb.headHTTP/1.0 200 OK
Connection: keep-alive
Content-Length: 1161864
Last-Modified: Wed, 26 Nov 2008 16:05:05 GMT
X-Original-Source: http://za.archive.ubuntu.com/ubuntu/pool/main/c/cups/cups_1.3.9-2ubuntu4_i386.deb
Date: Fri Dec 19 07:08:19 2008

This shows the same file size, and the date & time agree as well, taking into account my listing is GMT+2.

I was at my wits' end with this, so I deleted the three files and again ran "apt-get update" and "apt-get upgrade". That solved the problem for this file, but of course not for the others. To get the others sorted out I wrote the following one-liner, which you can pull apart if you like:

echo Y | apt-get upgrade 2>&1 | grep "^Failed to" | cut -d" " -f4 | awk -F/ '{system("locate " $NF)}' | xargs rm

Check the results by running "apt-get upgrade".

Before running it, though, save the output of your "apt-get upgrade" command, so you can check later if everything went well.

It's a bit over the top, but saves a LOT of typing. It will only work if before you run it, you have updated your "locate" data base by running "updatedb".

You would also do well first to have checked what was upgraded in the process: in my case Firefox was updated and needed a re-start.

[Eduard Bloch (edi-gmx)]

a) relax
b) relax
c) make a backup. If you don't have extra space then you could create a pseudo backup with a tree of hardlinks, thus the files/directories won't be lost if the cache is nuked for some reasons. (how? "cp -al /var/cache/apt-cacher-ng /var/tmp/apt-cacher-ng-cache-backup")
d) which version was actually used to (originally) download the files in question? I guess it was the old rotten one from current ubuntu "stable"
e) with the new version, you have more options in the expiration web frontend. First, you can see the files scheduled for removal. Second, you can run an integrity check (file contents verification). If you like colored log messages, set the "verbose" flag too and watch the output of checksumming. When it's through then you can check the list of files-for-removal and then directly remove them (see new buttons on the expiration control page).

Have I already mentioned the need for backups? ;-)

[Rolf Leggewie (r0lf)]

The hardy version generally isn't well supported. The best option and the one that we usually advise is to use the one from my PPA which is much more usable

https://launchpad.net/~r0lf/+archive/ppa

[Rolf Leggewie (r0lf)]

Hm, now I see.

John, how did you end up with 0.3.3 in an Intrepid system? Did you install the debian package?

[Rolf Leggewie (r0lf)]

John, thank you for filing this report. Without further information, there isn't much we can do. Closing the report. Feel free to reopen with more information.