W: Bizarre Error - File size is not what the server reported

[msp3k]

I have a private, GPG-signed repository that I use to distribute pre-packaged software to our servers and workstations. Use of this private repository has been without a hitch for years until now: Attempting to distribute a very large (2.3GB) package file, apt-fails with the following error message:

----------
# apt-get install <package>
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  <package>
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 2,224 MB of archives.
After this operation, 4,416 MB of additional disk space will be used.
Get:1 http://<url>/ stable/main <package> amd64 4 [2,224 MB]
Fetched 1 B in 38s (0 B/s)
Failed to fetch http://[<url-deleted>]/./pool/<package>_4_amd64.deb Size mismatch
W: Bizarre Error - File size is not what the server reported 2224455066 18446744071639039386
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
----------

The correct file size of the package file is: 2224455066

The entry for this package in the repository's Packages file:
----------
Package: <package>
Version: 4
Architecture: amd64
Maintainer: Michael Peek <email address hidden>
Installed-Size: 4312636
Filename: ./pool/<package>_4_amd64.deb
Size: 2224455066
MD5sum: e0daf5d8a6588210f06ddd155a0119ea
SHA1: a47121319aa4262ebf6fc9286820a9b78067cc3a
SHA256: e38a7f4d5ccce10bd51ea07bc3515c0b35bf57eb80caeea5b19ecdd27f701758
Section: non-free/admin
Priority: extra
Homepage: <url>
Description: <description>
 <long description>
----------

Note that the size listed in the Packages entry is correct, and corresponds to the first number reported by the "Bizarre Error" warning returned by apt-get. But the second number, 18446744071639039386, is nowhere to be found.

The package has been re-built and the repository package files re-generated several times, to no avail. This bug may be related to this one: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=623137 (Apr 17, 2011).

NOTE: This bug has been reported before, and reportedly fixed, as bugs #122294 (2007) and #176675 (2009). Since this bug is clearly still happening and is referenced by an upstream Debian bug report I thought I would merely reference the other bugs here and open a new report.

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: apt 0.8.13.2ubuntu4.1
ProcVersionSignature: Ubuntu 2.6.38-10.46-generic 2.6.38.7
Uname: Linux 2.6.38-10-generic x86_64
NonfreeKernelModules: nvidia
Architecture: amd64
Date: Mon Jul 25 09:15:24 2011
InstallationMedia: Ubuntu-Server 11.04 "Natty Narwhal" - Release amd64 (20110426)
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 LANGUAGE=en_US:
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)

[David Kalnischkies (donkult)]

The latest incarnation of this in debian is http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=632271
(your referred bugreport is just a report to change the message, which was done).

This bug is hopefully finally fixed in the version currently sitting in debian experimental. It would be splendid if you could try this version in a vm/chroot or alike and report back as it is untested so far (there are not many users around which have packages >2 GB or even >4 GB handy… and i am not one of those)

[Alex Waite (alexqw85)]

This affects me as well on natty. I'd be willing to test out the fix if it is put in a repo somewhere.

---Alex

[msp3k (peek-nimbios)]

I have been breaking packages up into multiple packages, where each one
< 2GB. This workaround has worked so far.

Michael

On 09/12/2011 04:09 PM, Alex Waite wrote:
> This affects me as well on natty. I'd be willing to test out the fix if
> it is put in a repo somewhere.
>
> ---Alex
>

[Alex Waite (alexqw85)]

I've been playing with the compression in order to make my debs smaller. I add the following to my debian/rules file
override_dh_builddeb:
        dh_builddeb -- -Zlzma -z9
This dropped a package from 3GB using the default settings to 1.9 GB using lzma -z 9.

It also looks as if this may have been fixed in Oneiric. The Debian bug report indicates that apt 0.8.16~exp3 contains the fix, which is the current version in Oneiric. I was unable to test this because the Oneiric installer crashes during install in my virtual box VM.

---Alex

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 0.8.16~exp12ubuntu1

---------------
apt (0.8.16~exp12ubuntu1) precise; urgency=low

  [ Michael Vogt ]
  * merge from debian/experimental:
    - new ABI

  [ Steve Langasek ]
  * apt-pkg/algorithms.cc: iterate Breaks the same way as Conflicts, so that
    we resolve virtual package Breaks more effectively. Thanks to Colin
    Watson for the patch. Closes: #657695, LP: #922485.
  * apt-pkg/algorithms.{cc,h}: use an int to represent resolver scores, not
    a signed short, because large upgrades can result in an overflow for
    core packages. Thanks again to Colin Watson. Closes: #657732,
    LP: #917173.
  * Multi-Arch: none build-deps should be DEB_HOST_ARCH, not DEB_BUILD_ARCH.
    Closes: #646288.

apt (0.8.16~exp12) experimental; urgency=low

  [ Michael Vogt ]
  * apt-pkg/deb/dpkgpm.cc:
    - fix segfault on pkg removal

  [ David Kalnischkies ]
  * apt-pkg/cacheiterators.h:
    - return the correct version arch for all+foreign, too
  * apt-pkg/packagemanager.cc:
    - ignore breaks on not-installed versions while searching for
      breakage loops as we don't have to avoid them
  * debian/control:
    - remove APT from the short descriptions as lintian doesn't like it
      and it doesn't transport any information for a reader anyway
    - apply typofixes by Pascal De Vuyst, thanks! (Closes: #652834, #652835)
  * debian/rules:
    - apply patch to enable usage of hardning CPPFLAGS and LDFLAGS by
      Moritz Muehlenhoff, thanks! (Closes: #653504)
  * methods/https.cc:
    - use curls list append instead of appending Range and If-Range by hand
      which generates malformed requests, thanks Mel Collins for the hint!
      (Closes: #646381)
  * test/libapt/run-tests:
    - hurd doesn't have dmesg yet and we don't really need it either,
      so use with $0 a more stable data source for hashsumming

  [ Pino Toscano ]
  * test/libapt/globalerror_test.cc:
    - errno 0 has a different strerror on hurd, so generate the expected
      message dynamically instead of hardcoding 'Success' (Closes: #656530)

apt (0.8.16~exp11) experimental; urgency=low

  [ David Kalnischkies ]
  * apt-pkg/deb/dpkgpm.cc:
    - redirect out/input of dpkg --assert-multi-arch to /dev/null
    - if multi-arch is detected ensure that pkg:all is reported as pkg:all

apt (0.8.16~exp10) experimental; urgency=low

  [ David Kalnischkies ]
  * apt-pkg/depcache.cc:
    - implicit conflicts (for multiarch) are supposed to conflict
      only with real packages, not with virtual providers
  * apt-pkg/pkgcache.cc:
    - ignore implicit conflicts on providers in AllTarget, too
  * apt-pkg/deb/dpkgpm.cc:
    - check if dpkg supports multiarch with --assert-multi-arch
      and if it does be always explicit about the architecture
  * apt-pkg/contrib/fileutl.h:
    - store the offset in the internal fd before calculate size of
      the zlib-handled file to jump back to this place again

  [ Michael Vogt ]
  * apt-pkg/contrib/fileutl.h:
    - fix segfault triggered by the python-apt testsuite

apt (0.8.16~exp9) experimental; urgency=low

  [ Julian Andres Klode ]
  * apt-pkg/cdrom.cc:
    - Accept .bz2, .xz files in addition to .gz fil...

[Chris J Arges (arges)]

I've tested this on Oneiric and Precise. It fails in Oneiric and works in Precise.

[Rolf Leggewie (r0lf)]

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".