"bizarre error - file size is not what the server reported" with private ppa

Bug #807303 reported by Martin Pool on 2011-07-08
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Low
Unassigned
apt (Debian)
Fix Released
Unknown
apt (Ubuntu)
Undecided
Unassigned

Bug Description

My natty desktop machine has been happily subscribed to a private ppa for a long time. When I updated it today (for the first time in a month) it showed this error:

Fetched 262 MB in 3min 47s (1,150 kB/s)
Failed to fetch https://mbp:<email address hidden>/canonical-ux/walled-garden/ubuntu/pool/main/u/ubuntu-font-family-sources/ttf-ubuntu-font-family_0.71.2-0ubuntu5+phasedbeta3~natty_all.deb Size mismatch
W: Bizarre Error - File size is not what the server reported 1728827 1712444
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

It's possible this is some local error but it seems unlikely any network problem could interfere with a file served over https.

Related branches

Martin Pool (mbp) wrote :

A second attempt to fetch it succeeded, so perhaps this was an intermittent server or client side error. Perhaps the http stream was aborted on the server side, in which case there might be a server side log?

I'm not sure how to rate this: it's a user visible error so it seems to count as critical but maybe it's also a one-off.

Some possibilities:
 - we don't write the file atomically
   unlikely, you can't reference a deb till its in the index, and it
gets written entirely beforehand
 - ssl disconnect partway through - if using http/1.0 that counts as
'EOF' unless content-length was emitted
 - but in apt where content-length + early disconnect gives this error.

Changed in launchpad:
status: New → Triaged
importance: Undecided → Low
summary: - "bizarre error" fetching from private ppa
+ "bizarre error" file size is not what the server reported with private
+ ppa

I suspect an apt client side issue FWIW - possibly interacting with ssl on LP in some way, so we should keep an LP task.

On Friday 08 July 2011 02:21:56 you wrote:
> Some possibilities:
> - we don't write the file atomically
> unlikely, you can't reference a deb till its in the index, and it
> gets written entirely beforehand

In previous instances of this problem it's always been a transparent proxy
getting in the way.

On Friday 08 July 2011 02:29:54 you wrote:
> I suspect an apt client side issue FWIW - possibly interacting with ssl
> on LP in some way, so we should keep an LP task.

It's all served statically from Apache, how would LP be involved?

@bigjools
re: proxy - this is https://, so no interception can be occuring at the http layer.
re: LP task - LP is involved at the project level, and until we track down the cause, or permit bugs on launchpad-project directly, this is as good a place as any other to start gathering data about the problem.

Martin Pool (mbp) on 2011-07-12
summary: - "bizarre error" file size is not what the server reported with private
+ "bizarre error - file size is not what the server reported" with private
ppa
Julian Edwards (julian-edwards) wrote :

When the PPA publisher runs it does not do anything atomically in dists/. This might be the cause of the bug as there could be a race we've not thought about.

William Grant (wgrant) wrote :

Well, each file is written then moved into place near-atomically. But, critically, pool/ is written strictly before dists/, and there can be no file conflicts, so any file referenced in Packages or Sources must already be on disk.

Anders Kaseorg (andersk) wrote :

I can reproduce this message by adding many https repositories to sources.list. For example, with these sources (which are public, so you can try this yourself):

deb https://debathena.mit.edu/apt precise debathena
deb-src https://debathena.mit.edu/apt precise debathena
deb https://debathena.mit.edu/apt quantal debathena
deb-src https://debathena.mit.edu/apt quantal debathena
deb https://debathena.mit.edu/apt raring debathena
deb-src https://debathena.mit.edu/apt raring debathena
deb https://debathena.mit.edu/apt saucy debathena
deb-src https://debathena.mit.edu/apt saucy debathena
deb https://debathena.mit.edu/apt trusty debathena
deb-src https://debathena.mit.edu/apt trusty debathena
deb https://debathena.mit.edu/apt utopic debathena
deb-src https://debathena.mit.edu/apt utopic debathena
deb https://debathena.mit.edu/apt vivid debathena
deb-src https://debathena.mit.edu/apt vivid debathena

I get:

$ sudo rm /var/lib/apt/lists/debathena*
$ sudo apt-get update

W: Size of file /var/lib/apt/lists/partial/debathena.mit.edu_apt_dists_quantal_debathena_source_Sources.bz2 is not what the server reported 31194 16209
W: Size of file /var/lib/apt/lists/partial/debathena.mit.edu_apt_dists_saucy_debathena_source_Sources.bz2 is not what the server reported 31029 24393
W: Size of file /var/lib/apt/lists/partial/debathena.mit.edu_apt_dists_trusty_debathena_source_Sources.bz2 is not what the server reported 26334 16209
W: Size of file /var/lib/apt/lists/partial/debathena.mit.edu_apt_dists_utopic_debathena_source_Sources.bz2 is not what the server reported 26237 16209
W: Size of file /var/lib/apt/lists/partial/debathena.mit.edu_apt_dists_vivid_debathena_source_Sources.bz2 is not what the server reported 14 251

This does not happen with the same repositories over plain http.

Anders Kaseorg (andersk) wrote :

I submitted a patch to https://bugs.debian.org/781509.

Changed in apt (Debian):
status: Unknown → New
Changed in apt (Debian):
status: New → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.0.9.9ubuntu1

---------------
apt (1.0.9.9ubuntu1) wily; urgency=low

  * merged from debian/sid

apt (1.0.9.9) unstable; urgency=medium

  [ David Kalnischkies ]
  * parse specific-arch dependencies correctly on single-arch systems
    (Closes: 777760)
  * remove "first package seen is native package" assumption.
    Thanks to Axel Beckert for testing (Closes: 782777)

apt (1.0.9.8) unstable; urgency=medium

  [ David Kalnischkies ]
  * fix another d(e)select-upgrade typo (LP: #1399037)
  * properly handle expected filesize in https.
    Thanks to Robert Edmonds and Anders Kaseorg for initial patchs
    (Closes: 777565, 781509) (LP: #807303)
  * avoid depends on std::string implementation for pkgAcquire::Item::Mode
    (Closes: 781858)
  * demote VectorizeString gcc attribute from const to pure
  * keyids in "apt-key del" should be case-insensitive (Closes: 781696)
  * parse specific-arch dependencies correctly on single-arch systems
    (Closes: 777760)

  [ Michael Vogt ]
  * fix crash in order writing in pkgDPkgPM::WriteApportReport() (LP: #1436626)

 -- Michael Vogt <email address hidden> Thu, 07 May 2015 10:40:54 +0200

Changed in apt (Ubuntu):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.