Ubuntu on EC2

apt-get update fails using S3-backed mirrors

Bug #940320 reported by Tobias McNulty on 2012-02-24

This bug report is a duplicate of: Bug #948461: apt-get hashsum/size mismatch because s3 mirrors don't support http pipelining correctly. Edit Remove

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	Ubuntu on EC2	Triaged	High	Unassigned

Bug Description

I got a failure trying to apt-get update using the s3-backed mirror us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com, from availability zone A. subsequent attempts worked fine but I thought it was important to highlight the potential failure (as I thought reliability was one thing the S3-backed mirror was expected to improve). The AMI is ami-fd589594 with a modified /etc/apt/sources.list.

[ec2-23-20-108-124.compute-1.amazonaws.com] sudo: apt-get update
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security InRelease
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty InRelease
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates InRelease
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:1 http://security.ubuntu.com natty-security Release.gpg [198 B]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:2 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty Release.gpg [198 B]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:3 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates Release.gpg [198 B]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:4 http://security.ubuntu.com natty-security Release [39.8 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:5 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty Release [39.8 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:6 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates Release [39.8 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:7 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main Sources [862 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:8 http://security.ubuntu.com natty-security/main Sources [89.3 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:9 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe Sources [4,380 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:10 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main amd64 Packages [1,550 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:11 http://security.ubuntu.com natty-security/universe Sources [20.9 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:12 http://security.ubuntu.com natty-security/main amd64 Packages [274 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:13 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe amd64 Packages [6,004 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main TranslationIndex
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe TranslationIndex
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:14 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main Sources [151 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:15 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main amd64 Packages [40.5 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:16 http://security.ubuntu.com natty-security/universe amd64 Packages [74.5 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:17 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe amd64 Packages [429 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/main TranslationIndex
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/universe TranslationIndex
[ec2-23-20-108-124.compute-1.amazonaws.com] out: Get:18 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main TranslationIndex [137 kB]
[ec2-23-20-108-124.compute-1.amazonaws.com] out: E: Unable to parse package file /var/lib/apt/lists/partial/us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com_ubuntu_dists_natty-updates_main_i18n_Index (1)

Revision history for this message

Tobias McNulty (tmcnulty1982) wrote on 2012-02-24:

Download full text (8.9 KiB)

Just got another failure from a different machine:

Just got another failure from a different machine:

[ec2-23-20-76-71.compute-1.amazonaws.com] sudo: apt-get update
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty InRelease
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates InRelease
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:1 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty Release.gpg [198 B]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:2 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates Release.gpg [198 B]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:3 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty Release [39.8 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:4 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates Release [39.8 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security InRelease
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:5 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main Sources [862 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:6 http://security.ubuntu.com natty-security Release.gpg [198 B]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:7 http://security.ubuntu.com natty-security Release [39.8 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:8 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe Sources [4,380 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:9 http://security.ubuntu.com natty-security/main Sources [89.3 kB]                   
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:10 http://security.ubuntu.com natty-security/universe Sources [20.9 kB]                
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:11 http://security.ubuntu.com natty-security/main i386 Packages [273 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:12 http://security.ubuntu.com natty-security/universe i386 Packages [75.1 kB]           
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/main TranslationIndex                       
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/universe TranslationIndex
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:13 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main i386 Packages [1,550 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:14 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe i386 Packages [6,021 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main TranslationIndex
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe TranslationIndex
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/main Translation-en_US
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:15 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main Sources [151 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:16 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main i386 Packages [40.5 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/main Translation-en           
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:17 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe i386 Packages [428 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/universe Translation-en_US       
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://security.ubuntu.com natty-security/universe Translation-en          
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:18 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/main TranslationIndex [137 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe TranslationIndex
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Get:19 http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe Translation-en [46.9 kB]
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Err http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe Sources
[ec2-23-20-76-71.compute-1.amazonaws.com] out:   403  Forbidden
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main Translation-en_US
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/main Translation-en
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe Translation-en_US
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe Translation-en_US
[ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty-updates/universe Translation-en
[ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [14 Packages bzip2 0 B]                                                                                               [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [14 Packages bzip2 0 B]                                                                                               [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [14 Packages bzip2 0 B]                                                                                               [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [Working]                                                                                                             [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [15 Sources bzip2 0 B]                                                                                                [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [Working]                                                                                                             [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [16 Packages bzip2 0 B]                                                                                               [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [Working]                                                                                                             [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [17 Packages bzip2 0 B]                                                                                               [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [Working]                                                                                                             [ec2-23-20-76-71.compute-1.amazonaws.com] out: 99% [19 Translation-en bzip2 0 B]                                                                                                                          2,309 kB/s 0sbzip2: (stdin) is not a bzip2 file.
[ec2-23-20-76-71.compute-1.amazonaws.com] out:                                                                                                                           [ec2-23-20-76-71.compute-1.amazonaws.com] out: Ign http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com natty/universe Translation-en
[ec2-23-20-76-71.compute-1.amazonaws.com] out: 98% [Working]                                                                                                             [ec2-23-20-76-71.compute-1.amazonaws.com] out:                                                                                                                           [ec2-23-20-76-71.compute-1.amazonaws.com] out: Fetched 14.2 MB in 7s (1,853 kB/s)
[ec2-23-20-76-71.compute-1.amazonaws.com] out: W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/dists/natty-updates/universe/source/Sources  403  Forbidden

[ec2-23-20-76-71.compute-1.amazonaws.com] out: W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com_ubuntu_dists_natty-updates_main_binary-i386_Packages  Hash Sum mismatch

[ec2-23-20-76-71.compute-1.amazonaws.com] out: W: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com/ubuntu/dists/natty-updates/main/i18n/Index  No Hash entry in Release file /var/lib/apt/lists/partial/us-east-1.ec2.archive.ubuntu.com.s3.amazonaws.com_ubuntu_dists_natty-updates_main_i18n_Index

[ec2-23-20-76-71.compute-1.amazonaws.com] out: E: Some index files failed to download. They have been ignored, or old ones used instead.

Fatal error: sudo() encountered an error (return code 100) while executing 'apt-get update'

Revision history for this message

Ben Howard (darkmuggle-deactivatedaccount) wrote on 2012-03-06:

Tobias, thanks for reporting this. This is a duplicate of Bug 948461, where APT is misnaming the local cached files.

Changed in ubuntu-on-ec2:
status:	New → Triaged
importance:	Undecided → High

Report a bug

This report contains Public information

Everyone can see this information.

Duplicate of bug #948461 Remove

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.