apt throws bzip2/gzip errors for duplicated sources.list entries or duplicated componenes (race condition?)

[ifergus]

Reproducable with a /etc/apt/sources.list like this:

deb http://archive.ubuntu.com/ubuntu/ feisty main restricted
deb http://archive.ubuntu.com/ubuntu/ feisty main

---------------
While attempting to upgrade using the update manager started with gnome terrminal I recieved the following error:

 Failed to fetch http://archive.canonical.com/ubuntu/dists/feisty-commercial/main/binary-i386/Packages.bz2 Sub-process bzip2 returned an error code (2)

[Michael Vogt (mvo)]

Thanks for your bugreport.

Can you please attach the files in /var/lib/apt/lists/partial to this report? Are you behind a proxy?

thanks,
 Michael

[ifergus (ifergus)]

Hi Micheal,
that was quick. The folder /var/lib/apt/lists/partial contains no
visible or hidden files and no I am not behind a proxy and have never
had any download problems with the ubuntu download manager.
Thanks,
Ian

On Tue, 2007-04-03 at 19:02 +0000, Michael Vogt wrote:
> Thanks for your bugreport.
>
> Can you please attach the files in /var/lib/apt/lists/partial to this
> report? Are you behind a proxy?
>
> thanks,
> Michael
>
> ** Changed in: Ubuntu
> Status: Unconfirmed => Needs Info
>

[Michael Vogt (mvo)]

Thanks for this additional information.

Could you please attach the files in /etc/apt/sources.list and /etc/apt/sources.list.d/ to this bugrpeort? I wonder if this is caused by a duplicated sources.list entry (we had simialar bug in the past that were triggered by it).

Thanks,
 Michael

[ifergus (ifergus)]

Here are the requested files the folder /etc/apt/sources.list.d/ is
empty so i sent the other files in the first folder:

Ian

## Add comments (##) in front of any line to remove it from being checked.
## Use the following sources.list at your own risk.

deb cdrom:[Ubuntu 6.10 _Edgy Eft_ - Release i386 (20061025)]/ edgy main restricted
deb http://archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse

# deb http://archive.ubuntu.com/ubuntu edgy-proposed main restricted universe multiverse

## MAJOR BUG FIX UPDATES produced after the final release
deb http://archive.ubuntu.com/ubuntu/ edgy-updates main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ edgy-updates main restricted universe multiverse

## UBUNTU SECURITY UPDATES
deb http://security.ubuntu.com/ubuntu edgy-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu edgy-security main restricted universe multiverse

## BACKPORTS REPOSITORY (Unsupported. May contain illegal packages. Use at own risk.)
deb http://archive.ubuntu.com/ubuntu/ edgy-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ edgy-backports main restricted universe multiverse

## PLF REPOSITORY (Unsupported. May contain illegal packages. Use at own risk.)
deb http://medibuntu.sos-sts.com/repo/ edgy free
deb http://medibuntu.sos-sts.com/repo/ edgy non-free
deb-src http://medibuntu.sos-sts.com/repo/ edgy free
deb-src http://medibuntu.sos-sts.com/repo/ edgy non-free

## CANONICAL COMMERCIAL REPOSITORY (Hosted on Canonical servers, not Ubuntu
## servers. RealPlayer10, Opera, DesktopSecure and more to come.)
deb http://archive.canonical.com/ubuntu edgy-commercial main

## Listen

deb http://easyubuntu.cafuego.net main easyubuntu
## Automatix repo

#AUTOMATIX REPOS START

deb http://archive.canonical.com/ubuntu edgy-commercial main

deb http://dl.google.com/linux/deb/ stable non-free

deb http://archive.ubuntu.com/ubuntu/ edgy-security main restricted universe multiverse
#AUTOMATIX REPOS END
deb http://ubuntu.geole.info/ edgy universe multiverse

deb http://archive.ubuntu.com/ubuntu/ edgy-proposed restricted main multiverse universe
deb-src http://archive.ubuntu.com/ubuntu/ edgy-proposed restricted main multiverse universe
deb http://ubuntu.geole.info/ edgy universe multiverse
deb http://ntfs-3g.sitesweetsite.info/ubuntu/ edgy main main-all
deb http://flomertens.keo.in/ubuntu/ edgy main main-all
deb http://mirror.ubuntulinux.nl edgy-seveas all

## Add comments (##) in front of any line to remove it from being checked.
## Use the following sources.list at your own risk.

deb cdrom:[Ubuntu 6.10 _Edgy Eft_ - Release i386 (20061025)]/ edgy main restricted
deb http://archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse

# deb http://archive.ubuntu.com/ubuntu edgy-proposed main restricted universe multiverse

...

[ifergus (ifergus)]

I solved the problem by deleting doubled sources and turning off third party sources during upgrade.

[Michael Vogt (mvo)]

This problem should be fixed in my local tree, it will be uploaded to feisty+1 (and we may consider a version for feisty-updates too)

[X (u78qir8a9-deactivatedaccount)]

This should be fixed in feisty urgently in my opinion. I managed to "trigger this condition" on a fresh feisty install I was doing for a friend. I didn't edit sources.list or anything, I just clicked around in repository settings and synaptic. I suspect gnome-app-install introduced the duplicate.

[Phillip Calvin (phillipc)]

I agree with Ulrik: I have also never modified my sources.list except via the Synaptic interface.

[Dave (dh-iucr)]

It took me ages to find this thread. Just to add I've also just had this problem on a fresh install of 7.04

Lots of people suggesting clearing /var/lib/apt/lists/partial but it was empty on mine anyway.

After implement the suggested commenting out above it still failed but suggested I use the --fix-missing option

when I do this I get lots of

Failed to fetch http://....deb MD5Sum mismatch

[zenon (eahm04)]

I have same proplem

http://archive.canonical.com/ubuntu/dists/feisty-commercial/main/binary-i386/Packages.bz2: MD5Sum mismatch

/var/lib/apt/lists/partial

[Rick Silva (silvari)]

I have encountered the same problem (bzip2 error being thrown when trying to upgrade [from edgy to feisty]) and I believe there may be a cause for this, other than "duplicated entries in sources.list file"

Detailed description of the problem I've encountered can be found here:
http://ubuntuforums.org/showthread.php?t=660619

In trying to implement the solution suggested here above, I messed around quite extensively with my original sources.list file (i.e. trying to remove any duplication) but none of my changes produced a different result. So i got a new auto-generated sources.list, and am still encountering the same failure. The new vanilla sources.list is attached.