Lots of packages shipping with broken md5sums

[Jürgen Kreileder]

This problem is easily noticeable on oneiric if you install the debsums package and run 'debsums -s'

On a server system with only very small set of packages installed, I see broken md5sums for these packages:
% sudo debsums -s
debsums: changed file /usr/share/doc/libaprutil1-dbd-sqlite3/changelog.Debian.gz (from libaprutil1-dbd-sqlite3 package)
debsums: changed file /usr/share/doc/libaprutil1-ldap/changelog.Debian.gz (from libaprutil1-ldap package)
debsums: changed file /usr/share/doc/libxcb-render0/changelog.Debian.gz (from libxcb-render0 package)
debsums: changed file /usr/share/doc/libxcb-shm0/changelog.Debian.gz (from libxcb-shm0 package)
debsums: changed file /usr/share/doc/perl/changelog.Debian.gz (from perl-base package)
debsums: changed file /usr/share/doc/racoon/changelog.Debian.gz (from racoon package)

On a desktop system you'll see a *lot* more problems. See here for example: http://ubuntuforums.org/showthread.php?t=1859706 (report from another person who noticed the same problem)

Some of the wrong checksums are probably caused by including checksums for symlinked files from other packages into md5sums.
Example:

% ls -l /usr/share/doc/libxcb-shm0/changelog.Debian.gz
lrwxrwxrwx 1 root root 30 2011-10-14 21:45 /usr/share/doc/libxcb-shm0/changelog.Debian.gz -> ../libxcb1/changelog.Debian.gz
% aptitude download libxcb-render0
Get: 1 http://archive.ubuntu.com/ubuntu/ oneiric/main libxcb-render0 amd64 1.7-3 [11.9 kB]
Fetched 11.9 kB in 0s (146 kB/s)
% dpkg -x libxcb-render0_1.7-3_amd64.deb .
% dpkg -e libxcb-render0_1.7-3_amd64.deb
% grep changelog.Debian.gz DEBIAN/md5sums
15276f194d3ca77e0c9682105f2f004c usr/share/doc/libxcb-render0/changelog.Debian.gz
% md5sum usr/share/doc/libxcb-render0/changelog.Debian.gz
md5sum: usr/share/doc/libxcb-render0/changelog.Debian.gz: No such file or directory
% ls -l usr/share/doc/libxcb-render0/changelog.Debian.gz
lrwxrwxrwx 1 jk jk 30 2011-06-11 01:23 usr/share/doc/libxcb-render0/changelog.Debian.gz -> ../libxcb1/changelog.Debian.gz

But there are also packages that actually have wrong md5sums for files included in the package.
Example:

% aptitude download perl-base
Get: 1 http://archive.ubuntu.com/ubuntu/ oneiric/main perl-base amd64 5.12.4-4 [1,430 kB]
Fetched 1,430 kB in 0s (2,440 kB/s)
% dpkg -x perl-base_5.12.4-4_amd64.deb .
% dpkg -e perl-base_5.12.4-4_amd64.deb
% grep changelog.Debian.gz DEBIAN/md5sums
b060dab6fca4e84eceacdefb619fa79b usr/share/doc/perl/changelog.Debian.gz
% md5sum usr/share/doc/perl/changelog.Debian.gz
ec4b1914bcbe7a3143e0b16f81b9a329 usr/share/doc/perl/changelog.Debian.gz

Both of this cases should be fixed.

[Raphaël Hertzog (hertzog)]

It's strange indeed, but it's not the fault of dpkg. dpkg is not generating the md5sums files. I guess the problem is related to pkgbinarymangler. IIRC it shortens the changelog.Debian.gz files, and optimizes .png file, and other operations like this.

[Martin Pitt (pitti)]

The wrong md5sums for changelog symlinks was fixed in bug 795845:

pkgbinarymangler (100) oneiric; urgency=low

  * test/run: Add forgotten check_deb_integrity() to documentation symlink
    tests. This catches the wrong md5sums of symlinked doc files.
  * pkgstripfiles: Remove doc files from md5sums which get replaced with a
    symlink. (LP: #795845)
  [...]

So we need no-change rebuilds of these packages. Most of them will fix themselves with the merges/syncs in precise. However, I'll keep one task open to do no-change rebuilds for the remaining packages after the autosyncs are done.

[Martin Pitt (pitti)]

I verified that rebuilding consolekit against current pkgbinarymangler fixes this. Opening task for another affected package to keep an open task for beta-1 to fix the rest.

[Launchpad Janitor (janitor)]

This bug was fixed in the package consolekit - 0.4.5-1build1

---------------
consolekit (0.4.5-1build1) precise; urgency=low

  * No-change rebuild to fix changelog md5sum. (LP: #875466)
 -- Martin Pitt <email address hidden> Wed, 09 Nov 2011 12:58:18 +0100

[Sebastien Bacher (seb128)]

gnome-media got merged on debian and uploaded since, closing the bug

[Martin Pitt (pitti)]

Adding another affected package to keep an open task.

[Martin Pitt (pitti)]

lp-i is fine as well now. Moving task to an affected package.

[Martin Pitt (pitti)]

I checked the whole main archive now, this is the list of remaining packages with broken md5sums:

gparted (usr/share/gnome/help/gparted/ru/figures/*.png)

/usr/share/doc/* symlinks:

dictd
ggzcore-bin
iodbc
libbsd0-dbg
libbsd-dev
libesmtp-dev
libgc-dev
libggzcore-dev
libggz-dev
libggzmod4
libgpgme11-dev
libgsl0-dbg
libice6-dbg
libice-dev
libiodbc2-dev
libiw-dev
libmeanwhile-dev
libpaper-dev
libpaper-utils
librcc-dev
librccgtk2-0
librsync-dbg
librsync-dev
libsm6-dbg
libsm-dev
libtokyocabinet-dbg
libtokyocabinet-dev
libxbase2.0-dev
libxcb-ewmh1-dev
libxcb-icccm4-dev
libxcomposite1-dbg
libxcomposite-dev
libxdamage1-dbg
libxdamage-dev
libxext6-dbg
libxext-dev
libxft2-dbg
libxft-dev
libxinerama1-dbg
libxinerama-dev
libxml++2.6-dbg
libxml++2.6-dev
libxrender1-dbg
libxrender-dev
libxt6-dbg
libxt-dev
libxxf86vm1-dbg
libxxf86vm-dev
perl
pptpd
rdiff
wireless-tools

[Martin Pitt (pitti)]

Affected source packages:

dictd
ggz-client-libs
gpgme1.0
gsl
libbsd
libesmtp
libgc
libggz
libice
libiodbc2
libpaper
librcc
librsync
libsm
libxbase
libxcomposite
libxdamage
libxext
libxinerama
libxml++2.6
libxrender
libxt
libxxf86vm
meanwhile
perl
pptpd
tokyocabinet
wireless-tools
xcb-util-wm
xft

[Martin Pitt (pitti)]

I uploaded the first batch of rebuilds: ggz-client-libs libesmtp libggz librcc libxbase xcb-util-wm

These are the packages which are not seeded anywhere and thus can build during the beta-1 freeze.

[Martin Pitt (pitti)]

This is the list of universe packages which are affected:

a52dec
adplug
ballz
cfengine2
cfengine3
cipux-cat-web
cipux-object
cipux-task
clam
clutter-imcontext
cminpack
cwiid
dahdi-tools
dante
dirac
dmedia
dmtcp
e17
eb
ebnetd
edbus
edje
efreet
eina
faifa
flatzebra
freeplane
ftplib
fts
g15daemon
gdome2
geocode-glib
gfcui
gloox
glrr
glrr-widgets
gnuplot
goocanvasmm
gpe-expenses
gsl
gts
gwc
ibm-3270
ike
jets3t
lib3ds
libantlr3c
libccrtp
libcdaudio
libchipcard
libcrypto++
libdkim
libepsilon
libewf
libfakekey
libg15render
libgcal
libgdf
libgpeschedule
libhtp
libitl
liblouisxml
liblrdf
libmsv
libopkele
libppd
libsmi
libsoundgen
libtrace3
libuser
libview
libwebcam
libxr
libzn-poly
libzrtpcpp
love
lua-gtk
maxima
mono-basic
music
necpp
newlib
ntl
ode
oflib
ogdi-dfsg
ois
opencascade
openh323
orange
perlindex
picard-tools
pilot-qof
poedit
protobuf-c
pyxine
raincat
rtai
ruby-addressable
ruby-algorithm-diff
ruby-amazon-ec2
ruby-amrita
ruby-amrita2
ruby-barby
ruby-bdb
ruby-bsearch
ruby-color-tools
ruby-feedparser
ruby-globalhotkeys
ruby-gnuplot
ruby-hdfeos5
ruby-hmac
ruby-maruku
ruby-mecab
ruby-ncurses
ruby-netcdf
ruby-net-scp
ruby-net-sftp
ruby-net-ssh-multi
ruby-odbc
ruby-romkan
ruby-rspec
ruby-setup
ruby-termios
ruby-tmail
ruby-transaction-simple
ruby-will-paginate
ruby-wirble
scamper
sfst
sparsehash
sprng
squidguard
ssh-contact
tacacs+
tcltrf
tokyocabinet
twolame
vde2
webcit
wesnoth-1.8
xcb-util-image
xcb-util-keysyms
xmlsec1
xsp

[Martin Pitt (pitti)]

I uploaded the remaining batch now.

libxt (1:1.1.1-2build1) precise; urgency=low

  * No-change rebuild against current pkgbinarymangler to fix broken
    md5sums. (see LP #875466)

[Phillip Susi (psusi)]

Can this be SRUed to Oneiric?