Pbuilder gives a different md5sum for a source .diff.gz archive in result than the one it started with

[Scott Ritchie]

Binary package hint: pbuilder

To build my Wine packages on different arches, I do the following procedure:

First I modify the source to make sure it works properly. Then I build a source archive using dpkg-buildpackage -S -rfakeroot. This gives me a .dsc file, a .orig.tar.gz file, and a .diff.gz file.

I then use pbuilder on the i386 computer to build an i386 version of the package for upload to my apt repository. I then upload everything in pbuilder's results directory to my APT repository, and add them to create an i386 version of the new package.

Then, I take those .dsc, .orig.tar.gz, and .diff.gz file to an amd64 computer and tell pbuilder to build the package. Everything builds as normal, however some of the files in the pbuilder result directory will have a different md5sum than expected - namely, the .diff.gz file. Since the source archive is the same, there's no reason for this file to have changed, unless pbuilder messed with it in some way - the most likely thing is decompressing it and then rebuilding it with greater compression settings, rather than simply copying the .diff.gz file into the result folder.

This presents a problem, since the resulting .changes file now points to what should be the same .diff.gz file as the i386 package, but with a different md5sum. If I upload these files to my apt repository and try to tell reprepro to add them, reprepro will throw an error because of the conflicting "identical" file with a different md5sum.

As a workaround, so far I've been manually removing the offending line from the second .changes file (and ignoring the new .diff.gz file altogether), however a much more elegant solution is to fix pbuilder itself.

[André Klitzing (misery)]

I have the same problem here...

[Tyler Wagner (tyler)]

I have the same issue, same package. Building wine for i386 and amd64 with pbuilder. The .diff.gz has a different checksum (individual patches inside the diff are reordered) on both architectures as from the original:

<pre>tyler@baal:~/src/wine$ md5sum ~/src/wine/wine_1.0.0-2eitri1.diff.gz /var/cache/pbuilder/hardy-i386/result/wine_1.0.0-2eitri1.diff.gz /var/cache/pbuilder/hardy-amd64/result/wine_1.0.0-2eitri1.diff.gz
a14bfb87b2145cc792434fe9e930b3f6 /home/tyler/src/wine/wine_1.0.0-2eitri1.diff.gz
d977d4327e34d888c279b964fb620dbf /var/cache/pbuilder/hardy-i386/result/wine_1.0.0-2eitri1.diff.gz
2518130e20fbe2277202300e3a927e76 /var/cache/pbuilder/hardy-amd64/result/wine_1.0.0-2eitri1.diff.gz</pre>

[Tyler Wagner (tyler)]

I've confirmed the same behaviour compiling "hello".

[Tyler Wagner (tyler)]

Found the solution. By default pbuilder calls dpkg-buildpackage like so:

DEBBUILDOPTS="$DEBBUILDOPTS -rfakeroot"
dpkg-buildpackage -us -uc $DEBBUILDOPTS

That causes dpkg-buildpackage to rebuild the diff.gz and .dsc files. Add a -b in there, and it won't. It also means the resulting .changes file will only reference the .deb file. Which is what you want, I think.

The easy solution is to add a line to your .pbuilderrc:

DEBBUILDOPTS="-b"

[Loïc Minier (lool)]

You should be calling pbuilder as usually for the first upload to your archive (will include source) and with --binary-arch for all subsequent uploads.

It might be interesting to see why the .diff.gz is modified; it could either be a difference in the output on i386 and on amd64, or it could be a timestamp in the gz stream.

[Scott Ritchie (scottritchie)]

By the way my current workflows gets around the above issue entirely: I upload the source package to a PPA, and then it spews out .changes files for each binary that don't refer to the .diff.gz

[Tyler Wagner (tyler)]

Loïc, the issue seems to be the way diff parsed the directories for comparison. The difference between the two .diff.gz files is the order of files. My guess is that it compares directories in directory order, not sorted by name.

[Loïc Minier (lool)]

@Tyler: that'd be a good explanation; however pbuilder just calls into dpkg-dev (via dpkg-source/dpkg-buildpackage) to create/unpack .diff.gzs; I suspect we should address this in dpkg-dev.

[Raphaël Hertzog (hertzog)]

This has been fixed upstream in dpkg 1.15.8.

[Maarten Bezemer (veger)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.
Your issue should have been fixed in dpkg a while ago.
It would help us a lot if you could test it on a currently supported Ubuntu version. When you test it and it is fixed for you, kindly upload the updated logs by running apport-collect 118180 and any other logs that are relevant for this particular issue.

[Guillaume Hain (zedtux)]

@Tyler thank you so much!

[Scott Ritchie (scottritchie)]

Closing since it's presumed fixed.