One additional thing we noticed is that the http generator is very naive in that it assumes *anything* it downloaded (be it a 404 error page, or perhaps a gz-format file, or anything else you point it to) is actually a .xz in disguise. It blindly trusts the user and will simply rename (to $name-$checksum.tar.xz) and place in the pool the given file.
This leads to the above error or similar when generating deltas, because the generate_delta method also assumes that it will only ever deal with xz files (does tools.xz_uncompress without catching exceptions or checking actual magic or mime types).
I think the http generator could check the file to ensure it's .xz actually, and if it isn't, output a message and discard the file. Otherwise the pool gets quite messed up. It's worth looking at our expectations for files in the pool (i.e. are they always xz'd tars?) so maybe http generator could further validate them.
One additional thing we noticed is that the http generator is very naive in that it assumes *anything* it downloaded (be it a 404 error page, or perhaps a gz-format file, or anything else you point it to) is actually a .xz in disguise. It blindly trusts the user and will simply rename (to $name-$ checksum. tar.xz) and place in the pool the given file.
This leads to the above error or similar when generating deltas, because the generate_delta method also assumes that it will only ever deal with xz files (does tools.xz_uncompress without catching exceptions or checking actual magic or mime types).
I think the http generator could check the file to ensure it's .xz actually, and if it isn't, output a message and discard the file. Otherwise the pool gets quite messed up. It's worth looking at our expectations for files in the pool (i.e. are they always xz'd tars?) so maybe http generator could further validate them.