[jaunty] lxml FTBFS on all platforms

[Morten Kjeldgaard]

These are the key lines in the build logs:

running build_ext
error: Cython does not appear to be installed on platform 'posix'
[22293 refs]
make: *** [dbg-build-python2.4] Error 1
dpkg-buildpackage: failure: debian/rules build gave error exit status 2

[Morten Kjeldgaard (mok0)]

Not solved yet, but this is a far as I've come. I've done some experiments on my cleanroom jaunty sbuilder.

* The error does NOT happen when the package is built with cython_0.9.8-1. Thus it seems to be a regression in cython_0.10.2-1
* The error happens when setup.py is run by the -dbg version of Python.
* The error message comes from the file: /usr/share/python-support/cython/Cython/Distutils/build_ext.py, which is a part of cython. This file has not changed at least since cython_0.9.6.14-1 in intrepid (same md5sum)

[scoder (scoder)]

During installation, Cython now compiles some of its own modules into C extensions to speed up compilation (by quite a bit, actually). If it can't import them for some reason (e.g. they were built for a different Python version), you will get the error above.

BTW, this does not affect lxml as long as you only use released sources (which do not require Cython).

[Morten Kjeldgaard (mok0)]

Since 2.1.1-2.1 lxml packaging was changed to always re-compile the
*.pyx files (see
http://packages.debian.org/changelogs/pool/main/l/lxml/current/changelog )

[Morten Kjeldgaard (mok0)]

Attached is a patch that fixes the FTBFS problem with lxml. What I've done is to avoid the recompilation of the *.pyx files with cython, thus removing the need for cython in Build-Depends.

Cythonizing the *.pyx files was introduced as a bug-fixing measure because of a bug in one of the .pyx files.

In general, however, it is probably a bad idea to cythonize the *.pyx files, since upstream already provides compiled *.c files in the tarball, appropriately compiled with whatever cython version upstream currently uses. Thus, It is the .c files that have been debugged and tested by upstream to function with the lxml modules,

If the *.pyx modules are automatically re-compiled with a random version of cython, there is no guarantee that the lxml modules will still work as intended.

[scoder (scoder)]

[regarding lxml recompilation:]
I hope the Debian package users of lxml are happy with untested,
unreleased sources. Cython is a quickly moving target that can generate
very different code from one day to another. lxml is rarely released with
C code generated from an official Cython release, so regenerating the
sources can introduce all sorts of diverging behaviour.

[Morten Kjeldgaard (mok0)]

Invalidating this bug in the Cython project, as it's really not a bug there.

[Matthias Klose (doko)]

> regarding lxml recompilation:]
> I hope the Debian package users of lxml are happy with untested,
> unreleased sources. Cython is a quickly moving target that can generate
> very different code from one day to another. lxml is rarely released with
> C code generated from an official Cython release, so regenerating the
> sources can introduce all sorts of diverging behaviour.

Debian wants to be able to build from source, and be able to change the source; if this cannot be done, then lxml has to be removed from Debian. It's unfortunate that lxml relies on such unstable software.

[scoder (scoder)]

> Debian wants to be able to build from source, and be able to change the
> source; if this cannot be done, then lxml has to be removed from Debian.

It can be done, lxml should generally work just fine with the latest
release version of Cython. It's just that using a different version won't
yield the same C sources, so even small source changes won't result in a
local change in the binary. And releasing lxml with a specific Cython
version won't solve the problem as Debian would want to build it with the
Cython version it ships, as would Ubuntu, and Redhat, and ...

> It's unfortunate that lxml relies on such unstable software.

gcc generates different code depending on which version is used. Sometimes,
using one gcc version can result in a crash, while other gcc versions work
just fine. Does that make GCC unstable software? Updating GCC can lead to
non-local changes in binaries, just like updating Cython does. I think the
only difference is that GCC is a much slower moving target than Cython.

[Matthias Klose (doko)]

> gcc generates different code depending on which version is used. Sometimes,
> using one gcc version can result in a crash, while other gcc versions work
> just fine. Does that make GCC unstable software?

GCC is available as a versioned package in the distribution, so you have the choice of using an earlier version. cython is not packaged this way. so if lxml requires a legacy version of cython, we have to package this one for Debian as well.

[Matthias Klose (doko)]

fixed in 2.0.5-1, doesn't need to be fixed in cython.