Test suite doesn't support libxml2 builds without HTTP or zlib

[Nick Wellnhofer]

HTTP support will eventually be removed from libxml2. Support for zlib and lzma maybe as well.

It would be nice if the lxml test suite could skip affected tests if the features aren't available, preferably with a runtime check using xmlHasFeature(XML_WITH_HTTP) and xmlHasFeature(XML_WITH_ZLIB).

Regarding output compression, lxml already has some code to compress output using Python's gzip module. It might make sense to always handle compression directly instead of relying on libxml2.

[scoder (scoder)]

Thanks for the notification, Nick.

lxml can probably live without HTTP support in libxml2, given that most HTTP access actually means HTTPS these days.

But the nice thing about direct gzip support is that there's neither Python object overhead for the compression/file access, nor a dependency on the Python GIL, nor the need to hold larger uncompressed output chunks in memory. I'd rather start talking to zlib directly than losing all that. However, linking against zlib means even more complication for the build process. The same applies to lzma. While both are part of the normal Python installation, linking against the (sometimes) Python shipped C libraries isn't easy or even possible, depending on the platform.

I'll add at least feature flags to the API so that users can detect if support is available.

[Nick Wellnhofer (nick-aevum)]

Regarding compression support, one of the major issues is that libxml2 tries to decompress input files by default without any API controlling the behavior. This allows trivial DoS attacks with zip bombs. We could add new API features like disabling decompression, but this would ultimately require changes in 100+ downstream projects which seems unrealistic. At some point, libxml2 should be secure-by-default and disable automatic decompression.