Release wheels for Python 3.11

[Jason R. Coombs]

It's that time of year again. Similar to #1929359, pipelines running on Python 3.11 are broken because they rely on wheels and don't have the logic to build lxml from source.

I see #1969912 added support for building on Python 3.11. Thanks for that.

Please provide wheels for lxml.

I understand you may be reluctant to release wheels before Python reaches RC or final. If that's the case, it would be nice if you could publish pre-release wheels somewhere they can be referenced prior to the official wheels published in PyPI.

[Sorin Sbarnea (ssbarnea)]

Any change to address this soon as I even failed to compile it after providing the header files. I suspect it might be related to me running on arm64 architecture instead of amd64.

I logged the compilation failures and more details on https://github.com/ansible/ansible-navigator/issues/1249

[Ewout ter Hoeven (ewouth)]

While the 4.9.1 release did add Manylinux and Musllinux wheels for Python 3.11, unfortunatly, there are still no Windows or macOS wheel for Python 3.11. Could those be added?

See https://pypi.org/project/lxml/#files

[Ewout ter Hoeven (ewouth)]

I created a PR to prepare for Python 3.11 wheel building: https://github.com/lxml/lxml/pull/355

However, Python 3.11 isn't available on AppVeyor yet, so I also opened a feature request for that: https://github.com/appveyor/ci/issues/3844

[Ewout ter Hoeven (ewouth)]

Unfortunately the AppVeyor build has been failing for over a month. Has anyone looked into it by any chance?

I openend an issue for this here: https://bugs.launchpad.net/lxml/+bug/1993962

[bastimeyer (bastimeyer)]

> While the 4.9.1 release did add Manylinux and Musllinux wheels for Python 3.11

Don't forget the aarch64 wheel for manylinux, which is also missing

[Cosimo Lupo (lupocos)]

Can you please also try to switch to https://cibuildwheel.readthedocs.io/en/stable/ as that takes care of all the nuances and should be capable of cross-compiling wheels for M1/M2 arm-based macOS architecture (arm64, universal2)? Github runners are x86_64 only but can cross-compile for arm64/universal2 (you won't be able to run the tests though, but that's better than not providing any wheels for Apple chips).
Thanks!

[wuTree (hpytree)]

Downloading and installing a wheel package is much easier than compiling it myself.(especially on windows) But lxml has no wheel for windows. Thank you very much.

[scoder (scoder)]

I agree that using cibuildwheel would be nice. Help with that is welcome.

Also see https://bugs.launchpad.net/lxml/+bug/1913032 regarding macOS M1/universal2 wheels

[Ewout ter Hoeven (ewouth)]

AppVeyor added Python 3.11 to their Windows images, so I opened a PR to add 3.11 jobs to the AppVeyor configuration: https://github.com/lxml/lxml/pull/360

I'll try to look at cibuildwheel at some point, but let's first get Python 3.11 wheels out.

[thijs (thijss)]

Please add macOS wheels too :).

macOS seemed to have missed the boat for python 3.10, so I've created a separate issue for that:
https://bugs.launchpad.net/lxml/+bug/1998212

[Patrick Lehmann (paebbels)]

What's taking so long to get lxml fixed for Python 3.11?
This thread looks quite sleepy for such an important package.

Last year lxml had the same problem for Python 3.10.

Btw. others have managed to build and package lxml for Python 3.11 in a wheel.
See https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

[scoder (scoder)]

lxml 4.9.2 has been released with Py3.11 wheels.

[Patrick Lehmann (paebbels)]

Thanks.

[Bob Kline (bob.kline)]

I remember saying years ago on the mailing list that I would love to see this package become part of the Python standard library. I remember the reaction I got being something like "why on earth would you want that? what possible advantage could that bring?".
😂

[scoder (scoder)]

> I remember saying years ago on the mailing list that I would love to see this package become part of the Python standard library. I remember the reaction I got being something like "why on earth would you want that? what possible advantage could that bring?".

I remember that question being discussed a couple of times. From what I recall, the main arguments at the time were "it's way too big and involves several complex external libraries" and "the release cycles don't match up well". Basically, neither the CPython devs nor I were keen to go through the hassle of integrating and maintaining lxml in the standard library. And requiring a recent Python in order to get certain lxml features (and otherwise dealing yourself with not having them available) also does not sound very tempting to many users.

Really, the standard library was never a solution to the problems at hand.