LXML dependency needs to be updated to fix segmentation fault

[Dennis Plöger]

After upgrading our pacemaker/corosync cluster to Ubuntu 22.04 we had segmentation faults when running "crm configure". Using faulthandler to generate a python stacktrace yielded:

Fatal Python error: Segmentation fault

Current thread 0x00007efe7677a1c0 (most recent call first):
  File "/usr/lib/python3/dist-packages/crmsh/cibconfig.py", line 58 in show_unrecognized_elems
  File "/usr/lib/python3/dist-packages/crmsh/cibconfig.py", line 2779 in initialize
  File "/usr/lib/python3/dist-packages/crmsh/ui_configure.py", line 492 in requires
  File "/usr/lib/python3/dist-packages/crmsh/ui_context.py", line 265 in enter_level
  File "/usr/lib/python3/dist-packages/crmsh/ui_context.py", line 84 in run
  File "/usr/lib/python3/dist-packages/crmsh/main.py", line 196 in handle_noninteractive_use
  File "/usr/lib/python3/dist-packages/crmsh/main.py", line 249 in main_input_loop
  File "/usr/lib/python3/dist-packages/crmsh/main.py", line 370 in run
  File "/usr/sbin/crm", line 46 in <module>

Extension modules: lxml._elementpath, lxml.etree (total: 2)
Segmentation fault

At that point, the code was using the findall method of lxml._Element to search through our (valid) cib configuration.

Using pip3 install --upgrade lxml to update lxml to version 4.9.2 solved this problem.

I'm not really sure what specific bug was fixed, but I assume that it was this one in 4.9.1: https://lxml.de/4.9/changes-4.9.1.html

[Lena Voytek (lvoytek)]

Thank you for the bug report. I created a PPA containing the relevant changes in 4.9.1 that may fix this issue here: https://launchpad.net/~lvoytek/+archive/ubuntu/lxml-fix-iterwalk-crash

If you would like to test the fix you can run the following:

$ sudo add-apt-repository ppa:lvoytek/lxml-fix-iterwalk-crash
$ sudo apt update
$ sudo apt upgrade

If this works for you please let us know, thanks!

[Dennis Plöger (dploeger2000)]

Hey Lena. Thanks, but it didn't work. The segmentation fault still happens...

[Paride Legovini (paride)]

Hello Dennis, thanks for testing the PPA. The patch Lena identified was really promising: by looking at the commit messages I can't identify a better candidate between 4.9.0 and 4.9.2:

  https://github.com/lxml/lxml/compare/lxml-4.9.0...lxml-4.9.2

Could you please double check you actually have the PPA version installed, e.g. by doing

  dpkg -l | grep lxml

and checking that the installed lxml packages have ~ppa1 in their version string? Also: please make sure that while testing the PPA packages installed from pip are not interfering.

If everything looks good, could you please try installing version 4.9.1 from pip? This way we'll be able to tell which version contains the fix. You can install a specific version from pip with e.g.:

  pip3 install lxml==4.0.1

Thanks!

[Dennis Plöger (dploeger2000)]

Yes. The ppa-version is installed:

    ii python3-lxml:amd64 4.8.0-1ubuntu0.1~ppa1 amd64 pythonic binding for the libxml2 and libxslt libraries

And yes, installing lxml 4.9.1 using pip worked.

[Sergio Durigan Junior (sergiodj)]

Thanks for the followup.

I read lxml's code and commits from 4.8.0 (the version we have in Jammy) to 4.9.1 (the version you confirmed to be working). There is really nothing that stands out. There are some minor things that caught my attention, but I'm not convinced that any of them is related to this issue.

The best scenario here would be to have a reproducer for this bug. I understand that this may not be possible/feasible, so I would like to request more data from you:

- Could you provide more details about the stack trace? For example, were you able to obtain a stack trace for lxml as well, since the bug happens there?

- Can you check if the segfault happens with version 4.9.0 from pip, please? And while you're at it, can you double check that you can still reproduce the problem with version 4.8.0 *from pip*?

- I know that lxml needs C bindings to work, and it probably compiles them when you install the package using pip. I'm wondering if there's some compiler flag that's affecting the bindings. My first guess would be LTO, but I noticed that it's not enabled for the build.

[Dennis Plöger (dploeger2000)]

I'm sorry, I can't reproduce this for the time being. I don't want to migrate our cluster again just to test this and I can't reproduce it in a test VM, even when I load our own configuration.

So, honestly, I don't know what this is. Maybe it was already fixed somewhere else or something.

Thanks für your support!

[Launchpad Janitor (janitor)]

[Expired for crmsh (Ubuntu) because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for lxml (Ubuntu) because there has been no activity for 60 days.]