lxml wheels posted to pypi today causing etree.so failures

[Michael Johnson]

The new wheels for lxml uploaded today appear to be causing failures:

kernel: traps: openstack[17118] trap invalid opcode ip:7f77d0231bee sp:7ffd5214a1d8 error:0 in etree.so[7f77d0013000+4b7000]

File Type Py Version Uploaded on Size
lxml-3.7.0-cp27-cp27m-manylinux1_x86_64.whl (md5, pgp) Python Wheel 2.7 2016-12-22 6MB
lxml-3.7.0-cp27-cp27mu-manylinux1_x86_64.whl (md5, pgp) Python Wheel 2.7 2016-12-22 6MB
lxml-3.7.0-cp33-cp33m-manylinux1_x86_64.whl (md5, pgp) Python Wheel 3.3 2016-12-22 6MB
lxml-3.7.0-cp34-cp34m-manylinux1_x86_64.whl (md5, pgp) Python Wheel 3.4 2016-12-22 6MB
lxml-3.7.0-cp35-cp35m-manylinux1_x86_64.whl (md5, pgp) Python Wheel 3.5 2016-12-22 6MB
lxml-3.7.0.tar.gz (md5, pgp) Source 2016-12-10 3MB

The wheel download:
lxml-3.7.0-cp27-cp27mu-manylinux1_x86_64.whl

This is causing many OpenStack projects to fail their test gates.
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22lib%2Fkeystone%3Acreate_keystone_accounts%3A1%20%20%3A%20%20%20exit_trap%5C%22

[Matt Riedemann (mriedem)]

A link to the failure in a CI job:

http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/syslog.txt.gz#_Dec_22_22_25_10

[Matt Riedemann (mriedem)]

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22trap%20invalid%20opcode%5C%22%20AND%20message%3A%5C%22in%20etree.so%5C%22&from=7d

[vdloo (rickvandeloo)]

We also experience repeatable 'Illegal instruction' core dumps (on a Django project)

(venv)jenkins@hn-integration5:/tmp/tmptY9EFF/pino$ gdb --args venv/bin/python manage.py
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from venv/bin/python...(no debugging symbols found)...done.
(gdb) r
Starting program: /tmp/tmptY9EFF/pino/venv/bin/python manage.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x00007ffff1a34bee in xmlRegisterInputCallbacks () from /tmp/tmptY9EFF/pino/venv/local/lib/python2.7/site-packages/lxml/etree.so

[Bo Bayles (bbayles)]

The offending wheels have now been removed from PyPI, so I think the harm from this issue is mitigated. Many thanks for fixing.

Just a comment: This was quite confusing in that pinning an lxml dependency to 3.7.0 didn't protect against the issue.

[scoder (scoder)]

I uploaded a new "binary-only" release 3.7.1 with conservatively built wheels. Please test.