Seeing out of memory errors after libexpat1 upgrade to 2.2.5-3ubuntu0.4 (bionic) or 2.2.9-1ubuntu0.2 (focal)

[Francis Ginther]

I started seeing errors parsing XML files today (Feb 22, 2022) after my system was updated to 2.2.5-3ubuntu0.4. This is on a bionic server.

The parsing is being done by python3's xmltodict module, which uses python3 expat as the actual parser. This is the error it raises:

xml.parsers.expat.ExpatError: out of memory: line 1, column 0

So far this is happening on multiple xml files, although they all come from the same source (these are jenkins config.xml files). I'm working on coming up with a minimal test case which I'll provide once I have it cleaned up of any private data.

[System info]
$ lsb_release -rd
Description: Ubuntu 18.04.6 LTS
Release: 18.04

$ apt-cache policy libexpat1
libexpat1:
  Installed: 2.2.5-3ubuntu0.4
  Candidate: 2.2.5-3ubuntu0.4
  Version table:
 *** 2.2.5-3ubuntu0.4 500
        500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2.2.5-3 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages

[Francis Ginther (fginther)]

I was able to workaround the issue by downgrading to the release version of the package:

$ sudo apt-get install libexpat1=2.2.5-3 libexpat1-dev=2.2.5-3

[Francis Ginther (fginther)]

Attached is an example config file and python script which demonstrates the parsing error. This was reproduced on focal with:

$ apt-cache policy libexpat1
libexpat1:
  Installed: 2.2.9-1ubuntu0.2
  Candidate: 2.2.9-1ubuntu0.2
  Version table:
 *** 2.2.9-1ubuntu0.2 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2.2.9-1build1 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

This example worked with 2.2.9-1build1 but not 2.2.9-1ubuntu0.2.

To run the example:

$ ./parser_example.py ./sample-config.xml

The sample-config.xml was generated by jenkins (version 2.303.2) of a freestyle project with only the job name and description set.

[Francis Ginther (fginther)]

This may be an issue with the python3 xmltodict module. Further testing indicates that parsing a file with just xml.parsers.expat is working just fine.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Francis Ginther (fginther)]

Moving this to xmltodict as I've been able to parse multiple xml files through just python3's xml.parsers.expat without error, but none of them through xmltodict.

[Po-Hsu Lin (cypressyew)]

Note that version 2.2.9-1ubuntu0.2 has gone on bionic.

[Valentin Lorentz (progval)]

This can be reproduced without xmltodict, using only the stdlib:

>>> from xml.parsers import expat
>>> parser = expat.ParserCreate(namespace_separator=':')
>>> parser.Parse('<foo></foo>', True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
xml.parsers.expat.ExpatError: out of memory: line 1, column 0

[EugeneLVB (eugenelvb)]

Confirm the problem on focal

[Olivier Feys (folfure)]

This issue is probably linked to a misuse of libexpat1 in the pyexpat python library.
Also happening on debian

[Janne Myllylä (jamyllyla)]

Libexpat1 is in wide use and reducing functionality without versioning seems like a bad idea. After this security update those "misusing" - or those depending from pyexpat now stopped working properly.

[Martin Holmes (mholmes)]

This also affects the rnv RELAXNG validator, which although a bit long in the tooth is still useful.

[Robert Coup (rcoup)]

Upstream issue: https://github.com/libexpat/libexpat/issues/572

[Robert Coup (rcoup)]

There’s an upstream fix that has been merged: https://github.com/libexpat/libexpat/pull/577

[Corey Bryant (corey.bryant)]

It seems this is fixed in Ubuntu Lunar in expat 2.5.0.

[Corey Bryant (corey.bryant)]

This is also fixed in kinetic via expat 2.4.8

[Corey Bryant (corey.bryant)]

Also fixed in jammy expat 2.4.7

[Corey Bryant (corey.bryant)]

Also fixed in focal expat 2.2.9-1ubuntu0.6

[Corey Bryant (corey.bryant)]

And fixed in bionic expat 2.2.5-3ubuntu0.9