element with xmlns attribute is not rendered properly

[Tedd Terry]

lxml 4.4.0 introduces an issue with the following XML:

<?xml version="1.0" encoding="utf-8"?>
<NAMM_PO version="2009.2" xmlns="http://namm.com/PO/2009.2">
<Id>TEST_ID</Id>
<NAMM_PO>

This is the output XML:

<?xml version="1.0" encoding="utf-8"?>
<NAMM_PO version="2009.2" xmlns:="http://namm.com/PO/2009.2">
<Id>TEST_ID</Id>
<NAMM_PO/></NAMM_PO>

Note the missing closing tag of the NAMM_PO element, the additional NAMM_PO element, and the addition of a colon to the xmlns attribute.

Version info:

Python : sys.version_info(major=2, minor=7, micro=10, releaselevel='final', serial=0)
lxml.etree : (4, 4, 0, 0)
libxml used : (2, 9, 9)
libxml compiled : (2, 9, 9)
libxslt used : (1, 1, 33)
libxslt compiled : (1, 1, 33)

To repro:

Open and unarchive attached 'lxml_4_4_0_bug.zip'. Create a virtual environment, install requirements, and run 'python repro.py' to produce the above output.

[scoder (scoder)]

The input is not well-formed XML, so I guess you used the "recover" option to parse it at all.
Just reject invalid input instead.

[Tedd Terry (tterry)]

Sorry, my mistake, I messed up the input while trying to trim it down for a repro case.

There is definitely a bug (or unexpected change in behavior) that reproduces with lxml 4.4.0 and not lxml 4.3.5 or earlier.

Please find updated well formed XML with the same issue which caused XML parsed with lxml 4.4.0 to be rejected by a .NET service further down the line. Again, note the colon in the declaration of the xmlns attribute of the NAMM_PO element:

Input XML:
<?xml version="1.0" encoding="utf-8"?>
<NAMM_PO version="2009.2" xmlns="http://namm.com/PO/2009.2">
<Id>TEST_ID</Id>
</NAMM_PO>

Output XML (lxml 4.4.0):
<?xml version="1.0" encoding="utf-8"?>
<NAMM_PO version="2009.2" xmlns:="http://namm.com/PO/2009.2">
<Id>TEST_ID</Id>
</NAMM_PO>

Output XML (lxml 4.3.5):
<?xml version="1.0" encoding="utf-8"?>
<NAMM_PO version="2009.2" xmlns="http://namm.com/PO/2009.2">
<Id>TEST_ID</Id>
</NAMM_PO>

[scoder (scoder)]

Sorry, but I'm not going to dig through BeautifulSoup to find out how (and why) it is generating this output. I'm pretty sure it's not lxml doing that.

[Tedd Terry (tterry)]

You don't have to dig too far... here's where lxml is invoked in the library: https://bazaar.launchpad.net/~leonardr/beautifulsoup/bs4/view/head:/bs4/builder/_lxml.py

This bug doesn't reproduce if I use lxml 4.3.5 with BeautifulSoup so it sounds like an lxml issue to me.

[scoder (scoder)]

It's BS that generates the output here, though. It shouldn't write out "xmlns:=…". If you can find the reason why it does that, we'll know if the problem is in lxml or BS.

[Gleb (punkerpunker)]

Experiencing exact same problem. Originated for me when upgrading from lxml==4.3.5 to lxml==4.7.1.
BeatufiulSoup version remains unchanged. Since from x.y.z "y" was changed, it shouldn't break backward compatibility when actually it does (if we think that the BS is the root of the issue). Otherwise, sounds like an lxml issue.

Will try to dig deeper, anyway

[Paride Legovini (paride)]

Thanks, marked as Incomplete waiting for further information.

[Gleb (punkerpunker)]

One thing to mention - seems like the issue is resolved when using bs4==4.10.0 (could be reproduced with bs4==4.7.1)

[scoder (scoder)]

> issue is resolved when using bs4==4.10.0

Thanks for investigating. That means that it's no longer an issue then, right?
Closing this as a third-party issue.

[Gleb (punkerpunker)]

I think so, thank you for your help.

[Launchpad Janitor (janitor)]

[Expired for beautifulsoup4 (Ubuntu) because there has been no activity for 60 days.]

[Paride Legovini (paride)]

Jammy ships bs4 4.10.0:

 beautifulsoup4 | 4.10.0-2 | jammy | source

so I'm marking the beautifulsoup4 devel task as Fix Released.