Comment 1 for bug 1570388

Revision history for this message
Frank Steggink (fsteggink) wrote :

I'm experiencing a similar solution. Unfortunately the links to the tracebacks don't work anymore, so I'll post mine:

Traceback (most recent call last):
  File "/usr/local/bin/stetl", line 4, in <module>
    __import__('pkg_resources').run_script('Stetl==1.3', 'stetl')
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 739, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1494, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/EGG-INFO/scripts/stetl", line 41, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/EGG-INFO/scripts/stetl", line 32, in main
    etl.run()
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/etl.py", line 159, in run
    chain.run()
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/chain.py", line 174, in run
    packet = self.first_comp.process(packet)
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/component.py", line 218, in process
    packet = self.next.process(packet)
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/component.py", line 218, in process
    packet = self.next.process(packet)
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/component.py", line 218, in process
    packet = self.next.process(packet)
  File "/usr/local/lib/python2.7/dist-packages/Stetl-1.3-py2.7.egg/stetl/component.py", line 204, in process
    packet = self.invoke(packet)
  File "/etl/bgt/stetlbgt/subfeaturehandler.py", line 146, in invoke
    xf.flush()
  File "src/lxml/serializer.pxi", line 1347, in lxml.etree.xmlfile.__exit__
  File "src/lxml/serializer.pxi", line 1685, in lxml.etree._IncrementalFileWriter._close
  File "src/lxml/serializer.pxi", line 1691, in lxml.etree._IncrementalFileWriter._handle_error
  File "src/lxml/serializer.pxi", line 199, in lxml.etree._raiseSerialisationError
lxml.etree.SerialisationError: unknown error -2055577339

Yes, I'm still using Python 2.7, and I know I need to upgrade asap, but unfortunately I'm not able to do that at this moment due to other priorities. This error still occurs in the latest version of lxml (version 4.4.2), and also in version 3.7.

I also did some investigation, and I found suspect this error is caused by the fact that libxml returns a signed 32 bit value when calling the function xmlOutputBufferClose. See http://www.xmlsoft.org/html/libxml-xmlIO.html#xmlOutputBufferClose.

My local test file which I monitored was slightly larger than 2.1 GB, or 2^31. The error code was just a bit smaller than -2^31. So I decided to sum both of them (negating the sign of the error code of course), and the sum was exactly 2^32.

I have no hope that this error will ever be fixed in lxml, since that would have major complications (or hopefully they're working on a 64 bit port). But I hope it will be feasible for lxml to have this error fixed.

The file which I'm eventually generating is valid XML, so as a workaround I decided to catch the etree.SerialisationError (which can either be caused by etree.xmlfile.flush or etree.xmlfile.__exit_), and afterwards to check whether the file is valid with etree.iterparse.