[upstream] Calc hangs indefinitely opening Excel 2003 file saved as .xml

[Thomas Novin]

Binary package hint: openoffice.org

1) lsb_release -rd
Description: Ubuntu 11.04
Release: 11.04

2) apt-cache policy libreoffice-calc
libreoffice-calc:
  Installed: 1:3.3.2-1ubuntu5
  Candidate: 1:3.3.2-1ubuntu5
  Version table:
 *** 1:3.3.2-1ubuntu5 0
        500 http://us.archive.ubuntu.com/ubuntu/ natty-proposed/main i386
Packages
        100 /var/lib/dpkg/status
     1:3.3.2-1ubuntu4 0
        500 http://us.archive.ubuntu.com/ubuntu/ natty/main i386 Packages

3) What is expected to happen in LibreOffice Calc via the Terminal:

cd ~/Desktop && wget https://bugs.launchpad.net/ubuntu/+source/openoffice.org/+bug/91759/+attachment/46342/+files/test.xml && localc -nologo test.xml

it opens quickly and successfully.

4) What happens instead is it hangs indefinitely posting:

Warning: at xsl:stylesheet on line 28 of
file:///usr/lib/libreoffice/basis3.3/share/xslt/import/spreadsheetml/spreadsheetml2ooo.xsl:
  Running an XSLT 1.0 stylesheet with an XSLT 2.0 processor
Warning: on line 4476 of
file:///usr/lib/libreoffice/basis3.3/share/xslt/import/spreadsheetml/spreadsheetml2ooo.xsl:
  Creating an attribute here will fail if previous instructions create any
children

WORKAROUND: Convert the file via Gnumeric's ssconvert and open in LibreOffice Calc:

ssconvert test.xml test.ods && localc -nologo test.ods

apt-cache policy gnumeric
gnumeric:
  Installed: 1.10.13-1ubuntu1
  Candidate: 1.10.13-1ubuntu1
  Version table:
 *** 1.10.13-1ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ natty/universe i386 Packages
100 /var/lib/dpkg/status

WORKAROUND: Use Gnumeric.

ProblemType: Bug
Date: Mon Mar 12 21:51:56 2007
DistroRelease: Ubuntu 7.04
Uname: Linux pc-thnov-ubuntu 2.6.20-9-generic #2 SMP Mon Feb 26 03:01:44 UTC 2007 i686 GNU/Linux

[Andrew Ziem (ahziem1)]

Loading the invalid Microsoft Excel 2003 XML file causes OpenOffice.org 2.0.3
Linux to hang. The issue may be the same as issue 62683.

$ xmllint /tmp/hang.xml
/tmp/hang.xml:11: parser error : Opening and ending tag mismatch: Cell line 11
and cell
<Cell><Data ss:Type="String">Test1</Data></cell>
                                                ^
/tmp/hang.xml:12: parser error : Opening and ending tag mismatch: Cell line 12
and cell
<Cell><Data ss:Type="String">Test2</Data></cell>
                                                ^

[Andrew Ziem (ahziem1)]

Created attachment 38001
Microsoft Excel 2003 XML hangs OOo 203

[kpalagin (kpalagin)]

Happens to me as well (m180 on XP SP2 english).

[Frank-l (frank-l)]

Hi Swante,

seems to be yours.

Frank

[Thomas Novin (thomasn80)]

Example of a file that cannot be opened.

[Brian Murray (brian-murray)]

I have reproduced this using openoffice.org version 2.2.0~rc3~oof680m10-0ubuntu3 with the spreadsheet you provided. I actually received a crash report and have attached it.

[Thomas Novin (thomasn80)]

Confirmed again on 2.2.0-0ubuntu2

[Dima Ryazanov (dima-gmail)]

I ran into a similar problem when opening a PowerPoint file with an embedded Excel graph - OpenOffice just freezes, using up 100% of the CPU. But, it opens fine in the same version of OpenOffice, 2.2.0, on Gentoo Linux.

The test.xml in this bug report, though, freezes OpenOffice on Gentoo, too.

[Andrew Ziem (ahziem1)]

OOo 2.2 Windows XP: no freeze, but document is blank.
OOo 2.2 Linux : no freeze, but document is blank and error printed to console:
"SystemId Unknown; Line #11; Column #44; The element type "Cell" must be
terminated by the matching end-tag "</Cell>"

[Brian Murray (brian-murray)]

Reproduced using openoffice.org version 2.2.0-1ubuntu3. An automatic crash report was generated and retraced in bug 115115. However, the retrace doesn't seem particularly useful. It would be helpful if you could try and reproduce this issue with the upstream version of OpenOffice.org. Thanks!

[Thomas Novin (thomasn80)]

How do I do that? Do I install the debian-packages by adding a debian repository for apt or do I download the original .tar.gz-file and do the install? If I should add a debian rep., do you know what entry I should add?

[Brian Murray (brian-murray)]

I setup up a Debian sid chroot following steps at https://wiki.ubuntu.com/DebootstrapChroot and installed openoffice.org version 2.2.1~rc1-1. With that version of openoffice.org I was able to reproduce the crash.

[Brett Alton (brett-alton-deactivatedaccount)]

Confirmed using openoffice.org version 2.2.1-2ubuntu1 on Ubuntu Gusty (2007-07-08).

Linux bateman 2.6.22-7-generic #1 SMP Mon Jul 9 01:12:32 EST 2007 i686 GNU/Linux.

[Thomas Novin (thomasn80)]

Confirmed using openoffice.org version 2.2.1-9162 (rpm's downloaded from openoffice.org and converted to deb using alien) on Ubuntu Feisty.

I have also opened an issue at openoffice.org, see http://www.openoffice.org/issues/show_bug.cgi?id=79650

[Thomas Novin (thomasn80)]

Confirmed using openoffice.org version 2.3.0-1ubuntu5.3.

[Nesshof (nesshof)]

move target from 2.x to 3.0

[Chris Cheney (ccheney)]

The powerpoint attachment above can now be opened with openoffice.org 1:2.4.0~rc2-1ubuntu3, however the original report of the MS Excel 2003 XML file not working is still a problem.

[Svante-schubert (svante-schubert)]

I am sorry, I am no longer working on this format.
Changed target to Office later.
Is anybody able to take this over? And/or provide a patch for this problem?

Thanks in advance,
Svante

[randall (randall-iclasspro)]

i am a web developer and made a php class to create xmlss files. i have had this lockup in openoffice v2.2/windows, 2.3/ubuntu7.10, and 3.0rc4/ubuntu7.10. it seems to be related to larger files. the files open in ms excel 2003/2007. this has to be an openoffice problem, not os specific to affect linux and windows through many versions. i am attaching a simple xml spread sheet file (one that demonstrates this issue, and opens fine in excel).

[Chris Cheney (ccheney)]

This bug is currently blocked by the xalan 2.7.1 issue 'Could not compile stylesheet'. Once this works again we can see if this bug is fixed.

[Maxremov (maxremov)]

MSO 2007 opens file attached normally. Under Ubuntu 10.10.

[Libreoffice-0 (libreoffice-0)]

Perhaps you forgot to attach your file?

[Maxremov (maxremov)]

Created attachment 44713
FILEOPEN: Spreadsheet hangs

unpack the zip

[Libreoffice-z (libreoffice-z)]

Reproduciblen with reporter's sample document and LibO 3.3.1 WIN7. I will do further tests soon.

[Libreoffice-z (libreoffice-z)]

[Reproducible] with "LibreOffice 3.3.2RC2 – WIN7 Home Premium (64bit) German UI [OOO330m19 (Build:202 / tag 3.3.2.2)]".
No Idea whether LibO supports SpreadsheetML, but of course, it should not hang or crash.

OOo-dev 3.4 also hangs, MS EXCEL Viewer was not able to open the sample document (gave message, did not hang), OOo 3.1.1 tries text-import.

@Kohei:
Can you please check the hang problem?

[Libreoffice-0 (libreoffice-0)]

I don't officially support Excel 2003 XML file format, which is what this file is. The current file uses XSLT which is not scalable & chokes on large-ish files (like this document). And the only way to make this filter perform better is to write a new one from scratch, using raw C++ and ditch the current, poor-performing XSLT based filter.

So, our current filter is "better than nothing" but please don't expect a speedy file opening experience.

[Libreoffice-0 (libreoffice-0)]

c.f. http://en.wikipedia.org/wiki/Microsoft_Office_XML_formats

[Libreoffice-0 (libreoffice-0)]

(In reply to comment #5)
> I don't officially support Excel 2003 XML file format,

I meant to say "I don't think we officially support"...

[Libreoffice-z (libreoffice-z)]

@Kohei
I agree with your comments concerning SpreadsheetML support.

Please excuse me, I strongly disagree with your classification of this bug report. The simple rule is:

  Opening a document never causes a crash or a hang.

The report shows a vulnerability of LibO, that can not be accepted. If a user tries to open a unsuspicious looking document with .xls name extensions, and suddenly the program hangs and he looses data of work from last 1/4 hour (or may be even worse), he will think some unfriendly things about LibO

Although it seems that SpreadsheetML are not so often used (it seems I never saw one before), this hang is a major problem.

So I restored Importance.

Please feel free to reassign if this hang problem isn't your area.

[Libreoffice-0 (libreoffice-0)]

It's not a hang. It's just taking a long time to open. There is a difference. And to fix this we either remove this filter entirely or write one from scratch. So changing the severity won't change the outcome.

[Libreoffice-0 (libreoffice-0)]

Nobody works in this area, to be brutally honest.

[Maxremov (maxremov)]

Just comment: Gnumeric opens this file normally but gives error message at first: "'Page 1'!A6781 : Invalid attribute 'Height', expected number, received '13.6875'"

[Libreoffice-0 (libreoffice-0)]

(In reply to comment #11)
> Just comment: Gnumeric opens this file normally but gives error message at
> first: "'Page 1'!A6781 : Invalid attribute 'Height', expected number, received
> '13.6875'"

Then that would serve as your workaround for now; open it in gnumeric and save as the real xls file to open it in LibO.

The gnumeric folks are at least smart enough to avoid XSLT to write their import filter. Their filter is written in pure C, which is the way it should be.

[Pjotr (pjotr)]

At first glance I see two problems processing the document:

create-header-footer-style makes assumptions about the content of @x:Data in PageSetup/Footer that don't hold for the document at hand (content of @x:Data should be "&[0-9]&[A-Z]", but here its "&Ц&С".

then, processing the rows in the document does a recursion for each row, redundantly evaluating some rather expensive expressions each time. depending on the maximum allowed recursion depth of the processor, it either quits with an error at some point or just takes *really* long on a sufficiently large document.

[Libreoffice-0 (libreoffice-0)]

Reassigning to Peter as we discussed on the mailing list. I'll be in CC.

[Pjotr (pjotr)]

The current xslt based import is not designed to open spreadsheets larger than a couple of hundret rows or columns. It's algorithm recurses into the same template for each row and column in the spreadsheet. It is not only incredibly slow but also bails out after reaching the maximum recursion depth of the xslt processor used or the maximum stack size, whichever is smaller (usually after some 2000 or 3000 iteration steps).

Unfortunately, avoiding that recursion is not trivial, as calculating the current row/column index according to span rules and explicit row/column indices requires to modify a variable outside the iteration scope while looping over the rows / columns, which is not possible in xslt (at least not in xslt 1.0).

To work around this limitation in xslt, it should be possible to write an extension function to simply store integers, thus avoiding the recursion. This, and possibly another extension function to do unit conversions, should speed up conversion considerably.

So much for now.

[Pjotr (pjotr)]

I'd suggest setting the importance to normal, because the export actually returns after a couple of minutes, so LibO doesn't hang indefinetely, causing data loss.

[Libreoffice-z (libreoffice-z)]

I can live with severity "normal".
But I doubt that a normal users without knowledge concerning all this discussion here will be patient enough to wait 1/4 hour whether LibO will start to respond again. At the latest after 14 minutes he will lose hope that LibO will start responding again and kill LibO with task manager - dataloss!

[Pjotr (pjotr)]

I think it should be possible to use the InteractionHandler passed to the import/export filter to ask a "continue/abort" question whenever the import/export exceeds a predefined timeout, and I'm going to try to add that to the libxslt based xslt filter (because I know there's a thread started there doing the actual transformation, which I could watch and interrupt if necessary). Actually I think this could and should be done centrally, applying to all IO or at least to loading and saving any kind of document, but that's currently beyond my capabilities.

That wouldn't improve importing the file attached to this bug, but would generelly prevent data-loss caused by broken filters.

[Libreoffice-z (libreoffice-z)]

@Peter Jentsch:
It would be great to have that "emergency brake" in LibO

[Pjotr (pjotr)]

A (poor) fix for this is available on to master: it introduces a timeout of 60 seconds when importing files. When the timeout elapsed, a general IO error is announced along with the choice to cancel or continue.

User-experience wise this can be improved considerably, but at least it fixes the "opening bogus file apparently crashes LibO" part of this bug (for XSLT based filters, at least).

@kohei: could you please have a look at the patch: it's my first commit, and altough I tested the fix/workaround with the file attached to this bug on a not too old state of master, I feel like it might do something stupid that I don't recognize as such.

Fix committed as
http://cgit.freedesktop.org/libreoffice/filters/commit/?id=2e9f9d82110342601d28408ae77d63b673993ebe

I'm marking this bug as resolved and would ask Max or Rainer to verify that the situation has improved and to file a new bug for the remaining part (LibO is still unable to import the attached file).

[Libreoffice-0 (libreoffice-0)]

(In reply to comment #20)

> @kohei: could you please have a look at the patch: it's my first commit, and
> altough I tested the fix/workaround with the file attached to this bug on a not
> too old state of master, I feel like it might do something stupid that I don't
> recognize as such.

Sorry. I have zero experience in this area. Could you send a note about your commit to the mailing list? Someone else may have more experience in this area.

[Sumuthu (sumuthu)]

I guess throwing a dialog box every 60 sec is annoying...Can we instead, throw the dialog only for the first timeout and wait indefinitely when user chooses 'continue'? Or a way to disable the timer - e.g. say the user ticks 'huge file' from the file open dialog? Or better could be find the filesize before actually reading the file and adjust the timeout accordingly?

[Sumuthu (sumuthu)]

Also, I don't know if this makes sense, but, would it be possible to move this to one layer above so that it works for any type of file-open? Assuming we are able to dynamically set the timeout and avoid multiple dialogs/timeouts?

[Pjotr (pjotr)]

(In reply to comment #22)
> I guess throwing a dialog box every 60 sec is annoying...Can we instead, throw
> the dialog only for the first timeout and wait indefinitely when user chooses
> 'continue'? Or a way to disable the timer - e.g. say the user ticks 'huge file'
> from the file open dialog? Or better could be find the filesize before actually
> reading the file and adjust the timeout accordingly?

It'll be rather easy to display the warning only the first time the timeout elapses. So you'd be able to tell LibO to try as long as it takes to open a file.

I would suggest not to invent any heuristics about when to show and when not to show the timeout: after all, it depends very much on the filter implementation which files will take how long to load. The file attached to this bug ain't particulary large with regards to file size, it's not even a very large spreadsheet.

[Pjotr (pjotr)]

(In reply to comment #23)
> Also, I don't know if this makes sense, but, would it be possible to move this
> to one layer above so that it works for any type of file-open? Assuming we are
> able to dynamically set the timeout and avoid multiple dialogs/timeouts?

That makes a lot of sense to me:
I'd love the timeout to not show an unspecific error message, but work and look very much like the dialog that opens in FF whenever a script on a page takes to long to execute, like "this file takes unexpectedly long to load, do you want to continue or stop loading the file". And it would be great if the warning dialog doesn't block the ongoing load-process, but simple disappear if the load finished with out the user reacting to the notification.

Unfortunately, that's not a trivial thing to do and currently beyond my skills.

[Björn Michaelsen (bjoern-michaelsen)]

[This is an automated message.]
There are no new official OpenOffice.org releases in Ubuntu packaging anymore => Won't Fix

If the problem persists, please mark this bug as "also affects project Libreoffice" or "also affects distribution Libreoffice (Ubuntu)" if that has not happened already.

Please leave references to upstream OpenOffice.org bugs in place to allow cross pollination.

[Björn Michaelsen (bjoern-michaelsen)]

Fix commited upstream, too big for backporting, will be released when upstream does:
http://cgit.freedesktop.org/libreoffice/filters/commit/?id=2e9f9d82110342601d28408ae77d63b673993ebe

[Björn Michaelsen (bjoern-michaelsen)]

From reading the comment, this doesnt exactly sound like RESOLVED FIXED, or am I getting this wrong? Locks more like a RESOLVED WONTFIX/WORKSFORME to me, or do I miss something?

[Libreoffice-z (libreoffice-z)]

MinGW Master "LibO-dev 3.5.0 – WIN7 Home Premium (64bit) English UI [(Build ID: 2ba5d12-e8c71c5-41e7bcd-4b83b90)] (daily/MinGW_cross-compilation2011-10-25_00.12.09)"
shows an "General Input/Output Error" 90 seconds after I started opening attached sample document. What ever that might mean.

@Bjoern
I agree, this one is not really fixed, The patch might be useful to prevent LibO from hanging, but it's not a real solution of the problem.

[Pjotr (pjotr)]

(In reply to comment #27)
> MinGW Master "LibO-dev 3.5.0 – WIN7 Home Premium (64bit) English UI [(Build ID:
> 2ba5d12-e8c71c5-41e7bcd-4b83b90)]
> (daily/MinGW_cross-compilation2011-10-25_00.12.09)"
> shows an "General Input/Output Error" 90 seconds after I started opening
> attached sample document. What ever that might mean.
>
> @Bjoern
> I agree, this one is not really fixed, The patch might be useful to prevent
> LibO from hanging, but it's not a real solution of the problem.

Hi all,

I tried to summarize the current status in comment #20: it's fixed with respect to hanging LibO (and thus potentially losing changes), but not regarding the broken Excel2003ML import filter. I'd still suggest to file a new bug for the broken excel 2003 xml import. As an alternative, we could change the subject to remove the dreaded HANG keyword (because it no longer hangs :-)

I stopped working on the issue both because of limited ability and uncertainty about the gravity of the remaining problem: Excel2003 XML should be quickly diminishing in use.

Really fixing the problem would at least require to develop and extension function for libxslt or saxon that allows to calculate the cell position in the sheet, whatever the remaining shortcomings of that particular import filter are.

What about moving the filter out to an extension? We'd loose word2003/excel2003 impex in the core, but the support is really limited currently and it'd be easier to explain the limitations to sb who deliberately installs the filter extension?

[Björn Michaelsen (bjoern-michaelsen)]

Fix released with 3.5.0 in precise. It prevents crash the crash, but there still import artifacts. Please open a new bug for those if needed.

[Julian Alarcon (julian-alarcon)]

I cant open this kind of files...

Just try this, copy this and save it as an .xml file:

<?xml version="1.0" encoding="UTF-8"?>
<?mso-application progid="Excel.Sheet"?>
<Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet">
<OfficeDocumentSettings xmlns="urn:schemas-microsoft-com:office:office">
</OfficeDocumentSettings>
<ExcelWorkbook xmlns="urn:schemas-microsoft-com:office:excel"/>
<ss:Worksheet ss:Name="Sheet1">
<Table>
<Column/>
<Row>
<Cell><Data ss:Type="String">Test1</Data></cell>
<Cell><Data ss:Type="String">Test2</Data></cell>
</Row>
</Table><x:WorksheetOptions/></ss:Worksheet>
</Workbook>

Libreoffice can't open it.

Usong Libreoffice 3.5 on Ubuntu 12.04 fully updated.

[penalvch (penalvch)]

Julian Alarcon (alarconj), please file a new bug as this one is Fix Released.

[Julian Alarcon (julian-alarcon)]

Thanks Christopher, I already reported the bug:
https://bugs.launchpad.net/bugs/951153

[OOoForum (oooforum)]

According with 1st message

*** This issue has been marked as a duplicate of issue 62683 ***