[Upstream] [ooo-build] images deleted from file after auto-save occurs

[glandux]

I have opened an existing file for the first time in Gutsy with OOo 2.3. After some modifications brought to this file, I noticed some images had disappeared, but not all images, and an error message was displayed in a frame of the size of the disappeared image indicating a read error. Then, I don't saved it to keep my file in its last correct state, and I made a copy of this file and opened it in parallel. And at this time the images reappeared in my first modified file. So, I saved it and closed. After reopening, all images which had disappeared were not present (not an empty frame, but totally removed this time).

After, I opened a new copy of my file and I insert a space and remove it in order to have "save" button enabled and my file not modified, and then I saved it. I wondered why the size increased from 155566 bytes to 159378 with exactly the same content, maybe a new version of ODF used in OOo 2.3 as this file was created with OOo 2.0 and last modified on 2.1??? I have redone the same modifications and this time there was no problem.
I noticed that when I had finished to redo the same modifications, automatic saving was triggered, that I didn't see the first time. I say that as this is may be linked to this problem, and the images in the first time disappeared after a duration of the same order, but this is only an hypothesis and I'm not sure at all of that (I will try to made some new tests).

I condider this as critical as it leads to a content loss.

[glandux (glandux)]

I confirm my first doubt, I modified save autorecovery information to 1 minute and indeed 1 minute after a modification, some images disappeared (always the same images).
I attached a capture of the empty frame with error message displayed instead of the image.

[glandux (glandux)]

I have made a search of odt files to find another file example to reproduce this bug and I found one file which produce the same problem:
http://odftoolkit.openoffice.org/odf-toolkit.odt
(BTW, among all files I have tested, some of them made writer crash when opening, I have reported it in bug #157512.)

To reproduce this:
- Tools -> Options -> Load/save ->General -> Save AutoRecovery information every -> 1 minute (in order to not to have to wait a long time)
- open the file odf-toolkit.odt (URL given above)
- modify it (insert a character for example)
- wait 1 minute until automatic saving
- go to page 5: the image has disappeared, and now there is a frame with message "Read-Error"

If the page 5 is displayed when automatic saving happens, the image is still visible, then I go to another page and go back to refresh display and I see the empty frame instead of the image.

If I modify the file and save it immediatly and then modify it again waiting for the automatic saving, 1 minute later when this happens, then the problem doesn't appear.

[glandux (glandux)]

I confirm myself as I tried on another machine running Ubuntu and OOo 2.3

[Chris Cheney (ccheney)]

I can reproduce this on Ubuntu's openoffice.org 1:2.4.0~rc2-1ubuntu3 but not on upstream's openoffice.org 2.4.0~rc2.

[Chris Cheney (ccheney)]

Confirmed on 1:2.4.1-1ubuntu1.

[Chris Cheney (ccheney)]

If you open the file listed below and make a change to the document while having auto-save turned on the image on page 4 will disappear and be replaced with 'read-error' once the auto-save occurs. This seems to be an ooo-build issue. It might be related to the other issue I reported that showed read-error but I am not certain.

Example file that shows this issue:

http://odftoolkit.openoffice.org/odf-toolkit.odt

[Rodo (rodo)]

I don't see image on page 4 in this document. Everything looks fine after autosave thought => assuming fixed.

[Kyu (kyu)]

There is really an image on Page 4....
But I can not find any problem after autosave => Clsoed

[Chris Cheney (ccheney)]

Created an attachment (id=265880)
screenshot of read error on 3.0.1~rc1

It still happens for me.

[Chris Cheney (ccheney)]

Reopen bug...

[indium (indium)]

same for me. Only the latest added picture remains in the file, all others are lost. No error messages whatsoever

[Bryan Quigley (bryanquigley)]

I think the importance of this bug should be bumped, as:
Autorecovery is on by default
Using images is quite common
User will LOSE data if they just save the file

[Bryan Quigley (bryanquigley)]

Attaching file that exhibits the issue. I have manually remove over 22 graphics from the file (removing from content.xml, manifest.xml and pictures folder when unzipped). I had to do this manually due to the issue that when I resave the file, it no longer will exhibit this behavior.

I have also tested this on the Windows version of go-oo and the bug does not appear to be platform specific.

Steps to reproduce:
Set autorecovery on and to 1 minute.
Open for-upload.odt (It must NOT be opened read only, and you cannot save the file with OpenOffice)
Add some text to the first page
Read a couple blog posts in Firefox (or just wait)
Go back to the file and enjoy the bug.

You may also have to scroll up/down a bit to get it to work.

[Bryan Quigley (bryanquigley)]

Created an attachment (id=362821)
File that will show read only

[Bryan Quigley (bryanquigley)]

I meant to say "File that will show read error"

[Bryan Quigley (bryanquigley)]

Created attachment 42354
Read_Error_file

Attaching file that exhibits the issue. I have manually remove over 22
graphics from the file (removing from content.xml, manifest.xml and pictures
folder when unzipped). I had to do this manually due to the issue that when I
resave the file, it no longer will exhibit this behavior. This behavior occurred originally without any manual editing of odt files, but is much more difficult to reproduce.

This is a long-standing regression compared to OpenOffice.org

Steps to reproduce:
Download the file to the desktop (DO NOT OPEN DIRECTLY)
Open the file
Set autorecovery on and to 1 minute.
Change/Add text on the first page (delete 09)
Wait at least one minute, maybe two
Note the Read-Error Message

For the data loss part, simply save the file, then reopen.

This bug has been reported elsewhere:
https://bugs.launchpad.net/openoffice/+bug/157249
https://bugzilla.novell.com/show_bug.cgi?id=440795

[Libreoffice-z (libreoffice-z)]

I see the effect with "LibreOffice 3.3.0 RC4 - WIN7 Home Premium (64bit) German UI [OOO330m19 (build 6 / tag 3.3.0.4)]". The question is what we can do, currently it seems to be a damaged document causing some problems. The effect seems to disappear when I copy / paste all contents to a new WRITER document, but I also was not able to reproduce the problem with the original document in further tests.

@Bryan Quigley:
Please contribute information concerning your Platform and OS!
Any chance to reproduce

[Bryan Quigley (bryanquigley)]

I believe the file was originally created on Ubuntu 9.04 or 9.10 and the corresponding version of OpenOffice, but I am not entirely sure.

[Bryan Quigley (bryanquigley)]

Once you save the file, the issue will no longer reappear reliably. The method I used to make this file, was the best I could do at making it easy to reproduce the effect.

[Y9a7s7tj-steve-si9yrl01 (y9a7s7tj-steve-si9yrl01)]

Hi.
I had this in OO a year or so ago, think I filed a bug, can't remember.
I have a feeling it related to EPS images in my case and then images just kept disappearing. What was happening was that the linking was damaged.
I.e after the the save when the image was read-error I could go into the odt file and the image was still there, what had broken was the reference to the image.

[penalvch (penalvch)]

Attachment pulled from FreeDesktop upstream bug.

[penalvch (penalvch)]

glandux, thank you for reporting this bug and helping make Ubuntu better. This issue was unreproducible using Ubuntu 11.04, LibreOffice Writer via the Terminal:

cd ~/Desktop && wget https://bugs.launchpad.net/ubuntu/+source/libreoffice/+bug/157249/+attachment/2119738/+files/bug-440795_for-upload.odt && lowriter -nologo bug-440795_for-upload.odt

on Page 1 type 09 -> click Tools -> Options... -> Load/Save -> General -> change Save AutoRecovery information every combo box to 1 -> OK button

notice the AutoRecovery save progress bar occurs after 1 minute at bottom, no errors.

Does this work for you?

lsb_release -rd
Description: Ubuntu 11.04
Release: 11.04

apt-cache policy libreoffice-writer
libreoffice-writer:
  Installed: 1:3.3.2-1ubuntu5
  Candidate: 1:3.3.2-1ubuntu5
  Version table:
 *** 1:3.3.2-1ubuntu5 0
        500 http://us.archive.ubuntu.com/ubuntu/ natty-proposed/main i386
Packages
        100 /var/lib/dpkg/status
     1:3.3.2-1ubuntu4 0
        500 http://us.archive.ubuntu.com/ubuntu/ natty/main i386 Packages

[Frédéric Buclin (lpsolit-deactivatedaccount)]

I can reproduce with LibO 3.4rc2. Why is the severity "normal" only? Dataloss is pretty critical.

[Libreoffice-z (libreoffice-z)]

(In reply to comment #5)
> I can reproduce

With what documents?

[Tlillqvist-k (tlillqvist-k)]

> Why is the severity "normal" only?

Because the severity field is fairly meaningless here, as anybody can change it.

[Frédéric Buclin (lpsolit-deactivatedaccount)]

(In reply to comment #6)
> With what documents?

With the document attached to this bug, as well as some of my own documents used at work.

[achristoffersen (achristoffersen)]

I have this also - and have had it with several versions of OOo / LO.

I can't say exactly how it happens, but I did most of my work on a linux box, exchanging documents with Windows users I begged and finnally convinced to download LO. Yey very often when email a version back and forth - this would happen.

Currently I am on a mac with LibreOffice 3.4.1 (Build:103) and it still happens. My collaborators does not complain about it, so I wonder if it is a *nix thing.

I really hope this will get some attention. Please tell me how I can help (technical bugfixing skillz = zero)

[Björn Michaelsen (bjoern-michaelsen)]

[This is an automated message.]
There are no new official OpenOffice.org releases in Ubuntu packaging anymore => Won't Fix

If the problem persists, please mark this bug as "also affects project Libreoffice" or "also affects distribution Libreoffice (Ubuntu)" if that has not happened already.

Please leave references to upstream OpenOffice.org bugs in place to allow cross pollination.

[Eugeneb (eugeneb)]

I am having the same problem with Read Error replacing my graphics in LO Writer 3.4.4 (build 402) since switching to LO. I am using native ODT format and occurs with several of my ODT docs. The images are actually lost from the data, 8 MB file becomes 1 MB file.

Documents were originally created in OO. We did not have the problem until moving to LO 3.3. I had hoped that 3.4.4 would resolve this significant issue.

Initially we thought this was related to errors caused by MAC users. We did some testing and opened the file under Mac LO 3.4.4, saved, closed, then opened in the same version with Windows 7 LO. The images were present, then at some point they were replaced with Read Error. Mac user does not have the same problem. This error was not seen when alternately opening OO files between Mac and Windows.

Error occurs whether files are on my local Windows 7 machine or on the network drive.

My testing shows that corruption occurs during autosave. Manually saving the file first does not eliminate the error. I am able to reproduce the problem easily.

Eugene

[Eugeneb (eugeneb)]

We spent a few hours going back and forth between Windows and Mac, opening vulnerable LO files in Mac and Windows, pasting the content into new files, saving, using network and local file copies, and editing with autosave enabled and disabled. The Read Error for the images is associated with autosave and occurs under Windows and Mac regardless of the other variables.

All of the images are png. We did find that a small png button that we placed during this testing was not lost, only the larger images.

We edited a file in Lotus Symphony and brought it back into LO in Windows and the Read Error still occurred with Autosave. Older files from OO also experienced the Read Error with autosave.

We have used OO for years to create user manuals, which contain many images and extensive use of styles. In addition to the image read error bug, we also found that when an image is inserted, the document view jumps to the bottom of the document. This is an undesirable characteristic that was introduced with LO, and not present in OO.

IBM Lotus Symphony opens the ODT docs without error, but under Mac it only opens the files on our server as read-only. Symphony was stable; autosave worked correctly, and image insertion did not change the document view. However, Symphony does not seem to have the math component which we used for formula objects, although it did allow editing of existing math objects. Nevertheless, it is a very good replacement candidate for those looking to avoid the mentioned LO bugs.

Corel does a poor job of opening ODT files, the formatting was completely lost.

For now, we are reverting to OO 3.3.0 until compelled to make a change or these LO bugs are fixed.

Eugene

[Cno (cno)]

Never seen this problem with whatever old files I use.

So to be able to track down to the source of this partictular issue (some damaged file, as suggested in an earlier comment?), having a simple test document would be useful.

[Daniel Tamm (daniel-tamm-tamm)]

We experience the same problem in a mixed network environment. It seems that the error appears mostly (maybe only) when our Mac users edit the documents. We are using LO 3.3.2 on the Mac which connects by SMB to a Linux server. The documents are in ODT format and created with LO.

This bug is very critical for us since it leads to data loss. However, it is difficult to give a file that reproduces the problem since you never know which image will disappear.
Saving the document after getting the "Read Error" will lead to reduced file size as reported previously, corresponding to the amount of data in the lost image.

[Libreoffice-z (libreoffice-z)]

I also observed picture loss with an own document and Parallel Dev-Installation of "LibreOffice 3.5.0 Beta1 - WIN7 Home Premium (64bit) German UI [Build-ID: 7362ca8-b5a8e65-af86909-d471f98-61464c4] Windows_Release_Configuration 11-Dec-2011 06:51". Not 100% sure that autosave is the reason.
Pictures will not reappear when I close and reopen the document.
I will try to get a sample document from the confidential test document.

"Bug 35631 - FILESAVE Images fail to show and are removed after autosave" might be a DUP

[Y9a7s7tj-steve-si9yrl01 (y9a7s7tj-steve-si9yrl01)]

*** Bug 35631 has been marked as a duplicate of this bug. ***

[Y9a7s7tj-steve-si9yrl01 (y9a7s7tj-steve-si9yrl01)]

Since disabling autosave and backup early this year I have not experienced this problem.

[Libreoffice-z (libreoffice-z)]

I sometimes see the effect, but until now I can't reproduce it in a reliable way. Simply opening a document and waiting (with 1 minute autosave interval) does not show the problem.

[Björn Michaelsen (bjoern-michaelsen)]

[This is an automated message.]
This bug was filed before the changes to Bugzilla on 2011-10-16. Thus it
started right out as NEW without ever being explicitly confirmed. The bug is
changed to state NEEDINFO for this reason. To move this bug from NEEDINFO back
to NEW please check if the bug still persists with the 3.5.0 beta1 or beta2 prereleases.
Details on how to test the 3.5.0 beta1 can be found at:
http://wiki.documentfoundation.org/QA/BugHunting_Session_3.5.0.-1

more detail on this bulk operation: http://nabble.documentfoundation.org/RFC-Operation-Spamzilla-tp3607474p3607474.html

[Libreoffice-z (libreoffice-z)]

Related to "Bug 32461 - Graphics disappear from document"

Inappropriate NEEDINFO reverted to NEW

[Gzizza (gzizza)]

I can confirm this bug too for both Windows and OSX version and it's also a quite annoing one.

[Cno (cno)]

*** Bug 45167 has been marked as a duplicate of this bug. ***

[Cno (cno)]

I hope that the description in Bug 45167 is helpful to reproduce the problem.

[Cno (cno)]

@ someone:
pls try the description here:
  https://bugs.freedesktop.org/show_bug.cgi?id=45167#c0

I tried again, and after some various work and returning to the document, the images are los: read-error!

[blackjam (blackjam)]

We are experiencing this bug very severely on OS X Lion and 3.5 version. All our templates are unusable because we are losing all logos and graphic elements put into them. This happens both on local files and files on SMB shares.

[Libreoffice-z (libreoffice-z)]

@Jakub Cerny
Are you able to create a test kit that reliably reproduces the bug with definitively proper documents?

@Michael:
May be you can try to get some progress here? The bug is not easy to reproduce, but for those who suffer from it it's worse than hell.
Please feel free to reassign (or reset Assignee to default) if it’s not your area or if provided information is not sufficient. Please set Status to ASSIGNED if you accept this Bug.

[Cno (cno)]

(In reply to comment #25)

> Are you able to create a test kit that reliably reproduces the bug with definitively proper documents?

here (striked because duplicate, not because it's solved.)
  https://bugs.freedesktop.org/show_bug.cgi?id=45167#c0
Also see comment #23 in this issue.

[Ato2gx51-andrew (ato2gx51-andrew)]

Trying to update documentation while recording changes on 64-bit Fedora 16 using LO 3.5 release version. Some of the graphics showed the "read error".

when I checked the new file, 12 of the 19 contained PNGs are simply gone. I mean they are missing from the internal picture directory.... and not too surprising, they are also not in the manifest. I looked at content.xml and found the frame should contain one of the missing images and it was simply not there.... completely gone. I will see if I can reproduce. On the plus side, I was tracking changes, so you can see everything that I did right up until the graphics went away!

[Ato2gx51-andrew (ato2gx51-andrew)]

Created attachment 57180
File at the start of editing.

I started editing this file, enabled change tracking..... And then started making changes.

[Ato2gx51-andrew (ato2gx51-andrew)]

Created attachment 57181
File saved the moment the bug appeared.

[Y9a7s7tj-steve-si9yrl01 (y9a7s7tj-steve-si9yrl01)]

I don't know if it is any help, but way back when I had this problem in OOO I looked into the files after the read error. First I noticed that the image file names in the zip file had changed, then a few saves later they disappeared from the zip file. It was as if the image file names got changed breaking the internal linking. I am pretty much over that bug now that I turn all auto-save and backup off.

[Ato2gx51-andrew (ato2gx51-andrew)]

Using Bryan's document, I am able to reproduce at will. Now if only I knew the code well enough to find the problem. I asked for help on the dev mailing list for some pointers as to where to look.

[Michael Meeks (michael-meeks)]

I reproduced the issue here - nasty. Reading the utterly badly designed GraphicManager for inspiration ... :-)

[Michael Meeks (michael-meeks)]

XML delta is:
- <draw:image xlink:href="Pictures/200004AD0000475F000033B381B9C98F.svm" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
+ <draw:image/>

[Michael Meeks (michael-meeks)]

with a dbgutil build I get this on autosave:

warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden

Breakpoint 1, SwGrfNode::_GetStreamForEmbedGrf (this=0x9042af0, _refPics=..., _aStrmName="200004AD0000475F000033B367F3281F.svm")
    at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810
810 OSL_FAIL( "<SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!" );
(gdb) l
805 uno::Reference < io::XStream > refStrm = _refPics->openStreamElement( _aStrmName, embed::ElementModes::READ );
806 pStrm = utl::UcbStreamHelper::CreateStream( refStrm );
807 }
808 else
809 {
810 OSL_FAIL( "<SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!" );
811 }
812 }
813
814 return pStrm;
(gdb) p _aStrmName
$1 = "200004AD0000475F000033B367F3281F.svm"

#0 SwGrfNode::_GetStreamForEmbedGrf (this=0x9042af0, _refPics=..., _aStrmName="200004AD0000475F000033B367F3281F.svm")
    at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810
#1 0xad85ae45 in SwGrfNode::SwapIn (this=0x9042af0, bWaitForData=0 '\000') at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:452
#2 0xad72a7f2 in SwNoTxtFrm::PaintPicture (this=0x90525b8, pOut=0x8b18c0c, rGrfArea=...)
    at /data/opt/libreoffice/master/sw/source/core/doc/notxtfrm.cxx:869
#3 0xad728ccc in SwNoTxtFrm::Paint (this=0x90525b8, rRect=...) at /data/opt/libreoffice/master/sw/source/core/doc/notxtfrm.cxx:319
#4 0xad8deee1 in SwLayoutFrm::Paint (this=0x9052448, rRect=...) at /data/opt/libreoffice/master/sw/source/core/layout/paintfrm.cxx:3255
#5 0xad8e154e in SwFlyFrm::Paint (this=0x9052448, rRect=...) at /data/opt/libreoffice/master/sw/source/core/layout/paintfrm.cxx:3928
#6 0xad7bd326 in SwVirtFlyDrawObj::wrap_DoPaintObject (this=0x9052520) at /data/opt/libreoffice/master/sw/source/core/draw/dflyobj.cxx:533
#7 0xad7bc869 in drawinglayer::primitive2d::SwVirtFlyDrawObjPrimitive::get2DDecomposition (this=0x9400198, rViewInformation=...)
    at /data/opt/libreoffice/master/sw/source/core/draw/dflyobj.cxx:274
#8 0xb5c77067 in drawinglayer::processor2d::VclPixelProcessor2D::processBasePrimitive2D(drawinglayer::primitive2d::BasePrimitive2D const&) ()
   from /data/opt/OOInstall/program/libmergedlo.so
#9 0xb5c64d8c in drawinglayer::processor2d::BaseProcessor2D::process(com::sun::star::uno::Sequence<com::sun::star::uno::Reference<com::sun::star::graphic::XPrimitive2D> > const&) () from /data/opt/OOInstall/program/libmergedlo.so
#10 0xb6654af5 in sdr::contact::ObjectContactOfPageView::DoProcessDisplay(sdr::contact::DisplayInfo&) ()
   from /data/opt...

[Michael Meeks (michael-meeks)]

adding:

--- a/sw/source/core/graphic/ndgrf.cxx
+++ b/sw/source/core/graphic/ndgrf.cxx
@@ -798,6 +798,13 @@ SvStream* SwGrfNode::_GetStreamForEmbedGrf(
             }
         }

+ fprintf( stderr, "look for '%s' %d\n",
+ rtl::OUStringToOString( _aStrmName, RTL_TEXTENCODING_UTF8 ).getStr(),
+ _refPics->hasByName( _aStrmName ) );
+
+ fprintf( stderr, "look for [200004AD0000475F000033B381B9C98F.svm] %d\n",
+ _refPics->hasByName( rtl::OUString::createFromAscii("200004AD0000475F000033B381B9C98F.svm" ) ) );
+
         // assure that graphic file exist in the storage.
         if ( _refPics->hasByName( _aStrmName ) &&
              _refPics->isStreamElement( _aStrmName ) )

Shows that it is indeed the 2nd autosave that fails, and/or the first autosave that busts things:

look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
...
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
...
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200006B1000048A900003B26E43FDA1F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200006B1000048A900003B26AFABBAD6.svm' 0
look for [200004AD0000475F000033B381B9C98F.svm] 1
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:817: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden
look for '200004AD0000475F000033B367F3281F.svm' 0
look for [200004AD0000475F000033B381B9C98F.svm] 1
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:817: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden

And the cockup is all related to the last checksum field of the ID: fun ! :-)

[Michael Meeks (michael-meeks)]

Interestingly, if I load and immediately save I get a document with the images included, -but- this:

look for '200004AD0000475F000033B367F3281F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 0
look for '200004AD0000475F000033B367F3281F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 0

So - the new checksums in this case are being found; a quick check in the .zip file shows the svms have been renamed Pictures/ in the .odt:

before:
200004AD0000475F000033B381B9C98F.svm 200006B1000048A900003B26E43FDA1F.svm
after:
200004AD0000475F000033B367F3281F.svm 200006B1000048A900003B26AFABBAD6.svm

So - I guess, the act of saving is updating these URLs to point to new names that are allocated for the same svm in the backup file, but not renaming the local files in the document that shares these GraphicObjects.

I guess this is the ultimate joy from the $#%#$%ing decision to use random string names for all of this lot - which (in turn) I would argue comes from the decision to use UNO - which makes string / id passing 'obvious' ;-)

Wunderbar !

[Michael Meeks (michael-meeks)]

vcl/source/gdi/gdimtf.cxx (GDIMetaFile::GetChecksum) was altered:

commit d1da85e8f5d3b6a0c5d1fdcc4fedb0eff5e90b65
Author: ka <email address hidden>
Date: Fri Feb 4 14:49:25 2011 +0100

    ka102: SVG import implementation

But - *surely* we can't be fragile on this algorithm changing; it should be fine to have any name for a file in Pictures/ surely !?

[Michael Meeks (michael-meeks)]

The difference between the serialized versions before and after appears to be an upgrade of the meta-file format:

before:
00000000 56 43 4c 4d 54 46 01 00 31 00 00 00 01 00 00 00 |VCLMTF..1.......|
00000010 01 00 1b 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 |................|
00000030 00 5f 47 00 00 b3 33 00 00 ad 04 00 00 84 00 01 |._G...3.........|
after:
00000000 56 43 4c 4d 54 46 02 00 32 00 00 00 01 00 00 00 |VCLMTF..2.......|
00000010 01 00 1b 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00 |................|
00000030 00 5f 47 00 00 b3 33 00 00 ad 04 00 00 01 84 00 |._G...3.........|

I assume we upgrade it on the auto-save. The files are rather different after the first four lines; diff -u shows little similarity after then.

[Michael Meeks (michael-meeks)]

With:

--- a/svx/source/xml/xmlgrhlp.cxx
+++ b/svx/source/xml/xmlgrhlp.cxx
@@ -621,6 +621,8 @@ sal_Bool SvXMLGraphicHelper::ImplWriteGraphic( const ::rtl::OUString& rPictureSt
                 }
                 else if( aGraphic.GetType() == GRAPHIC_GDIMETAFILE )
                 {
+ fprintf (stderr, "xmlgrhlp.cxx - write meta-file ! 0x%lx\n",
+ (long)aGraphic.GetChecksum() );
                     pStream->SetVersion( SOFFICE_FILEFORMAT_8 );
                     pStream->SetCompressMode( COMPRESSMODE_ZBITMAP );

@@ -643,6 +645,9 @@ sal_Bool SvXMLGraphicHelper::ImplWriteGraphic( const ::rtl::OUString& rPictureSt
                         rMtf.Write( *pStream, GDIMETAFILE_WRITE_REPLACEMENT_RENDERGRAPHIC );

                     bRet = ( pStream->GetError() == 0 );
+
+ fprintf (stderr, "xmlgrhlp.cxx - done write meta-file ! 0x%lx\n",
+ (long)aGraphic.GetChecksum() );

I get:

xmlgrhlp.cxx - write meta-file ! 0x67f3281f
xmlgrhlp.cxx - done write meta-file ! 0x67f3281f
xmlgrhlp.cxx - write meta-file ! 0xafabbad6
xmlgrhlp.cxx - done write meta-file ! 0xafabbad6

Which seems to suggest that the id change is over & done by the time we get to writing the graphic out; and/or is not immediately caused by the writeout.

[Michael Meeks (michael-meeks)]

@@ -387,6 +398,9 @@ sal_Bool SwGrfNode::ImportGraphic( SvStream& rStrm )
     const String aGraphicURL( aGrfObj.GetUserData() );
     if( !GraphicFilter::GetGraphicFilter().ImportGraphic( aGraphic, aGraphicURL, rStrm ) )
     {
+ fprintf (stderr, "Very curious set User Data of '%s' vs 0x%lx\n",
+ rtl::OUStringToOString(aGraphicURL, RTL_TEXTENCODING_UTF8).getStr(),
+ aGraphic.GetChecksum());
         aGrfObj.SetGraphic( aGraphic );
         aGrfObj.SetUserData( aGraphicURL );
         return sal_True;

Shows:

Very curious set User Data of 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B381B9C98F.svm' vs 0x67f3281f

which we would expect I guess, the URL name is set to the user data of the graphic there.

[Michael Meeks (michael-meeks)]

By now, I'm highly suspicious of:

void SwXMLTextParagraphExport::setTextEmbeddedGraphicURL(
    const Reference < XPropertySet >& rPropSet,
    OUString& rURL) const
{
    if( rURL.isEmpty() )
        return;

    SwGrfNode *pGrfNd = GetNoTxtNode( rPropSet )->GetGrfNode();
    if( !pGrfNd->IsGrfLink() )
    {
        String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
        aNewURL += String(rURL);
        pGrfNd->SetNewStreamName( aNewURL );

        // #i15411# save-as will swap all graphics in; we need to swap
        // them out again, to prevent excessive memory use
        pGrfNd->SwapOut();
    }
}

Which sets a new stream name for these images:

@@ -827,6 +848,9 @@ void SwGrfNode::_GetStreamStorageNames( String& rStrmName,
     if( !aUserData.Len() )
         return;

+ fprintf (stderr, "UserData '%s' NewStrmName '%s'\n",
+ rtl::OUStringToOString(aUserData, RTL_TEXTENCODING_UTF8).getStr(),
+ rtl::OUStringToOString(aNewStrmName, RTL_TEXTENCODING_UTF8).getStr());

shows wonders like:

UserData 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B381B9C98F.svm' NewStrmName 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B367F3281F.svm'

just before things go completely pear-shaped :-)

So - I guess, the ambiguity of what a vnd.sun.star.Package: URL points at: presumably not into the autosave package ;-) bites.

I hadn't realised that on top of the poor choice of string naming of images, we have the poor choice of ambiguous URLs.

[Michael Meeks (michael-meeks)]

the txtparae.cxx code that does the export does:

    // xlink:href
    OUString sOrigURL;
    rPropSet->getPropertyValue( sGraphicURL ) >>= sOrigURL;

This calls into /data/opt/libreoffice/master/sw/source/core/unocore/unoframe.cxx:1402ish that in a nutshell returns:
          pGrfNode->GetGrfObj().GetUniqueID()
Which is:
(gdb) p sOrigURL
$9 = "vnd.sun.star.GraphicObject:200004AD0000475F000033B367F3281F"

    OUString sURL(GetExport().AddEmbeddedGraphicObject( sOrigURL ));

This saves the embedded object into the output stream giving it a new name:

(gdb) p sURL
$11 = "Pictures/200004AD0000475F000033B367F3281F.svm"

    setTextEmbeddedGraphicURL( rPropSet, sURL );

And this then calls the 'SetNewName' on the graphic:

        String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
        aNewURL += String(rURL);
        pGrfNd->SetNewStreamName( aNewURL );

Which then causes the grief.

[Michael Meeks (michael-meeks)]

Nice to know this is a re-hash of 2005 - one 'thb' even advised on the topic back then ;-) I can't reproduce this bug with my moronic patch applied either. cf: https://issues.apache.org/ooo/show_bug.cgi?id=48434

[Michael Meeks (michael-meeks)]

patch is:

diff --git a/sw/source/filter/xml/xmltexte.cxx b/sw/source/filter/xml/xmltexte.cxx
index c62bce3..5c89944 100644
--- a/sw/source/filter/xml/xmltexte.cxx
+++ b/sw/source/filter/xml/xmltexte.cxx
@@ -221,7 +221,9 @@ void SwXMLTextParagraphExport::setTextEmbeddedGraphicURL(
     {
         String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
         aNewURL += String(rURL);
- pGrfNd->SetNewStreamName( aNewURL );
+
+// This is nonsensical.
+// pGrfNd->SetNewStreamName( aNewURL );

         // #i15411# save-as will swap all graphics in; we need to swap
         // them out again, to prevent excessive memory use

[Libreoffice-bugs (libreoffice-bugs)]

Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=0389b77a3cbea09ddbae238d7934d4c6349a8d37

fdo#33393 - tentative workaround for autosave image loss

[Michael Meeks (michael-meeks)]

Divining a sane and consistent purpose in Michael Brauer's original work here is somewhat difficult. Suffice it to say that -no-other- image save goes and tries to rename the stream name that we get the GraphicObject from.

Then again - other components don't wrap their own versions of SwapIn / SwapOut either - which looks extremely curious in ndgrf.cxx -

[Libreoffice-bugs (libreoffice-bugs)]

Michael Meeks committed a patch related to this issue.
It has been pushed to "libreoffice-3-5":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8a8eb870b8037801f81c680441d10a1ef95bfb06&g=libreoffice-3-5

fdo#33393 - tentative workaround for autosave image loss

It will be available in LibreOffice 3.5.1.

[Zeitlinie (zeitlinie)]

Today I've submitted Bug 46447
"Images disappear and corrupt Impress presentation"
to https://bugs.freedesktop.org/show_bug.cgi?id=46447

After that I was advised at <email address hidden> that this bug, although it happens in the Impress module, might actually be related to the bug here.
Any comments?

[Reisi007 (reisi007)]

Once had the problem with 3,5,1rc1 Win7 x64 SP 1

[Libreoffice-z (libreoffice-z)]

Fix is in 3.5, 3.6, there will be no 3.4.7, so I closed tis one. Please feel free to reopen if I did wrong.

[Iplaw67 (iplaw67)]

(In reply to comment #50)
> Fix is in 3.5, 3.6, there will be no 3.4.7, so I closed tis one. Please feel
> free to reopen if I did wrong.

Hi Rainer,

I still have this problem in 3.5.1 on Mac OSX. It seems like the problem is present for others too :

https://bugs.freedesktop.org/show_bug.cgi?id=47832

where it was tested on 3.5.2 RC1. Consequently, I'm re-opening.

Alex

[Colin-mercer (colin-mercer)]

This problem appears to be predicated on the auto-save combined with changing the picture. It appeared to be more noticeable when saving in/restoring from a .docx document.

Because it is related to auto-save, which by default is 15 minutes, then it will be as serious issue with long documents. it is a significant bug as the author will have little or no indication it has happened. Sending a document with missing drawings could affect the author's credibility.

[Cno (cno)]

Hi Alex,

(In reply to comment #51)

> I still have this problem in 3.5.1 on Mac OSX. It seems like the problem is
> present for others too :
>
> https://bugs.freedesktop.org/show_bug.cgi?id=47832
>
> where it was tested on 3.5.2 RC1. Consequently, I'm re-opening.

That bug says specifically 'Buttons on forms' thus is far more specific then this bug. So I doubt if one could see it as a duplicate. Maybe a related corner case?

Anyway, I'm testing 3.5.2 now with the steps I used previously to reproduce that bug ...

[Cno (cno)]

(In reply to comment #53)

> Anyway, I'm testing 3.5.2 now with the steps I used previously to reproduce
> that bug ...

I cannot reproduce this bug anymore.
So the other one must be different / special case.

[penalvch (penalvch)]

<email address hidden>, please do not toggle the status. For more on this please see: http://wiki.documentfoundation.org/BugReport_Details#Version

[Björn Michaelsen (bjoern-michaelsen)]

assume this to be fixed as per https://bugs.freedesktop.org/show_bug.cgi?id=33393#c47

[Geoff123 (gsking1)]

Hello - I just encountered this bug or something like it. Images show as read-error in LO and the xlsx file would not open in Office 2010. I'm using version 3.5.4 using Mac Lion. This is what I did:

1- started docx document in Office 2010 on Windows 7, added text, a few pictures, and table of contents.
2- opened docx in LO 3.5.4 on Mac Lion. Made various edits and added about 13 jpeg photos using Insert > Picture > From File in various places in the document. I did crop one of the images in LO and edited a few with "Edit with External Tool" (Mac Preview rotation) and all seemed okay from those.
3- re-Saved the file as docx and dropboxed it
4- re-opened the docx file at work today and got the following results.

MS Office 2010 will not open it now and says it has an error. Libreoffice 3.5.4 on Windows 7 will open it, but there are read-error notices where the pictures should be.

I also notice that the file is 4MB, which would be small considering that many pictures. They would have been a few MB each.

Also, the few images that were added originally in step 1 are still present and okay. All other images are missing with read-error.

Based on the file-size I'm assuming that the file was not saved correctly on the Mac at step 3 and the images were lost at that point. They were visible in the document upon closeing the file on the Mac at step 3.

I'm not sure if auto-save is turned on the Mac or not as I'm not at that computer right now, but assume it would be on as default. I assume dropbox did not mangle the file since the other original pictures added at step 1 are okay and the text is readable.

Please let me know what other info I can provide.

[Geoff123 (gsking1)]

Looking at the file again I wanted to note that 1 of the 13 pictures I added in step 2 was retained. I think it was the last one added and it was the one that was cropped (although it doesn't appear to be cropped now).

[Geoff123 (gsking1)]

Turns out that this is easy for me to reproduce on Windows 7 with LO3.5.4. using .docx format and autosave (Save AutoRecovery Information Every) turned on. Both docx and autosave are key aspects.

Here are my approximate steps.
1-opened LO and made sure autosave was on, and it is by default
2-changed autosave to 1 minutes (as noted in other comments) although it doesn't work at other intervals either. I figured it would be more likely to have problems if it autosaved more often
3-started new file and saved as docx
4-Added some text and formatting and a few images here and there, while waiting a few minutes between. Also clicked the save button once or twice to save the file between waits and adding images.
5-After about 10 minutes and minimal additions, I saved the file, close it, and closed LO.
6-Re-opened the file in LO and the first image I added was read-error. The other images were present, but some in slightly wrong places.

Hope this helps.

[Simon741 (simon-okko)]

Frech install of Linux Mint 13 Mate and just ran into this problem (with a docx file). So this bug has still to be considered as open (and a major problem).

[Okko7-r (okko7-r)]

Fresh install of Linux Mint 13 Mate with LibreOffice 3.5.3.2 and the bug is still present !

[Barta-c (barta-c)]

@okko7

please try installing brand new LibO 3.5.5 and tell if the bug is still reproducible. It's always better to use the latest available release.

[Wugs (wugs)]

Well, I would consider the problem(s) mentioned from comment #56 on as a
separate bug, because unlike the original problem they are limited to .docx and
.xlsx (and maybe other MSO2007+) file formats -- or am I missing something?

Therefore, I suggest to open a new bug report for the new problems, to copy
comment #56 et seqq. to the new bug report, and to close this (present) bug
report again. What do other people think?

[Okko7-r (okko7-r)]

Opened a new bug for the problem appearing with .xlsx and .docx files:
https://bugs.freedesktop.org/show_bug.cgi?id=52226 and closed the present bug report.

[Eugene Yurtsev (eyurtsev)]

Images are still lost.

Editing .docx file (Track changes are turned on. I did not try to recreate it without track changes on.)
Running ubuntu 12.04
Libreoffice 3.5.5.3

I had the same problem on my debian machine.

Should I just disable auto-save in the meantime?

[penalvch (penalvch)]

Simon741, this bug tracker is not for Linux Mint. If you have a problem in it, please submit bug reports to them.

Eugene Yurtsev, could you please file a new report following https://wiki.ubuntu.com/LibreOfficeBugWrangling ?

[Stéphane Guillou (stephane-guillou)]

I seem to have the same problem on Ubuntu 12.10 with LO 3.6.2.2

I don't think I came across this before a few days ago. Now it happens frequently, when I exchange versions of an .odt document that often has tracked changes and comments, between my Linux + LO netbook and my boss' Windows + M$O laptop.

Anyone seeing this again?

[penalvch (penalvch)]

Stéphane Guillou, thank you for your comments. However, this bug report has been marked Fix Released. Hence, could you please file a new report via a terminal:
ubuntu-bug libreoffice-writer

When opening up the new report, please provide the information following https://wiki.ubuntu.com/LibreOfficeBugWrangling , and feel free to subscribe me to it.

Please note, not filing a new report will delay your problem being addressed as quickly as possible.

Thank you for your understanding.

Helpful bug reporting tips:
https://help.ubuntu.com/community/ReportingBugs

[Tbehrens-u (tbehrens-u)]

My responsibilities have changed, returning bug to general LibreOffice maintainer pool.

[Jweber-i (jweber-i)]

No one will work on this issue anymore.