Thunderbird truncated messages

[Uday Reddy]

Alan Wehmann reports (2012-02-09, private email):

The other thing that I've been exploring is the sharing the INBOX file
with Thunderbird. When I tried this with the previous version of VM
that I was using, it mangled an email that was reported by Thunderbird
as truncated (I had Thunderbird options set not to download from the
POP server large files in their entirety). That email had two
attached PDF files (on the POP server).

Response from Uday Reddy:

Hi Alan, it also seems to me that it is not a good idea to let Thunderbird
"truncate" messages. MIME messages need to have certain separators. If the
separators are missing (which can happen if the message is truncated in a
naive way), then it won't be a valid mbox, and VM won't be able to process
it correctly.

Alan continues: (2012-02-09)

Now what I notice is that the email of interest is present--in
truncated form--in INBOX but it is not listed in the summary buffer
when function "vm-mode" is used on INBOX.

I also notice that the summary buffer is incomplete, at the end.

I attach GIF files illustrating both the missing summary buffer line
and its incompleteness at the end.

[Uday Reddy (reddyuday)]

Dear Alan, please see if you can isolate the truncated message in the VM folder file, and attach it here.

Are you saying that VM recovers from any problems that arise from the truncated message and continues to process the remaining messages? Or, is VM getting confused?

Were there any error messages given in the *Messages* buffer?

Cheers,
Uday

[Alan Wehmann (alan-wehmann)]

The following shows what is in the *Messages* buffer when I open the Inbox file from Thunderbird that has one or more truncated messages.

File Inbox is large (10MB), really open? (y or n)
Inbox<2>: Parsing messages... done
Inbox<2>: Reading attributes... 420
Inbox<2>: Attributes data upgraded for 6 messages
Inbox<2>: Reordering messages... done
Inbox<2>: Generating summary... done
MIME part missing header/body separator line [2 times]
Inbox<2>: Decoding MIME message... done
Inbox<2>: Recreating summary... done

That message is listed in the summary buffer and I can examine it just fine in the Presentation buffer.

Upon further examination there are numerous messages in that folder file with the header
X-Mozilla-Status: 0401

I closed the folder file, used Thunderbird on it again, and reopened it again. In *Messages* were these lines:

File Inbox is large (10MB), really open? (y or n)
Inbox<2>: Parsing messages... done
Inbox<2>: Reading attributes... 416
Inbox<2>: Attributes data upgraded for 6 messages
Inbox<2>: Reordering messages... done
Inbox<2>: Generating summary... done
Mark set
Inbox<2>: Decoding MIME message... done
Inbox<2>: Decoding MIME message... done
Inbox<2>: Decoding MIME message... done
Inbox<2>: Decoding MIME message... done

At this juncture I don't see the point of attaching here any of these truncated messages. They don't seem to be causing any problem. One could consider having a VM flag for them, in the summary buffer.

[Uday Reddy (reddyuday)]

So, there is no bug here at all? What about the screen shots you posted earlier?

[Alan Wehmann (alan-wehmann)]

The problem evidenced in the screen shots attached to this bug report was caused by the mixture of line endings in the file. Where there was a mix of line endings, Emacs allowed the ^M characters to remain in the buffer for the line endings that were DOS type (due to Thunderbird adding lines with DOS line endings--from new messages--to a file that had been saved by Emacs with Unix line endings). Those extra ^M characters caused the summary buffer not to show the corresponding messages. Cleaning up the line endings and making Emacs use DOS line endings (via coding settings) when it saves the shared email folder avoids such problems. Bug 93092 is entitled "Dealing with bad line endings (CRLF problems)" & is related (https://bugs.launchpad.net/vm/+bug/930921).

[Alan Wehmann (alan-wehmann)]

In the previous comment there is a mistaken reference to bug 93092. It should have read bug 930921. I see that those numbers become hot links. I wonder if just 930921 becomes "hot" also? We'll see.