mozilla attachments are dowloaded multiple times

[hillofbeans]

Binary package hint: thunderbird

scenario. open thunderbird and a message (new or old, doesnt matter) with a big attachment, lets say a .jpg that is 2Mb. and for the sake of argument lets also say it is the first new message of your day. So TBird automatically focuses on the first message, and you can watch the progress bar as it loads the message....

some time later, the load is done. hey, the image has a funny resolution in this view, so i double-click on the message and pop up a message window. AND THEN THE MESSAGE WINDOW DOWNLOADS THE oh-so-fargin-lovely 2Mb AGAIN.

no, this is not all. No, TBird can be made to DL that sucker in all sorts of odd ways.

It is very annoying and seems so unnecessary. Are mail messages ever actually volatile? And even if they were, for what fraction of hte people would they be like that?

this is probably not a "bug" per se; but my bet is a lot of people would like to download that 2Mb just once.

b.t.w., in this scenario I have an IMAP-SSL connection to my server.

ubuntu 8.04 hardy

thunderbird:
  Installed: 2.0.0.16+nobinonly-0ubuntu0.8.04.1
  Candidate: 2.0.0.17+nobinonly-0ubuntu0.8.04.1
  Version table:
     2.0.0.17+nobinonly-0ubuntu0.8.04.1 0
        500 http://mirror.pacific.net.au hardy-security/main Packages
 *** 2.0.0.16+nobinonly-0ubuntu0.8.04.1 0
        500 http://mirror.pacific.net.au hardy-updates/main Packages
        100 /var/lib/dpkg/status
     2.0.0.12+nobinonly-0ubuntu1 0
        500 http://mirror.pacific.net.au hardy/main Packages

ProblemType: Bug
Architecture: i386
Date: Sun Sep 28 21:21:28 2008
Dependencies:

DistroRelease: Ubuntu 8.04
NonfreeKernelModules: fglrx
Package: mozilla-thunderbird None [modified: /var/lib/dpkg/info/mozilla-thunderbird.list]
PackageArchitecture: all
ProcEnviron:
 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: thunderbird
Uname: Linux 2.6.24-19-generic i686

[Conrad-catalyzethis (conrad-catalyzethis)]

Confirmed on Mac OS X as well.

[Chrisjbolton (chrisjbolton)]

Confirmed - reproducible as above
Thunderbird 1.5.0.10 (20070221) with Mercury MTS 4.01b on Win XP SP2, No extensions except Talkback.

Definitely downloads attachments as soon as the mail is accessed, and won't display anything until the whole thing has been downloaded. Doesn't make any difference whether the message is viewed in the preview pane or opened in a new window. Monitoring network traffic confirms that there is no new traffic when opening the attachments, ie, they have been downloaded already.

Very large or multiple attachments are impossible to access or detach - for example, with 8 x 1Mb jpegs it times out after the first one.

Comments on the questions from konstantinos - mailnews.js sets the defaults for all profiles on the client machine - prefs.js over-rides those defaults.

Tried setting mail.imap.mime_parts_on_demand to false (using Config Ed.), closing Thunderbird, restarting and setting back to true - no difference

[Bienvenu (bienvenu)]

are the attachments attachments that we display inline, e.g., jpeg? If you don't want us to display & download attachments inline, you need to turn off view | display attachments inline.

mime parts on demand only applies to attachments that we're not displaying inline.

[Chrisjbolton (chrisjbolton)]

Display of attachments inline is set to false, and this is how Thunderbird behaves. Attachments are not displayed inline.

I have looked at the IMAP server logs when accessing a message with an attached jpeg:

11 UID fetch 4253 (UID RFC822.SIZE BODY.PEEK[])
* 227 FETCH (UID 4253 RFC822.SIZE 155140 BODY[] {155140}

I don't understand that fully, but the empty section specification [], according to RFC2060, denotes the entire message.

The bug therefore appears to be that Thunderbird is not picking up that mime_parts_on_demand is set to true, and is asking for the whole message.

[Mcicogni (mcicogni)]

Confirm this behavior also on 2.0.0.0 (20070326) on Win XP SP2, on various independent IMAP server implementations.

A fetch of (UID RFC822.SIZE BODY.PEEK[]) is just an IMAP idiom to get everything. Asking for BODY.PEEK (instead of BODY) just tells the server not to clear the "unread" flag.

This behavior definitely comes from the bayesian antispam implementation, since turning that off changes heavily the network traffic shape.

[Bienvenu (bienvenu)]

yes, that's correct - we're trying to fetch message text to do bayesian analysis on it. You have to turn that off to prevent this.

[Mcicogni (mcicogni)]

The "real problem" here, I feel, is not that TB's trying to do its job and do bayesian classification: it's the lack of feedback on the UI, and the impossibility to turn this off easily.

I mean: usually, I'll have my laptop connected to my office LAN, or to my home ADSL, and everything's good.
Sometimes, however, I'll be at the airport, with a *very* expensive WiFi or UMTS connection, and I'd like to have something better than having to start TB offline, disable Adaptive Spam Filter, go online, etc.

Maybe something like a "vulcan nerve pinch" to stop the bayesian filter downloads in emergency situations? Something better than just killing TB outright?

[Chrisjbolton (chrisjbolton)]

Disabling the Adaptive Spam filter doesn't change the behavior on my system. Thunderbird continues to ask for and download the whole message, irrespective of message size, and irrespective of whether MPOD is true or false. I have logs (too big to post here) from both the server and Thunderbird if that is any use for debugging; the server logs also record time.

Surely the Bayesian analysis should be done on the Inbox, not stored messages?
I would not expect anything with \Seen flag to be analyzed. Also, what is the point of Bayesian analysis of a mime-converted binary attachment; there will be nothing recognizable as spam? Thunderbird should recognise that MPOD has been asked for and, where Bayesian analysis is called for, do it on the parts as demanded, excluding any non-text parts, which will only dilute the ratings.

[Deeno-privat (deeno-privat)]

finally something happening here!!!

you all seem to be very into it, i almost haven't got a clou what you guys talking about, i just need to get this up+running... its been a year now living with this bug and its fine when i'm at the office or at home (DSL) but as soon as i am on the road it gets really annoying + expensive. last month i was almost twice a week and from may on i will be almost every day... is there no way to switch-off this misbehavior?

(i have to admit i was "flirting" with outlook, but i don't want to!!!)

also i tried to switch off junkmail filtering / bayesian analysis from tools-junk mail... but this option does not exist with my TB... i am using V2

[Bienvenu (bienvenu)]

yes, that ui does exist in 2.0 - it's per account, in tools | account settings | junk settings.

You may also want to turn off displaying attachments inline, if your large attachments are getting displayed inline.

view | display attachments inline

[Mcicogni (mcicogni)]

I'd like to stress the point that there are 2 things amiss here:

1. Bayesian analysis should NOT download the whole message, just text parts (good point Chris, there'll be no information in a MIME-encoded binary attachment)

2. If we really need to download the whole message, at least we should keep it somewhere, instead of throwing it all away and downloading stuff again if one needs to save an attachment.

And still, at the very minimum, there should be some way to easily disable Bayesian filtering when on slow connections, and UI feedback that there's something going on in the background when TB's downloading stuff to do Bayesian classification.

[Chrisjbolton (chrisjbolton)]

I don't think the Bayesian analysis is the root of the problem; as I say above, turning off the Bayesian filter does not fix the problem for me.

[Mcicogni (mcicogni)]

This is very strange.

Have you tried starting from a fresh new install of a recent version of TB, using a new profile?

[Deeno-privat (deeno-privat)]

strange think happens lately... problem does not persist anymore!
i dont know why? nothing changed, same laptop, same software, it works!
please confirm if problem has been fixed in an TB update...

[Mcicogni (mcicogni)]

konstantinos: Can you provide the exact version and build ID of the TB binary you're using (i.e. the numbers displayed in Help->About)?

[Deeno-privat (deeno-privat)]

version 2.0.0.4 (20070604)

[Deeno-privat (deeno-privat)]

i just received an email from a customer with 12 jpgs attached, each around 350kb small, we are using at the office a highspeed DSL6000 and it takes ages to download the attachments!!! itsbeen 20 min now, 4 remaining it seems it is hanging now... its my 5th try now!!! it is so annoying...
(it used to happen before as well sometimes)

it is definitely not a server problem because when i go into my webmail-interface and download the whole thing from there it takes only 20 seconds!!!

...i am so disappointed and fed up with all this bugs...
...why cant it just work?
...its been over a year now and still problems,
   i dont want to switch back to bloody MS
   but if problems still keep persisting then i cant work efficiently!
   (see also https://bugzilla.mozilla.org/show_bug.cgi?id=380275)

[Deeno-privat (deeno-privat)]

now its speeding up download,
i think its caching or preloading all the attachments and then downloading them again each one... almost the same error as before? i dont know anymore...

[Mcicogni (mcicogni)]

That's exactly the same behavior I see all the time.
But... didn't you write that you were not experiencing the undesirable behavior anymore?

Konstantinos, could you please vote for the bug, so that we may get it confirmed sooner or later?

[Mcicogni (mcicogni)]

David: do you think there's any point in doing Bayesian analysis on non-textual attachments?

Shouldn't it only work on the text (plain and/or HTML) parts?

[Deeno-privat (deeno-privat)]

i just voted!

yes, it does not preload the attachments on viewing the mail by just selecting it on the pane i guess... but again, maybe it has preloaded before! or my connection is so fast that i do not see it happening...

[Grngrnflcn (grngrnflcn)]

I too can confirm this bug. I have some large pdf attachments in e-mails within an IMAP e-mail account that are doing the same thing, tried countermeasures listed here to no avail. Since one of them is 24 Mb, it is...distressing...to be unable to view the text from the e-mail without waiting a long time, and then have to wait a long time again in order to download the pdf. Never had this problem in Outlook, just changed over to Thunderbird.

Thunderbird version 2.0.0.4 (20070604)
OS: Vista
Display attachments inline: off
Default settings for IMAP
To save disk space, do not download for offline use messages larger than 50 kb
Adaptive Spam filter on or off: no effect
Extension: lightning
Always reproducible

[Hskupin (hskupin)]

This is a 1.8 branch bug only. I cannot reproduce it with a current trunk build. Switching of display inline, shows me directly the message body. Using instead a 2.0.0.5pre build on Linux stills shows me this issue, even with deselected display inline option.

[Deeno-privat (deeno-privat)]

what is a 1.8 brunch bug?

[Hskupin (hskupin)]

(In reply to comment #24)
> what is a 1.8 brunch bug?

The main development happens on trunk which will be Thunderbird 3.0 in the future. But all releases of Thunderbird 2.0.0.x are build from a separate branch which is 1.8 now. You can test with a Tb 3.0 alpha release where it should work. But create another profile therefor.

[Deeno-privat (deeno-privat)]

when can we expect a 3.0 version? (i know, stupid question, but it might be vital for not switching back to Outlook, because this bug is really annoying)

[Hskupin (hskupin)]

David, do you know when or by which patch this was fixed on trunk? Is it worth to backport it for 1.8? Or won't it get approved because its not a security bug?

Konstantinos, AFAICT the release of Thunderbird 3 will be happen after Firefox 3.

[Bienvenu (bienvenu)]

Henrik, no, I don't know what the fix was. I was going to ask you if you were sure that you tested this on the trunk and 2.0 on the same profile with the exact same conditions because I can't think of anything that happened on the trunk but not the 2.0 branch in this area.

[Sucrabu-gmail (sucrabu-gmail)]

Yay!

I'm not the only one being driven crazy by this behavior. Until I found this I was blaming our mail server, tough.

(tb version 2.0.0.6 (20070728), winxp sp2, dovecot(?) server on our LAN).

[Hskupin (hskupin)]

David, it's really suspicious. I can confirm that behavior with the latest trunk and 1.8 branch builds within my normal profile. After creating a new one the issue is solved. Do we have a pref which handles the pre-loading of a message? Or are special IMAP settings responsible for? Before I start to test I just want to ask.

[Hskupin (hskupin)]

Ok, there was a preference I set some time ago for testing purposes I think. mail.server.default.mime_parts_on_demand was set to false. But why this preference has higher priority as mail.imap.mime_parts_on_demand?

After setting this pref to false and leave the values from comment 0 all is working fine for me in Thunderbird 2.0.0.8pre and Thunderbird 3 nightly build.

[Mcicogni (mcicogni)]

Henrik, do you have Bayesian classification on or off?

[Hskupin (hskupin)]

Adaptive junk mail controls are enabled, even the checkbox for scam and anti-virus.

[Mcicogni (mcicogni)]

I'm really annoyed by this behaviour now.

I tried turning off bayesian filtering, and that helped a bit, but TB is still always downloading the entire message behind my back.

And, whenever I ask it for an attachment, it goes ahead and downloads it AGAIN!

Anyone wanting to really look into this? Please?

[Lars Mai (mailars)]

(in reply to comment #34)
Mauro, i was able to disable this behaviour by setting mail.imap.mime_parts_on_demand=true . See the comments in bug 399200, specifically comment #5 and comment #6.

[Deeno-privat (deeno-privat)]

i did that ages ago... no change at all...
this is a really annoying bug!
we just opened a new branch in athens, and on all new PCs we installed OUTLOOK2003... broken-hearted...
but, its been over a year now living with this bug and no solutions yet...
i am even considering switching to outlook on my laptop + pcs in office thessaloniki + munich... its really a shame!

[Hskupin (hskupin)]

Konstantinos, does that also appear for newly created profiles? Or can you solve it per my comment 31? Can you see this with different IMAP servers e.g. Exchange, Courier. And could you try to run a test with the latest Thunderbird 3 nightly build? Please make a backup of your profile first!

[Deeno-privat (deeno-privat)]

thank you for the reply and suggestions... as soon as i find the time (that i do not really have) i will try them! different IMAP servers is not possible. i will give nightly build 3 a chance...

[Deeno-privat (deeno-privat)]

tried NB3... hmmm not sure if this worked... troubles me a bit because it takes ages again to download a 4MB attachment (while not preloading i think, as i switched to a 16.000 DSL) and from my webmailinterface its been downloaded almost imediatelly... there is still something happening in the background, but i think it works a bit better

[Paul-popperfamily (paul-popperfamily)]

Henrik Skupin on 2007-09-27 14:38:36 PST said:
> Ok, there was a preference I set some time ago for testing purposes I think.
> mail.server.default.mime_parts_on_demand was set to false. ...

YES! I found that my mail.server.default.mime_parts_on_demand was also set to false. So I toggled it to true and problem is resolved.

So this is true resolution to problem and I think others did not pay attention to what you said.

> After setting this pref to false and leave the values from comment 0 all is
> working fine ...

I assume this is typo and that you mean to say "After setting this pref to TRUE".

[Bienvenu (bienvenu)]

It's not new behavior - it was in Netscape 4.0, for example. However, there are some clients that will give binary attachments an inline disposition (Outlook, in particular), and that tricks us into fetching the part because the sender has said to display the attachment inline.

[Tanstaafl-bh (tanstaafl-bh)]

Ok, hmmm, wonder if there is any way to detect this (binary attachments given an inline disposition)...

Well, anyway, since you guys are working to finally fix this bugger once and for all, I want to start over testing the fixes...

Does the b2 release have all of these fixes? Or should I wait for the b3 release? Or, is there a specific reasonably stable nightly you can point me to that contains finished versions of all of the fixes discussed here?

Thanks David...

[Stevendejong (stevendejong)]

In response to #118

<<there are some clients that will give binary attachments an inline disposition (Outlook, in particular), and that tricks us into fetching the part>>

I have tested this by sending myself a big attachment using TB and then opening it on another computer in TB as well. I'm using the latest stable build.

As always, the attachment needed to be fully downloaded (taking a large amount of time) before I was allowed to read the text of the message, or do anything else in the Inbox.

As has been said quite some times before, there are many email programs that do not have this behavior, even Outlook Express (a notable exception being Outlook, in which IMAP support is quite a lot worse than in TB).

It seems the cycle within this thread continues -- a rather concentrated number of messages stating that the problem persists, a number of promising responses from developers, followed by a message denying that it is a problem, or stating that it has been fixed already long ago.

To get back to an earlier message here as well: fortunately the bug is now marked as important enough to be solved for TB3. But meanwhile, the mainstream user is still using TB2. I've seen quite some people go back to M$ products as soon as the first big attachment in an IMAP account arrived. Personally I haven't seen a bug as discouraging as this one in the entire TB.

[Bienvenu (bienvenu)]

Steven, are you displaying attachments inline or not?

[Stevendejong (stevendejong)]

(In reply to comment #121)

> Steven, are you displaying attachments inline or not?

David, thanks for a quick response! Once again, let me restate that I love TB. If I didn't, I would not be participating actively here.

To be perfectly sure:
I disabled offline storage and changed the inline attachments setting (not displaying them inline). I sent myself (from a different account within TB -- note that this is a POP mail account, I don't have two IMAP accounts available on this computer at the moment) a 7.5 MB RAR archive.

Result:
The message text was displayed almost immediately after I clicked the email.

Then:
I re-enabled inline attachments.

Result:
The message text was displayed almost immediately after I clicked the email. I am not sure whether it could have been in the cache (I exited TB and came back again).

Then:
I enabled offline storage.

Result:
After restarting TB, it displayed an hourglass for quite some time, until I clicked the message with the big attachment, which got displayed promptly.

Conclusion:
If you receive an email with an attachment that was sent using TB (a GMail POP account to be precise), the attachment is *not* downloaded before you can see the text of the email. David is completely right here.

But:
In my opinion, true attachments should be fetched only on demand, independent of who sent them or whether I display attachments inline. With offline mode on, they of course need to be fetched, but only once, and in a way that allows me to read the text of the message before the download starts. I realise it may be a difficult problem, but many other clients don't seem to have it.

[Stevendejong (stevendejong)]

In addition to what I just wrote (and sorry for spamming you all): it seems we are onto something regarding this bug.

Can anyone confirm that the problem
a) Happens when you display attachments inline in TB and receive a mail with a large attachment, sent from Outlook;
b) Does NOT happen when you do not display attachments inline in TB and receive a mail with a large attachment, sent from Outlook?

If so, the actual cause of the problem has been correctly identified: TB sees a normal attachment as an inline one due to the faulty way in which the mail was constructed by the sender's mail program.

[Bienvenu (bienvenu)]

TB never marks attachments with content disposition inline when sending, I don't think, so a RAR archive sent from TB won't be fetched when you click on a message, independent of the display attachments inline setting. I think most peoples' complaint has to do with image attachments, which we do know how to display inline. Turning off view | display attachments inline *should* make it so we don't try to fetch those images when you click on a message.

The offine auto sync issue is somewhat different. Autosync tries not to download large messages (> 50K) unless you're idle. But it has an aggressive definition of idle (no keyboard activity for 10 seconds, or an other window is in front). We could try to fetch large messages for offline use using chunking, to allow the process to be interrupted by the user, but it can slow down download drastically, which is extra bad for large messages. We deliberately don't cache partial messages in the offline store, so that we don't accidentally think we have the complete message when it's just partial.

You may have also run into a bug where we go idle on startup, if startup takes too long.

[Rsx11m-pub (rsx11m-pub)]

(In reply to comment #124)
> TB never marks attachments with content disposition inline when sending, ...

Default for mail.content_disposition_type is "inline" in TB2 and "attachment" for TB3, thus attachments may be send out with either disposition depending on that setting. Basically, the reverse of bug 452092 would be needed here, having a positive list of MIME types for which inline disposition makes sense to start with and only download those (preemptively or not) if they are inline and such attachments are to be displayed with the message.

[Tanstaafl-bh (tanstaafl-bh)]

I think this question may have gotten lost:

What version of TB3 should I be testing to see the latest/current bahevior with the patches created so far?

Also - any chance of being able to define a max_size that should be automatically downloaded if 'View > Display Attachments Inline' is enabled?

This way, tiny graphics (that are in many peoples signatures), and Pictures that have been resized to be small (ie < 100K) can be displayed, but not really big ones.

[Rsx11m-pub (rsx11m-pub)]

Most recent developments are included in the nightly builds for testing:
> http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/

[Tanstaafl-bh (tanstaafl-bh)]

Thanks... I know about nightlies, been burned more than once...

Is there a known relatively stable nightly that hass all of the fixes that you can recommend? If not, I'll just wait for b3...

[Tanstaafl-bh (tanstaafl-bh)]

And I'll ask this separately again...

Any chance at all of having a max_size setting for 'View > Dislay attachments inline', so that only attachments smaller than a certain size will be downloaded?

[Rsx11m-pub (rsx11m-pub)]

(In reply to comment #128)
> Is there a known relatively stable nightly that hass all of the fixes

Thunderbird 3.0 beta 2 has all changes up to late February, which includes
bug 436615 but potentially not all follow-up patches. It won't have the disk cache enabled, for example; I don't know which relevant patches may be missing. Nightlies are "per definition" unstable, but you could use a test profile and some free IMAP account where you do not mind the (unlikely but possible) case of loosing data.

> (comment #129) max_size setting for 'View > Display attachments inline'

That's not here but filed as bug 294763.

[Rsx11m-pub (rsx11m-pub)]

*** Bug 493668 has been marked as a duplicate of this bug. ***

[Ulysse (ulysse)]

*** Bug 449366 has been marked as a duplicate of this bug. ***

[Micah Gersten (micahg)]

Marking this Triaged as we have an upstream bug.

[Ludovic-mozillamessaging (ludovic-mozillamessaging)]

So Reading this bug just confuses the hell out of me and makes it quite unusable for anything except for discussion on how Thunderbird should work. There's nothing much that can be done with this bug.

Reminder : discussions should go on mozilla.dev.apps.thunderbird or the Tb-planning mailing list.

Some of the things reported here are already covered by other bugs - If not please file the missing one, with clear goals (and one goal per bug).

I'm resolving this INVALID because of it's too chaotic nature.

[Maf-soft (maf-soft)]

Yes, its chaotic here, but there are many less chaotic bugs that were marked as a duplicate of this bug. So maybe we should reopen one of them...

I think we all agree, that thunderbird should never download a big attachment more than once within a short time. Is this bug already filed somewhere else?

[Deeno-privat (deeno-privat)]

wow! great solution after FOUR!!! years since i submitted this bug...
no other solution provided so far now...
this IS really sad and not really a solution to say "There's nothing much that can be done with this bug." and "I'm resolving this INVALID because of it's too chaotic nature." ... where is the solution???

[John Pye (jdpipe)]

This is definitely NOT INVALID and NOT RESOLVED. The bug has been confirmed by many users, has received 34 votes. At least 7 bugs have been marked as duplicates of this bugs. 44 people have subscribed to the bug.

Please before closing the bug explain clearly what we're supposed to do to prevent this constant repeat-download issue that we're all seeing.

[Maf-soft (maf-soft)]

To get it fixed, we should have a short and clear description of the issue and not this lengthy discussion thread. But if we create a new bug it should be mentioned that it is in fact already 4 years old. Or maybe we can reopen one of the older duplicates...

[Nickel-chrome (nickel-chrome)]

Much of the discussion in this bug is now out of date, however it still remains under certain conditions as it has not been directly addressed.

FYI the resolved bugs that have *partly* addressed this one are bug 439731 and bug 436615. IMO the main outstanding issue is described in bug 439731#25.

If this bug has become to difficult to follow maybe a new bug could be opened with objectives relevant to the current state of Thunderbird.

[M-wada (m-wada)]

(In reply to comment #135)
> wow! great solution after FOUR!!! years since i submitted this bug...
> no other solution provided so far now...
> ... where is the solution???

To konstantinos(bug opener), what kind of real problem of your comment #0(original problem of this bug) still remains in latest Tb 3.1 or Tb 3.2pre?
(Im' not talking about added "image/jpeg with Display Attachments Inline=Checked" case, nor added "Content-Disposition:inline for non-inline displayable part" case.)
As seen in Bug 565852, current issues around multipart-on-demand(auto-sync is not enabled for an IMAP folder) are Bug 565852 and friends of that bug. i.e. Multipart-on-demand itself works as expected.

[Romariosso (romariosso)]

The original problem is still present in TB 5.0. Just as is the one in https://bugzilla.mozilla.org/show_bug.cgi?id=439731 (reloading of read messages).

[Rsx11m-pub (rsx11m-pub)]

Not exactly the specific issue described here, but more like bug 629738 now where the cache apparently isn't keeping track of when the message parts need to be reloaded, thus there are unnecessary invalidations and recaching of content.

[danson (danielwatts)]

I can confirm this bug is still present in the latest Thunderbird 5.0 relese to date.

1. First open a new email with 6MB attachment -
2. Notice long download, no message body until complete. (either blank or previously selected message body often still visible).
3. Right click attachment - save as - long download, progress bar visible and the attachment is re-downloaded from the IMAP server.
4. Click on a different email.
5. Click back on the 6MB attachment email again.
6. Repeat 2-5

Problems / Correct Behaviour:
1. Attachments should not download before the body.
2. Large attachments (as selected) should (optionally) only download when requested.
3. Attachments should be retained locally
4. When saving a downloaded attachment the local copy should be saved ~instantly.
5. When reselecting an email with an attachment, it should use the local attachment copy.

*In summary - an attachment should in ALL CASES be downloaded ONLY ONCE.*

However the backend is implemented, this is what the user should experience. There seems no sane reason to redownload attachments ever apart from perhaps coming up against some set limit for local cache size.

3.

[Charles (tanstaafl-libertytrek)]

(In reply to comment #142)
> However the backend is implemented, this is what the user should experience.
> There seems no sane reason to redownload attachments ever apart from perhaps
> coming up against some set limit for local cache size.

And morer specifically, if a folder is set to offline mode, then the emails/bodies/attachments should be stored in the offline store, NOT the cache.

I can also confirm that TB5 still redownloads attachments over and over again in folders set to offline mode - first, when you click/select the message, then again if you choose to 'Save As' the attachment, then again if you click a different message then go back to the message with the aqttachment.

Infuriating, when you have a lot of messages with large attachments on a remote/slow link to your imap store.

[Rsx11m-pub (rsx11m-pub)]

(In reply to comment #142)
The memory cache should be large enough now after bug 629247 removed the 4MB limit, but it's still a problem with caching of IMAP parts not working properly. I've seen 0-size attachments showing up on the 5.0 branch resembling on-demand attachments, with a "busy" indication in the status bar when viewing such a message suggesting an incomplete download of the MIME part. It's not clear to me if that's part of bug 629738 or a new problem now, somewhat hard to reproduce.

(In reply to comment #143)
Attachments should only be downloaded in offline mode if they haven't been synchronized yet. Do you see this also for old messages or after clicking on "Download Now" in the Synchronization tab of the Folder Properties?

[danson (danielwatts)]

I can confirm that further to my report in Comment 142 this particular folder *IS* set to offline mode with "Select this folder for offline use" ticked.

Interesting development:

1. Created new TEST subfolder (also marked for offline use)
2. Move EmailA (1MB attachment)
3. Run above test on EmailA - no redownloading.
4. Move EmailB (6MB attachment)
5. Run above test - (try saving, click off email then back on etc) - attachment redownloaded each time.

Conclusion - is there some kind of caching/offline limit set? What doesn't make any sense is I have 'Dont download messages larger than 50KB' ticked in my 'Synchronisation and Storage' settings for this account. Presumably this is being entirely ignored as 1MB > 50KB. Unless the bug is that the code thinks 50KB is actually 5000KB...trying this now with settings set to 5KB...

Nope - still wont' work for 6MB attachment but fine for 1MB one.

Also - when I click 'go offline' all the 1MB emails (body and attachments) are available offline. The 6MB email has no content at all - "The body of this message has not been downloaded from the server for reading offline...".

For some reason a 6MB attachment is being entirely left on the server - no use of offline file space.

SECOND TEST - Created new subfolder with 'use offline' switched off. Testing will TB use cache?

Result - identical behaviour. The 1MB file is cached. The 6MB file is redownloaded each time. Offline storage vs Cached makes no difference.

Hope this is useful feedback.

[danson (danielwatts)]

rsx11m - ah a 4MB limit! Well that explains my behaviour but my findings suggest this limit is NOT removed in TB 5.0.

[Dbienvenu (dbienvenu)]

There's no limit to the offline file store cache, so if the message is in the offline file store, saving attachments should just use the offline file store.

You can tell if the message is in the offline store by going offline (file menu, offline, work offline) and then clicking on the message. If we tell you the message body has not been downloaded, then it's not in the offline store.

The disk and memory cache complicate this quite a bit. Also, simply clicking on a message with a large external attachment should not end up with it getting put in the offline store. You'd need the imap auto sync to do that, or the aforementioned download now button.

[Rsx11m-pub (rsx11m-pub)]

(In reply to comment #146)
> rsx11m - ah a 4MB limit! Well that explains my behaviour but my findings

That's the previous limit of the *memory* cache, which caused issues in the past. The disk cache has a default limit of 50MB, and as David just stated, the offline store doesn't have any pre-set limit. It only goes into effect though if and when the message and/or its attachments are actually synchronized, until then you have to rely on the disk and/or memory cache (which, as stated, is somewhat broken).

[Charles (tanstaafl-libertytrek)]

(In reply to comment #147)
> The disk and memory cache complicate this quite a bit. Also, simply clicking
> on a message with a large external attachment should not end up with it
> getting put in the offline store. You'd need the imap auto sync to do that,
> or the aforementioned download now button.

I would love a simple way to completely and utterly disable disk/memory caching, and use only offline storage.

[danson (danielwatts)]

More info:

I set up an entirely empty IMAP account in thunderbird and moved the 1MB and 6MB files to it. I set this up for offline mode and ran the 'Download Now' to force the offline sync.

THIS WORKED!

The 6MB file is now loaded instantly and available when offline. I can save the attachment to my local disk and it doesn't download off the server.

So we can break down the problem into:

1. Normal cache not working - consensus is this seems to be broken generally (separate bug?)

2. Offline files sync broken for large mailboxes - seems to work on small mailboxes. Doing a 'sync now' on a large mailbox (100,000 emails, 100 folders, selected for offline use 10 folders, 10,000 emails) doesn't seem to actually work - perhaps some timeout in effect?

3. Cache -> Offline files - why on earth does TB not put out to offline files as it downloads email? Why is Offline files sync a separate process? Surely it makes sense to update the offline files storage from Cache so there is still a single download.

Are there open bugs for the above already? If not, and if people can replicate, we should open them. I get the feeling putting all this effort into a 'RESOLVED INVALID' bug may be falling of deaf ears.

[Marinus-4 (marinus-4)]

Five (5!) years later, in Thunderbird version 5.0, it is still having this most annoying bug. Mail with attachments takes forever to open, then, when you click on the attachment to open it, apparently the download starts afresh. Thunderbird has always done this, on all 4 computers in our house, for ALL IMAP accounts.

[Stevendejong (stevendejong)]

If you think about it, this bug and the way it is dealt with by the developers is exemplary for the product. It's "major updates" are minor patches. Who notices the heralded new account wizard as an exising user? Nobody. Who has problems with inline attachments? Everybody, since version 0.01 of the product. So let's develop a new account wizard and mark the inline attachments bug as "RESOLVED INVALID".

Similarly, all the other issues and feature requests I invested time and effort in on this forum have not been seen as relevant by the developers. (E.g. the fact that entering and especially copy-pasting or cutting multiple email addresses from e.g. the To: field is so ridiculously cumbersome; the fact that the search engine introduced in TB3 is incredibly slow; the fact that properly layouting HTML emails is really a pain; the ugly and inconvenient placement of buttons in the preview pane header; the fact that only new email composition does not happen in a tab; ...)

I've started contributing bugs and feature requests around version 1.x. We're now at version 5, five years later at least, and most major opportunities for improvement have not been addressed. Thunderbird as a consequence still is an immature product, and given how the developers prioritize their coding efforts, I'm sure version 8 will give us the 5th major GUI overhaul and a spectacular New Account Wizard... and it will still reload IMAP attachments every time you select an email.

I hate to say it as a definite non-fanboy of Apple, but if the TB developers would take a short look at Apple Mail (even the one in OS X 10.6), they'd notice everything that is wrong with TB. If Apple can do it, then it must not be too difficult to write a simple yet functional email program.

[Romariosso (romariosso)]

(In reply to Joe Smith from comment #140)
> The original problem is still present in TB 5.0. Just as is the one in
> https://bugzilla.mozilla.org/show_bug.cgi?id=439731 (reloading of read
> messages).

Actually, it seems to have been rather a user error in my case. :) The problem occured when I had limited the size of e-mails to be downloaded to 100kB. That was the case ever since I started using TB almost a year ago. So that those large e-mails (and also their attachments) were downloaded each time the e-mail was opened or the attachments saved.

After disabling of the mail size limit in account options the problem seems to be gone. TB then downloaded all the previously too large e-mails. It now still shows "loading" for a short time when a large e-mail is opened, but that seems to be loading from hard disk (which goes very busy during that time) while the network connection is idle. I gueass that rather short loading time now is probably a result of the compression I have activated in TB for the e-mail files, and the subsequent need to decompress when opening an e-mail.

So, there seems to be no issue for me after all.

[danson (danielwatts)]

Joe - interesting. Though what you're doing is solving the problem by downloading ALL attachments. The bug here is that attachments should not be downloaded unless requested (and then only once!).

[Charles (tanstaafl-libertytrek)]

Why is the status for this bug set to 'RESOLVED INVALID'? This bug is still very much alive and well.

Joe - do you have 'Tools' > 'Synchronization & Storage' 'Keep messages for this account on this computer' checked/enabled? If so, then that is not what this bug is about...

[Romariosso (romariosso)]

(In reply to Charles from comment #155)
> Why is the status for this bug set to 'RESOLVED INVALID'? This bug is still
> very much alive and well.
>
> Joe - do you have 'Tools' > 'Synchronization & Storage' 'Keep messages for
> this account on this computer' checked/enabled? If so, then that is not what
> this bug is about...

Yes, I have it enabled. Sorry and thanks for the info.