thunderbird - when deleting mail, mail is not deleted physically

[x]

With a recent thread on ML i came again across a problem with Thunderbird
that i had myself 1 or 2 years ago.

Thunderbird uses mailbox files to store mails and an aditionally *.msf file
for meta data of that mbox-file.
Now when you delete a mail in TB it only disappears in the mail pane but is
still physically stored on harddisc in the mbox.
To really delete a mail then s.o. has to "compact" that folder.

New users (or those not interrested in technic) wont notice that and then the
profile folder can become very large.
In the specific thread the OP had mails back from 2005 still in his profile
which had grown up to 2 gig.
2 gig is approximately where we come to filesystem limitations (max size per
file)

Now to prevent that i call for commend on change the default of:
pref("mail.prompt_purge_threshhold", false);

to
pref("mail.prompt_purge_threshhold", true);

that way a user gets a dialog when the threshold (default is 100kB iirc) is
reached and gets asked to "compact" (read as really physically delete all
previous in TB deleted mails) that folder.

I have searched bugzilla.mozilla.org for bugs like that and coulnd´t find
one. Also on launchpad is no similar bug report it seems.

So i would like to know is it feasible to solve that somehow?

Reproducible: Always

Steps to Reproduce:
1. send lots of mails
2. delete them in Thunderbird
3. check size of profile directory
Actual Results:
with default settings the profile is growing and growing, even when a user
delets his mails in Thunderbird.

Expected Results:
pref("mail.prompt_purge_threshhold", true);

delete mails physicaly after the default threshold (performance)

[x (xk2c-deactivatedaccount)]

With a recent thread on ML i came again across a problem with Thunderbird
that i had myself 1 or 2 years ago.

Thunderbird uses mailbox files to store mails and an aditionally *.msf file
for meta data of that mbox-file.
Now when you delete a mail in TB it only disappears in the mail pane but is
still physically stored on harddisc in the mbox.
To really delete a mail then s.o. has to "compact" that folder.

New users (or those not interrested in technic) wont notice that and then the
profile folder can become very large.
In the specific thread the OP had mails back from 2005 still in his profile
which had grown up to 2 gig.
2 gig is approximately where we come to filesystem limitations (max size per
file)

Now to prevent that i call for commend on change the default of:
pref("mail.prompt_purge_threshhold", false);

to
pref("mail.prompt_purge_threshhold", true);

that way a user gets a dialog when the threshold (default is 100kB iirc) is
reached and gets asked to "compact" (read as really physically delete all
previous in TB deleted mails) that folder.

I have searched bugzilla.mozilla.org for bugs like that and coulnd´t find
one. Also on launchpad is no similar bug report it seems.

So i would like to know is it feasible to solve that somehow?

Reproducible: Always

Steps to Reproduce:
1. send lots of mails
2. delete them in Thunderbird
3. check size of profile directory
Actual Results:
with default settings the profile is growing and growing, even when a user
delets his mails in Thunderbird.

Expected Results:
pref("mail.prompt_purge_threshhold", true);

delete mails physicaly after the default threshold (performance)

[x (xk2c-deactivatedaccount)]

Are you sure that new profiles are not getting cleaned up automatically?

- Alexander

=========================================

preconditions:
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
$ lsb_release -a
No LSB modules are available.
Distributor ID: Kubuntu
Description: Kubuntu 7.10
Release: 7.10
Codename: gutsy

$ LANG=C apt-cache policy thunderbird
thunderbird:
  Installed: 2.0.0.8~pre071022+nobinonly-0ubuntu0.7.10
  Candidate: 2.0.0.8~pre071022+nobinonly-0ubuntu0.7.10
  Version table:
 *** 2.0.0.8~pre071022+nobinonly-0ubuntu0.7.10 0

enigmail:
  Installed: 2:0.95.0-0ubuntu4
  Candidate: 2:0.95.0-0ubuntu4
  Version table:
 *** 2:0.95.0-0ubuntu4 0

$ alias dum
alias dum='du --max-depth=1 -k | sort -n'

* using german localisation

* using POP3 with own folders for each account
  (non global incomming)

* keep copies of mails on server

* each time when opend TB, searched for new mail and clicked on the available
folders

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

$ cd ~/.mozilla-thunderbird/pg1dsk50.default/

newly created profile:
$ dum
4 ./extensions
12 ./US
76 ./Mail
3496 .

test e-mail with atachment to self:

$ dum
4 ./extensions
12 ./US
15576 ./Mail
17368 .

thunderbird restarted

mails in thunderbird deleted and closed:

$ dum
4 ./extensions
12 ./US
15576 ./Mail
18736 .

thunderbird restarted - mails not visible in thunderbird:

$ dum
4 ./extensions
12 ./US
15576 ./Mail
18736 .

repeated last step, to be sure - same size

edit > config > additional > network & disc space > click "compress folder if
it saves more than 100 KB"
(translated from german)
retrived mails, clicked on all folders (no dialog appeared), closed TB

$ dum
4 ./extensions
12 ./US
15576 ./Mail
18768 .

started TB - retrived mails, clicked on all folders (no dialog appeared),
closed TB

$ dum
4 ./extensions
12 ./US
15576 ./Mail
18768 .

$ grep purge prefs.js
user_pref("mail.prompt_purge_threshhold", true);

------------------------------
started TB - retrived mails, clicked on all folders ( no dialog appeared)

file > compact all folders, closed TB

$ dum
4 ./extensions
12 ./US
76 ./Mail
3268 .
------------------------------

[x (xk2c-deactivatedaccount)]

hmm where is the "upstream bug" button?

anyway here is the link to the mozilla bugreport:

https://bugzilla.mozilla.org/show_bug.cgi?id=406848

[SK (stephantom)]

See also: http://kb.mozillazine.org/Compacting_folders#How_to_compact_folders

[Brian Murray (brian-murray)]

I've added a link to the upstream bug watch for you. I did this by clicking on the "Also affects project" link, choosing Thunderbird in the text box, and then adding the url for the upstream bug watch.

[x (xk2c-deactivatedaccount)]

thanks a lot Brian ;)

Stephan: that's exactly what i am doing.
                 thanks for that link notwithstanding.

[x (xk2c-deactivatedaccount)]

a quote from the Link Stephan provided:

<---------------------------------------------
The difference is that most other email clients by default automatically
compact the folder when a certain amount of space is wasted, whereas this has
to be activated by the user in Thunderbird.
--------------------------------------------->

so that's what this bug is about

[x (xk2c-deactivatedaccount)]

i was pointed to:
https://bugzilla.mozilla.org/show_bug.cgi?id=205756
and
https://bugzilla.mozilla.org/show_bug.cgi?id=286888

[Eric Forgeot (eforgeot)]

I strongly agree here, I've experienced this problem many times with users who expected "thunderbird" whould just work, without having to learn about compacting manually or browsing all the options. Most of the time, when someone is calling the "geeky friend" when they have an issue about thunderbird (even if I didn't install it myself)n it's because this option wasn't turned on. Check it, and set it to around 6000 kb and it will just work better for thunderbird users.

[Borden Rhodes (dominussuus-deactivatedaccount)]

When I found this 'feature' I asked for comments on Mozilla's forums and received this encouraging discourse: http://forums.mozillazine.org/viewtopic.php?t=601991 which ultimately led to Thunderbird being the first open source program I strongly discourage my clients from using.

Don't expect much support upstream on this. They believe that any rational person would understand that "empty trash" means "remove the e-mail from view but keep on the hard drive" and "compact" means "delete forever."

[John Vivirito (gnomefreak)]

Marked as incomplete awaiting upstream comments.

[x (xk2c-deactivatedaccount)]

just for the record: James Vega has already accepted the patch from Daniel Hahler.

[x (xk2c-deactivatedaccount)]

ups sorry
wrong bugreport updated

[Dveditz (dveditz)]

Confirming rather than duping to bug 205756 since that's morphed into the
Seamonkey version.

Tagging "privacy" because we regularly hear from users horrified to find their
"deleted" private mails are still sitting on their disk (often found after
having them imported into another progam).

Tagging "dataloss" because of the max file-size and/or quota issues raised in
bug 406848 comment 0

[Dveditz (dveditz)]

*** Bug 406848 has been marked as a duplicate of this bug. ***

[Bienvenu (bienvenu)]

yes, wanted, and since Emre is going to implement auto compact on idle, it should all be good.

[Mkmelin+mozilla (mkmelin+mozilla)]

Should be accompanied by bug 437657 also.

[Andrixnet (andrixnet)]

As I'm a sysadmin and had to install hundreds of computers with TB, this option being disabled by default gave a lot of nightmares.

I strongly suggest this option be on by default at install time (all versions) and the disk space gain threshold be set higher and in MB rather then KB. I usually set it to 76MB, similar to the default cache size of Firefox. In today's world where mails with attachements routinely have pretty large size, it seems a fair value.

Also, please resolve the race condition between automatic folder compacting and getting new mail. (bug #337554)

[Mozilla-bugs-micahscomputing (mozilla-bugs-micahscomputing)]

Ubuntu bug:
https://bugs.launchpad.net/bugs/174207

[Micah Gersten (micahg)]

Moved to thunderbird package for TB2+ support. Upstream has confirmed, so I marked it as Triaged awaiting upstream change.

[cameleon (el-cameleon-1)]

As a moderator on the french forums of geckozone, I Still see some users that have corrupted inbox because they don't use "compact folder" (and do not understand what it is...). This bug is very dangerous for average user, so it should be fix!

[Rsx11m-pub (rsx11m-pub)]

*** Bug 607315 has been marked as a duplicate of this bug. ***

[cameleon (el-cameleon-1)]

(In reply to comment #4)
> yes, wanted, and since Emre is going to implement auto compact on idle, it
> should all be good.

David, can you confirm that this bug is still wanted? It is very easy to fix and would avoid a lot of user's nightmare that we see on Geckozone forums: http://www.geckozone.org/forum/viewtopic.php?f=4&t=88068

[Flodcyclo (flodcyclo)]

This bug is very dangerous for average user, so it
should be fix!

[J2m006 (j2m006)]

hello

+1
As a moderator on the french forums of geckozone, I Still see some users that
have corrupted inbox because they don't use "compact folder" (and do not
understand what it is...).
I confirm .. This bug is very dangerous for average user, and many lose their posts when the box comes with 4 gigabytes .

j2m06

[Vseerror (vseerror)]

probably better from a UI point of view is Bug 286888 - Always make compacting folders automatic, with no UI

Bryan indicated in bug 236922 comment 7 that he liked the idea of doing Bug 286888.

bienvenu, how do you feel about the choice between this bug and Bug 286888?

---

preferable to have one or both of
bug 437657 increase autocompact threshold "mail.purge_threshhold"
bug 439089 Auto compact on idle time with better threshold
before implementing this or bug 286888 (though not strictly dependent)

[Bienvenu (bienvenu)]

(In reply to comment #13)

> bienvenu, how do you feel about the choice between this bug and Bug 286888?
>
This bug is a lot less work and thus would fit into 3.3

[Rsx11m-pub (rsx11m-pub)]

*** Bug 658924 has been marked as a duplicate of this bug. ***

[Rsx11m-pub (rsx11m-pub)]

Actually, the default was changed to "true" as part of bug 437657 and is effective with Miramar 3.3a3, thus marking this as a duplicate itself.

*** This bug has been marked as a duplicate of bug 437657 ***

[falka (farkasfalka)]

I have deleted all the mails from Inbox, but profile size is 2Gb