Thunderbird renders system unusable / uses 100% CPU while indexing new messages
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mozilla Thunderbird |
Expired
|
Medium
|
|||
icedove (Debian) |
New
|
Unknown
|
|||
thunderbird (Fedora) |
Fix Released
|
Medium
|
|||
thunderbird (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
Binary package hint: thunderbird
After starting Thunderbird, then open any available nntp-server (news.gmane.org, or news.individual.org (you'll need an account to access this server)). Thunderbird will immediately block on any newsgroup with a reasonable amount of new messages. It will consume 100% CPU, slow down the whole system significantly. After a while Thunderbird is grayed out most of the time. You can't access it anymore. Sometimes this lasts for hours. Working with Thunderbird is impossible.
ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: thunderbird 3.1.7+build3+
Uname: Linux 2.6.36.2 x86_64
Architecture: amd64
Date: Tue Jan 18 21:29:52 2011
ProcEnviron:
PATH=(custom, user)
LANG=de_DE.utf8
SHELL=/bin/bash
SourcePackage: thunderbird
Thomas Schweikle (tps) wrote : | #1 |
97 comments hidden Loading more comments | view all 138 comments |
In Red Hat Bugzilla #493000, triage (triage-redhat-bugs) wrote : | #99 |
In Red Hat Bugzilla #493000, urkle (urkle-redhat-bugs) wrote : | #100 |
This is still occurring in Fedora 15 and I can not change the release # in this bug.
In Red Hat Bugzilla #493000, bill-bugzilla.redhat.com (bill-bugzilla.redhat.com-redhat-bugs) wrote : | #101 |
I sent a message to Jan and Ola asking for a version update and re-open.
In Red Hat Bugzilla #493000, redhat (redhat-redhat-bugs) wrote : | #102 |
I have not had the problem for a while.
What I did was to delete everything in .thunderbird and restart from scratch.
The I turned off local download of all folders (sync and storage), turned off "check for new mail" and turned off adaptive junk check.
For me, these settings are ok, as my imap-servers supports idle (so I don't need to pull mail), I have spamassasin in front, so I dont care about thunderbirds spamchecking and I don't normally need my email while offline (IE I am "never" offline).
I guess for others, turning off all this is not as easy to live with...
In Red Hat Bugzilla #493000, collura (collura-redhat-bugs) wrote : | #103 |
regarding the ssl comment in comment#9
(https:/
note a possibly related bug:
https:/
talks about competition for single ssl thread
In Red Hat Bugzilla #493000, andre.ocosta (andre.ocosta-redhat-bugs) wrote : | #104 |
This still happens with thunderbird 6 on F15 x86_64, same setup (IMAP+SSL). Every once in a while when it says "Downloading message..." on the status bar CPU usage goes through the roof. Funny thing is that it makes X11 process to overload the CPU, and thunderbird itself appears on 2nd place on top's list.
IMHO it would be nice if someone could update this issue's title to reflect the fact that it still happens with latest version.
In Red Hat Bugzilla #493000, urkle (urkle-redhat-bugs) wrote : | #105 |
Can someone link bugzilla.
https:/
As this seems to be the "answer" to resolving this very annoying issue.
98 comments hidden Loading more comments | view all 138 comments |
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #6 |
Prior to the Mozilla email troubles Thunderbird was fine. Now it's nearly unusable for me. I assume something in my profile but I don't know how to test it or what to look for.
- On start up no window is displayed and Thunderbird consumes 100% cpu for around 60 seconds. Soon I get a popup that says "Warning: Unresponsive Script" for "Script: resource:
- On switching windows to Thunderbird (it was running in the background) the window is unresponsive, CPU is at 100% for about 30s. CPU returns to idle, Thunderbird is usuable
What can I give you that would help figure this out? I have some local folders, but most activity is via IMAP. The sizes of the folders are similar to what they were pre-catastrophe.
This is Thunderbird 8 on Linux. I have lightning installed, but this happens with it disabled also.
In Mozilla Bugzilla #712371, Vseerror (vseerror) wrote : | #7 |
I don't know what this means "Prior to the Mozilla email troubles". Please explain.
What were your results with safe mode?
http://
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #8 |
(In reply to Wayne Mery (:wsmwk) from comment #1)
> I don't know what this means "Prior to the Mozilla email troubles". Please
> explain.
Mozilla had major email outages with some data loss last week. Perhaps unrelated but it's the only major event recently.
> What were your results with safe mode?
> http://
No difference. The only output when run from a command line is "Error -> TypeError: GetSelectedMsgF
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #9 |
Phenomenon reported to this bug can easily be observed by next procedure.
(1) Copy small Tb profileP1) to new profile(P2) for test.
- Copy ...\Profiles\
- Start profile manager(
- profile as new profile. Create Profile, name=P2, Choose Folder, and select
- the copied ...\Profiles\
(2) Create many local mail folder files under Local Folders of copied P2.
- Create file and directory for subfolders.
file : ...\Profiles\
directory : ...\Profiles\
- Create many files for mail folder under ...\Test.sbd.
For example, create X-1 to X-9999 file for Test/X-1 to Test/X-9999 folder.
create null file named X-0,
Do NN=1 to 9999; Copy "X-0" "X-"||NN ; End; (Rexx like pseudo code)
(3) Delete panacea.dat in profile directory for P2.
delete ...\Profiles\
(4) Start Tb using the copied profile, P2,
with offline mode in order to prohibit server access.
thunderbird.exe -offline -p "P2"
=> It takes long, CPU 100%, Unresponsive Script warning
(5) Repeat step (4) several times.
Similar slowess can be observed without deletion of panacea.dat,
because very many local folder files are created intentionally.
panacea.dat file holds relation among mail folder name, .msf file path, mail folder file path(Inbox instead of Inbox.msf if folder of Inbox).
If panacea.dat is corrupted or lost, directory scan for mail folder file, for assciated .msf file, for associated .sbd directory, happens on any subdirectory under ...\Mail. Because directory scan is time consuming work and CPU power is consumed by OS, and because Tb produces many directory scans than manatory/minimum number of scans, it usually takes long.
Needless to say, directory scan is applicable to .msf file, offline-store file, .sbd directory for IMAP folder too(...\ImapMail directory).
And, if IMAP, sync'ed status is perhaps lost, then, re-fetch of mail headers of all mails in all mail folders, and re-download of all mail data of all offline-use folder, perhaps happens.
Bug opener, how many mail folders do you have?
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #10 |
Created attachment 584114
screenshot of folders
Folders are as attached. The folders locally have been around for months-years with no troubles. The majority of the IMAP folders have been around for weeks-months with no troubles.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #11 |
How many files/directories are currently defined by Tb for mail folder/newsgroups?
If Linux, you can roughly check by commad like "ls --recursive --a /Tb's-prof-
Note: If garbage of .sbd directory exists, it's not shown at folder pane, but such hidden directory is also scanned if panacea.dat is corrupted.
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #12 |
Created attachment 584155
dir listing
find output is attached
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #13 |
Following is seen in list.
> ./5hyolzp2.
> ./5hyolzp2.
It inidicates that, when hidden "smat mailboxes" account("Unified Folders") is defined, directory of "smart mailboxes" already exists, then directory with suffix of "-1" was created and used.
It may indicate something wrong happened.
However, only 15 following lines is seen in directory list,
> ./5hyolzp2.
and subfolders under Inbox of IMAP account in screen shot is 15 too.
So, "phenomenon of bug 520437 due to something wrong" didn't occur in your environment, and garbages of files/directories due to problem don't exist.
Number of files/directories doesn't seem large in your case. So, it's hard to guess that a cause of slowness is directory scan, if local directory(and because of not-MS-Win case).
Note: slowness in this case seems one like next:
{number of directories} * O( {number of files in a directory}**N )
i.e. if {number of files in a directory} increases,
slowness increases very rapidly.
My example is {number of files in a directory} of only one directory==9999 case
Do you mount remote drive as ./5hyolzp2.default?
If yes, slowness/high CPU usage by directory scan like "slowness when 9999 subfolders under one local mail folder on Win" may be exposed even on Linux.
If possible, can you check next, to rule out directory scan case?
1. Copy current profile, delete panacea.dat in copied profile.
2. Start Tb using the copied profile, with offline mode in order to prohibit IMAP server access upon restart of Tb.
thunderbird.exe -offline -p
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #14 |
(In reply to WADA from comment #7)
> Following is seen in list.
> > ./5hyolzp2.
> > ./5hyolzp2.
As far as I know I don't use any smart mailboxes. Would be fine if they went away. :)
> Do you mount remote drive as ./5hyolzp2.default?
No, it's on a local SSD.
> If possible, can you check next, to rule out directory scan case?
> 1. Copy current profile, delete panacea.dat in copied profile.
> 2. Start Tb using the copied profile, with offline mode in order to prohibit
> IMAP server access upon restart of Tb.
> thunderbird.exe -offline -p
I copied the profile and tried it. The file came back and the slowness was still present. The panacea.dat I deleted was 415k, the one that was created on restart was 14k.
On a whim, I restarted Thunderbird again after the 14k file was there and it opened quickly (yay) but the IMAP folders were gone. If I go online I assume they'll come back but haven't done that in case there is more I can do for this bug.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #15 |
(In reply to Wil Clouser [:clouserw] from comment #8)
> > If possible, can you check next, to rule out directory scan case?
> > 1. Copy current profile, delete panacea.dat in copied profile.
> > 2. Start Tb using the copied profile, with offline mode in order to prohibit
> > IMAP server access upon restart of Tb.
> > thunderbird.exe -offline -p
> I copied the profile and tried it.
> The file came back and the slowness was still present.
Same degree of slowness as comment #0?
"Delete panace.dat" is in order to surely force re-validation of relation between mail folder file and file for mail folder which invokes directory scan work for mail folder files.
If same slowness was observed by it, main cause of slowness in comment #0 may be time consuming work such as directory scan for mail folder files, re-validation of relation between mail folder and file for mail folder etc.
Loss of sync'ed status of IMAP folder, loss of validated status of relation between mail folder and mail folder file etc. can produce time consuming re-sync process, re-validation process/directory scan process etc.
Other may invoke such time consuming process.
If 9999 files are created under .sbd for local mail folder, even when slowness in restart of Tb is not observed by helthy panacea.dat, sufficient slowness is always observed upon first rename/delete of a local mail folder after restart. This is perhaps because Message Filter tracks rename/delete of folder and rename/delete or Message Filter perhaps scans all mail folder and files for mail folder upon first rename/delete after restart of Tb.
This may be applicable to IMAP folder.
This may be caused by non-initialized .msf, because almost all 9999 mail folders were not explicitly opened by folder click. I can't click all 9999 folders in testing, althogh I'll try to do it if it's mandatory to reproduce very important and critical problem.
(Off-topic)
> but the IMAP folders were gone.
You merely saw phenomenon of bug 698321.
If you Go Online with the copied profile, server access happend, and it interferes daily use(e.g. if you have POP3 account of "Leave Messages on Server"=off, mail is downloaded to copied profile, then removed from server's Mbox.)
It's for testing purpose only. Please be careful.
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #16 |
Yes, deleting panacea.dat and then starting thunderbird caused the same symptoms: 100% cpu and no window for 40 seconds, then a slow script warning, upon continue the window appeared. Also, the disk activity remains near idle during the entire startup - if it were scanning directories I wouldn't expect that.
I haven't been online again since we started this.
In Mozilla Bugzilla #712371, Ludovic-mozilla (ludovic-mozilla) wrote : | #17 |
Will in which office do you seat ?
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #18 |
I'm a remote worker
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #19 |
This is affecting my productivity and I'm just going to delete my profile and start over. Is there anything else I can do for this bug?
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #20 |
Created attachment 589116
NNTP log for "Repair Folder" of news.mozilla.org, mozilla.
NNTP log for next.
- news.mozilla.org, newsgroup=
- Mark newsgroup read is already done
(1) Click mozilla.general
(2) Folder Properties, Repair Folder
=> download dialog for mozilla.general is shown
request 500 header download, with "mark others as read"
=> second download dialog mozilla.general was shown
request 500 header download, with "mark others as read"
(In reply to Wil Clouser [:clouserw] from comment #6)
> Created attachment 584155
> dir listing
Your subscribed newsgroups.
./5hyolzp2.
./5hyolzp2.
./5hyolzp2.
./5hyolzp2.
./5hyolzp2.
./5hyolzp2.
./5hyolzp2.
news.mozilla.org,rc content in my environment after "Mark newsgroup read".
mozilla.general: 1-41397
mozilla.events: 1-212
mozilla.
mozilla.
mozilla.
mozilla.
mozilla.dev.l10n: 1-12356
mozilla.
As "500 headers" only(40898-41397), five hundreds XOVER response is seen in NNTP log.
Sending: XOVER 40898-41397
And, this was repeated twice by "Repair Folder"
When panacea.dat is deleted, re-sync may occur on news group too.
And, re-sync after lost synchronization is similar to "Repair Folder".
Following newsgroups you subscribe is relatively large,
mozilla.general: 1-41397
mozilla.
mozilla.dev.l10n: 1-12356
mozilla.
So, if re-sync process of these newsgroups happens, many XOVER reponse needs to be processed by Tb during restart.
And, problem of bug 695309 is known on News.
News may be relevant to your case.
Do you need all above newsgroups?
Do you see your problem with "large newsgrops unsubscribed"?
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #21 |
I deleted the news.mozilla.org account completely. No change in performance or size of the panacea.dat file.
In Mozilla Bugzilla #712371, Vseerror (vseerror) wrote : | #22 |
Wil, how many mail folders in "local folders"?
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #23 |
Directory listing is attachment 584155. 23 folders under Local Folders.
I've since copied that entire directory structure to my new profile and the new profile is as fast as it ever was before the slowness problems. Old profile is still laggy.
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #24 |
It's still taking ~25s before I have a window open when starting thunderbird but I'm no longer seeing the unresponsive script warning. Perhaps deleting the newsgroups got it under that threshold.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #25 |
(In reply to Wil Clouser [:clouserw] from comment #18)
> It's still taking ~25s before I have a window open when starting thunderbird
> but I'm no longer seeing the unresponsive script warning. Perhaps deleting
> the newsgroups got it under that threshold.
Gain by deletion of newsgroups seems 5 sec only.
Do you have big message filter? Because msgFilterRules.dat is read one byte each(Bug 571419), big msgFilterRules.dat may affect on startup time when "new mail check upon startup" is enabled. What is file size of msgFilterRules.dat files?
Slowness is observed upon any restart of Tb? Or first restart of Tb after login or re-boot?
On Win, because Fx/Tb/Sm modules are not loaded(and cached) by re-boot, first restart after re-boot is far slower than IE, but second or later restart is not so slow because already cached. Virus scanner may affect on it too, because virus scan may be invoked on first load but it may not be upon second load.
Is this kind of slowness involved in your case?
By the way, can you use monitor tools? If you can use monitor tool for application's file access, it may help your problem analysis, because activities of Tb during restart is reflected to file access by Tb.
Bug 571419 was fortunately discovered while I was viewing Process Monitor log on Win for duplication test of other different problem. Flood of log lines for specific file/directory, too many repeated access to specific file/directory etc., is sometimes signal of something wrong around process related to the file/directory, and, by flood of log lines for msgFilterRules.dat, I was aware of the problem.
In Mozilla Bugzilla #712371, Dbienvenu (dbienvenu) wrote : | #26 |
MSGDB:5,timestamp NSPR logging on the slow profile might tell us if the slow down has to do with opening db's on startup - https:/
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #27 |
> Gain by deletion of newsgroups seems 5 sec only.
> Do you have big message filter? Because msgFilterRules.dat is read one byte
> each(Bug 571419), big msgFilterRules.dat may affect on startup time when
> "new mail check upon startup" is enabled. What is file size of
> msgFilterRules.dat files?
Two are 25 bytes, the one for ImapMail is 1.3k.
> Slowness is observed upon any restart of Tb? Or first restart of Tb after
> login or re-boot?
Every time it starts and every few minutes while it is running.
> On Win, because Fx/Tb/Sm modules are not loaded(and cached) by re-boot,
> first restart after re-boot is far slower than IE, but second or later
> restart is not so slow because already cached. Virus scanner may affect on
> it too, because virus scan may be invoked on first load but it may not be
> upon second load.
> Is this kind of slowness involved in your case?
This is a linux box and there is no virus scanner.
> By the way, can you use monitor tools? If you can use monitor tool for
> application's file access, it may help your problem analysis, because
> activities of Tb during restart is reflected to file access by Tb.
> Bug 571419 was fortunately discovered while I was viewing Process Monitor
> log on Win for duplication test of other different problem. Flood of log
> lines for specific file/directory, too many repeated access to specific
> file/directory etc., is sometimes signal of something wrong around process
> related to the file/directory, and, by flood of log lines for
> msgFilterRules.dat, I was aware of the problem.
I can give you the output of anything built into linux. Not sure what would be helpful.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #28 |
(In reply to Wil Clouser [:clouserw] from comment #21)
> Not sure what would be helpful.
Helpful for you. If special pattern is seen in log, it may be relevant to problem.
Exmples.
(A) In temporary file creation, Tb tries to create xxx-{N}.tmp, and if already exists, Tb tries to create xxx-{N+1}.tmp. Tb execute it from 1 to X always for each tempory file, so following pattern appears in log.
> For temp file of nsmail :
> For first temp file :
> create nsmail-1=>fail, ..., create nsmail-100=>success
> For second temp file :
> create nsmail-1=>fail, ..., create nsmail-100=>fail, create nsmail-101=>success
> For third temp file :
> create nsmail-1=>fail, ..., create nsmail-100=>fail, create nsmail-101=>fail, create nsmail-102=>success
Because of this, if many garbages of nsmail-X.tmp remain, performace problem arise, and it can be detected by above log pattern.
(B) Similar pattern is also seen in directory scan type access.
Tb looks to check file relevant to mail folder by procedure like next.
For folder A, tries to find file A from directory list by "compare file#1 to file#N with A, and if found, stop compare". So, if A doesn't exist, all files in directory is compared.
This is done for all known folder names. So, in average, half of files in a directory is compared per one mail folder.
Above never represents actual Tb's behavior or algorythm, but log pattern like above is seen in file access log.
This kind of log pattern can be a reason why performance degradiation happens in an environment. If time to comlete above process pattern is nealy equals to slow startup time, we can consider that above access is cause of slowness.
This is reason why I asked you about number of mail folders and number of files for mail folder, and reason why asked about slowness when panacea.dat is deleted.
When I tried to catch slowness by access like above, I intentionally created 10,000 folders/files in a directory, so I could see 10,000 * "access from 1 to N(average=5000)" in log file.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #29 |
FYI.
Process monitor log attached to Bug 545126 is another example.
That bug is for slowness in showing image in HTML mail due to "all folder file name scan per one image in an HTML mail", and log pattern of "all folder file name scan" is seen multiple times("number of imaes" times) in log. After fix of that bug, number of "all folder file name scan" is reduced very much, but such scan still remains(bug 563677). This is another reason why I asked about number of folders.
Another concern.
Althogh bug 599119 is already fixed in Tb, Bug 485538 remains in Core. If plugin scan is invoked for many tasks for some reasons during restart, it may affect on startup time. Even after fix of bug 599119, disabling plugin by addon manager may be a effective workaround if such issue exists in your environment.
Plugin scan is relevant or not is also known by file access log - if number of access to pluginreg.data is not so many, it's irrelevant to problem.
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #30 |
Created attachment 590115
NSPR logging output
Here's the output from the logging
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #31 |
(In reply to Wil Clouser [:clouserw] from comment #24)
> NSPR logging output
"NN open DB's" of each second.
> 2012-01-20 01:19:53.645697 UTC - -1223960784[
> 2012-01-20 01:19:54.327791 UTC - -1223960784[
> 2012-01-20 01:19:55.246282 UTC - -1223960784[
> 2012-01-20 01:19:56.022794 UTC - -1223960784[
> (number of opened DB doesn't increase for 8 seconds)
> 2012-01-20 01:20:04.638781 UTC - -1223960784[
> 2012-01-20 01:20:05.100987 UTC - -1223960784[
> 2012-01-20 01:20:06.002171 UTC - -1223960784[
> (number of opened DB increases by 14 within 1 second)
> 2012-01-20 01:20:06.884807 UTC - -1223960784[
> (number of opened DB doesn't increase 9 seconds)
> 2012-01-20 01:20:15.541239 UTC - -1223960784[
> 2012-01-20 01:20:15.543044 UTC - -1223960784[
> 2012-01-20 01:20:15.544233 UTC - -1223960784[
> 2012-01-20 01:20:15.544889 UTC - -1223960784[
> 2012-01-20 01:20:15.545236 UTC - -1223960784[
> 2012-01-20 01:20:15.548396 UTC - -1223960784[
> 2012-01-20 01:20:15.550835 UTC - -1223960784[
> 2012-01-20 01:20:15.551328 UTC - -1223960784[
> 2012-01-20 01:20:15.551832 UTC - -1223960784[
> 2012-01-20 01:20:15.553258 UTC - -1223960784[
> 2012-01-20 01:20:15.699753 UTC - -1223960784[
> 2012-01-20 01:20:16.556996 UTC - -1223960784[
Is 01:20:16 startup completion time? (23 seconds since first log)
If so, two "no increase for 8-9 seconds" may be a reason.
Waiting for something for 8-9 seconds?
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #32 |
> Is 01:20:16 startup completion time? (23 seconds since first log)
I would guess 01:21:00 was about the startup time. I started thunderbird, let it run until it quit using 100% cpu, and then shut it down, so that timing seems closer to me.
> If so, two "no increase for 8-9 seconds" may be a reason.
> Waiting for something for 8-9 seconds?
Sounds like it. I don't know what it would be waiting on or how to determine it. I mentioned this is all on an SSD so I don't think it's disk (disk usage isn't up at the time). It seems CPU bound. The box is a quad core i7 930.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #33 |
(A) Continuous "closing database" starts from next log.
It looks start of termination after restart completion around 01:21:00.
> 2012-01-20 01:21:18.719964 UTC - -1223960784[
(B) Following log is seen.
Cause of two "no increase of No. of open DB for 8-9 seconds"?
> 2012-01-20 01:19:53.652355 UTC - -1223960784[
> 2012-01-20 01:19:53.652430 UTC - -1223960784[
(B) Funny log sequence is seen.
Just before each "NN open DB's", "closing database ..." line and
"nsMsgDatab
Why "close a DB" and "open other DB" is executed periodically?
Expiration of DB open time for non-selected mail folder?
If so, why close(and re-open in future interval) is done for specific
folders only? Why it's not "round robin" of all mail folders?
To open all mail folders, at least "physical read data of whole .msf file for all .msf files" is required(see bug 722183 comment #5).
What is total file size of .msf files under Mail, ImapMail, News directory?
Repeatedly closed/re-opened DB in your case is /Local Folders/
This kind of issue?
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #34 |
http://
> 76 const NS_MSG_
http://
> 135 #define NS_MSG_
> 136 #define NS_MSG_
http://
> 310 case NS_MSG_
> 311 case NS_MSG_
80550005 was NS_MSG_
80550006 is perhaps NS_MSG_
What is your mail folder which is not shown as "opened DB" in log?
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #35 |
FYI.
I had mail folder file greater than 4GB for testing in a profile. After forced kill of Tb while Tb is nearly hang for testing, restart with the profile took very very long with high CPU usage, and I killed Tb before restart completion, so "restart of Tb doesn't complete" status was kept. As I knew well about intentional file of "greater than 4GB", I deleted the file of "greater than 4GB", and problem disappered immediately by the "delete file" only.
If internal rebuild-index is invoked for large mail folder during restart of Tb, it'll take long. If internal re-build index during restart of Tb, FOLDER_
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #36 |
(In reply to WADA from comment #27)
> To open all mail folders, at least "physical read data of whole .msf file
> for all .msf files" is required(see bug 722183 comment #5).
> What is total file size of .msf files under Mail, ImapMail, News directory?
Most are a few K. There are a half dozen or so that are around 5M each. There is one outlier at 57M.
In Mozilla Bugzilla #712371, M-wada (m-wada) wrote : | #37 |
"nsMsgDatabase:
Following log is seen,
> 2012-01-20 01:19:53.652328 UTC - -1223960784[
> 2012-01-20 01:19:53.652355 UTC - -1223960784[
> 2012-01-20 01:19:53.652393 UTC - -1223960784[
> 2012-01-20 01:19:53.652420 UTC - -1223960784[
> 2012-01-20 01:19:53.652430 UTC - -1223960784[
and there is no log line for 2010.sbd/inbox.msf is seen here ater in the log file.
Above log looks for me;
1. Try to open 2010.sbd/inbox.msf -> FOLDER_
2. Close 2010.sbd/inbox.msf -> 2010.sbd/inbox.msf is erased due to out_of_date
3. Try to open 2010.sbd/inbox.msf -> FOLDER_
-> Internal Rebuild-Index starts.
4. Rebuild-Index of 2010.sbd/inbox.msf continues, and is still running
after folder pane display, thread pane display.
5. Terminate Tb before Rebuil-Index completes.
6. Restart Tb -> Go to step 1.
34 comments hidden Loading more comments | view all 138 comments |
Thomas Schweikle (tps) wrote : | #2 |
This is related to Thunderbird global indexer. It is @§$%&(/=?*'{[!"@@ inefficiently written. And while indexing killing the whole system. Second point: the indexer does a full index every time a new mail arrives. And to make the whole messy: it starts over and over again -- even if it is already running for every mail arriving!! In short: if two mails arrive, two indexer threads are started. If four mails arrive four. If 100 mail arrive, 100. Quite clear now why Thunderbird renders every machine it is installed unusable -- especially if the indexer is started before the previously started indexers finished. from time to time I have about 1000 indexer threads running ...!
That is a real bug! If one indexer was started, it shall block starting further indexers, running them later, after it has finished, not trying to start as many as the system allows!
Thomas Schweikle (tps) wrote : | #3 |
Further: indexers are started for nntp too. Fetching new messages from, say gmane.org --- 2543 indexer threads running. Bad!
Thomas Schweikle (tps) wrote : | #4 |
Why don't start indexing once AFTER news where fetched, not once for every message fetched??
33 comments hidden Loading more comments | view all 138 comments |
In Mozilla Bugzilla #712371, Vseerror (vseerror) wrote : | #38 |
Wil, is it possible foryou to try this profile on a non-SSD local drive? Even if it is a different computer, but same OS. (although one wouldn't think SSD would cause high CPU - SSD might be making things worse)
In Mozilla Bugzilla #712371, Wil Clouser (clouserw) wrote : | #39 |
I don't have one available.
66 comments hidden Loading more comments | view all 138 comments |
In Red Hat Bugzilla #493000, jhorak (jhorak-redhat-bugs) wrote : | #106 |
Is this issue still seen lately? Mozilla has been trying to remove morkDB for recent Thunderbird and this could also fix this issue.
In Red Hat Bugzilla #493000, bill-bugzilla.redhat.com (bill-bugzilla.redhat.com-redhat-bugs) wrote : | #107 |
Upstream bug is still unresolved. I don't see anything in the release notes for Thunderbird 11:
https:/
101 comments hidden Loading more comments | view all 138 comments |
Launchpad Janitor (janitor) wrote : | #5 |
Status changed to 'Confirmed' because the bug affects multiple users.
Changed in thunderbird (Ubuntu): | |
status: | New → Confirmed |
summary: |
- thunderbird renders system unusable + Thunderbird renders system unusable / uses 100% CPU while indexing new + messages |
Changed in thunderbird: | |
importance: | Unknown → Medium |
status: | Unknown → Confirmed |
Changed in icedove (Debian): | |
status: | Unknown → New |
102 comments hidden Loading more comments | view all 138 comments |
In Red Hat Bugzilla #493000, bcotton (bcotton-redhat-bugs) wrote : | #108 |
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.
(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.
Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
The process we are following is described here:
http://
In Red Hat Bugzilla #493000, pahan (pahan-redhat-bugs) wrote : | #109 |
I have this issue in Fedora 17 too. Bz853638.
On big folders (in my case it was 450 000 of messages) thunderbird eat CPU 100% of 1 core and memory about 1-2Gb and freezes periodically.
I can fix it only by delete some messages from that folder.
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #110 |
Pavel, what version Thunderbird?
There was a mime performance issue resolved in TB15 or TB16 iirc.
In Red Hat Bugzilla #493000, bill-bugzilla.redhat.com (bill-bugzilla.redhat.com-redhat-bugs) wrote : | #111 |
I'm still on tb15, due to a calendaring bug in newer versions, but there a great way to exaggerate this problem is to make a saved search on one of the very large folders. Perhaps it needs to do the search n times for each selector of the saved search? I can have saved search windows take 3 minutes to open on a decent quad-core machine with mirrored SATA disks with the result set only in the hundreds of messages (perhaps 5 criteria in the search). The folder it's searching is very large, though.
In Red Hat Bugzilla #493000, pahan (pahan-redhat-bugs) wrote : | #112 |
Ыщкнб forgot mention versions:
$ rpm -qa *xulrun* *thunderbird* *mozilla* *flash*
thunderbird-
xulrunner-
mozilla-
thunderbird-
flash-plugin-
flash-plugin-
71 comments hidden Loading more comments | view all 138 comments |
Vseerror (vseerror) wrote : | #40 |
Thunderbird bug 712371 is closed. And is not related to news.
Please recheck your bug using a current version.
Changed in thunderbird: | |
status: | Confirmed → Expired |
72 comments hidden Loading more comments | view all 138 comments |
In Red Hat Bugzilla #493000, bcotton (bcotton-redhat-bugs) wrote : | #113 |
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '17'.
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 17's end of life.
Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 17 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged change the
'version' to a later Fedora version prior to Fedora 17's end of life.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
In Red Hat Bugzilla #493000, bcotton (bcotton-redhat-bugs) wrote : | #114 |
Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.
If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.
Thank you for reporting this bug and we are sorry it could not be fixed.
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #115 |
I have symptoms of this bug. Fedora 20
During the 100% CPU utilization the strace shows read() and lseek() on various *.msf files.
In Red Hat Bugzilla #493000, andrew (andrew-redhat-bugs-1) wrote : | #116 |
I can confirm that having large mail folders will mean that every now and then TBird just freezes for a minute or so. During this time, one of the CPUs is at 100%. When I have previously examined this I also found that the activity was around the msf files for the big folders.
In Red Hat Bugzilla #493000, bill-bugzilla.redhat.com (bill-bugzilla.redhat.com-redhat-bugs) wrote : | #117 |
I dropped the ball on working on this, but if anybody wants to debug this, BMO had a suggested path:
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #118 |
That does not work (unfortunately).
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #119 |
This bug is so painful to me, that I'm willing to pay money to whoever resolve this issue:
https:/
In Red Hat Bugzilla #493000, bill-bugzilla.redhat.com (bill-bugzilla.redhat.com-redhat-bugs) wrote : | #120 |
Just to clarify, I looked at the BMO bug again, and Miroslav was experieincing this problem with Global search/indexing disabled. So, something else is pounding the msf files, apparently.
I only found an article about firefox, but this works for thundebird too, to run under gdb:
$ thunderbird -g -d gdb
(gdb) handle SIG33 noprint nostop
(gdb) run
(with xulrunner and thunderbird debuginfo installed, obviously)
I'll try it on my desktop which exibits this and see if I can get a useful backtrace.
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #121 |
For the record. The size of my
~/.thunderbir
is 7GB. And probably include milions of emails in dozens folders. So this can give you idea how big the directory have to be to have those symptomps nicely visibile.
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #122 |
please provide the details in https:/
thanks
Changed in thunderbird (Ubuntu): | |
status: | Confirmed → Fix Released |
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #123 |
(In reply to Andrew Meredith from comment #47)
> Created attachment 436336 [details]
> gdb backtrace of thunderbird freeze
>
> I noted the filename
> /home/meredith/
> msf as part of the trace. This file is about 100M.
(In reply to Andrew Meredith from comment #49)
> After rebuild it is now 110M
(In reply to Andrew Meredith from comment #72)
> I can confirm that having large mail folders will mean that every now and
> then TBird just freezes for a minute or so. During this time, one of the
> CPUs is at 100%. When I have previously examined this I also found that the
> activity was around the msf files for the big folders.
Andrew,
A 100M size .msf suggests the corresponding folder is ~1GB.
1GB would be very large for a junk folder, which will like cause performance issues. (Dont' know why junk, and trash, are worse than other large folders - perhaps because they are high activity?)
Why are you letting the junk folder get so big?
And what happens if you empty your junk folder? (right+click on junk, empty)
In Red Hat Bugzilla #493000, andrew (andrew-redhat-bugs-1) wrote : | #124 |
I was purely using the Junk folder as an example. It was so big because it had been a while since I last reviewed it and pushed the genuine spam through spamassassin.
Compacting made no difference to either the msf size or the occurances of regular stalls in TBird itself. After reducing it in size and doing another compact, the stalls ceased and the msf was minute.
This was all to further illustrate that TBird stalls for a disproportionately long time with bigger folders.
In Red Hat Bugzilla #493000, bcotton (bcotton-redhat-bugs) wrote : | #125 |
This message is a reminder that Fedora 20 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 20. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora 'version'
of '20'.
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.
Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 20 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #126 |
This is still valid in Fedora 22.
And I increased the bounty for this bug resolving to $150.
https:/
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #127 |
(In reply to Miroslav Suchý from comment #82)
> This is still valid in Fedora 22.
> And I increased the bounty for this bug resolving to $150.
>
> https:/
> freezes-
Very generous Miroslav. I hope this helps get a fix. I'd like to offer some suggestions to increase your odds of success:
#1 you should mention this in an appropriate bug at bugzilla.
#2 if you don't find an appropriate bug at bugzilla.
the key to finding a precise matching bug, if one exists, is narrowing the list of symptoms, and determining whether the list of symptoms is just one item, or a combination. Note - suggest you disable automatic compact first anything, in tools | options | advanced | network and disk. To name some possibilities:
- high memory (over 500mb, if so how much)
- high cpu
- global index off or on
- autosync (syncrhonize messages) off or on
- IDLE enabled or not (account settings, server settings, advanced)
- situational:
* only during startup
* only when manually or automatically getting new messages
* only when clicking a folder
The list of candidates bugs for the most part have pretty well narrowed summary titles - https:/
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #128 |
As stated in #78 the relevant report in Mozilla tracker is:
https:/
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #129 |
(In reply to Miroslav Suchý from comment #84)
> As stated in #78 the relevant report in Mozilla tracker is:
> https:/
Miroslav
Yes, that was what *I* suggested a year ago, so I was asking again because I may be wrong. But I see your comment 71 "shows read() and lseek() on various *.msf files.", so perhaps your issue is a solid fit to Bills. If you are satisfied they are the same then let's continue in bug 561272.
NOTE - there's still more we need to know about Bill's issue, and yours. And you do not seem to be answering questions like https:/
I'm hoping the problem will turn out to be autosync related. Regardless, a recap from you and Bill on the following points would be very helpful to anyone who would be interested in your bounty. So in a single comment in bug 561272 please give the following information:
- high memory? (over 500mb, if so how much)
- high cpu?
- fails with global index off?
- fails with autosync (syncrhonize messages) off?
- fails with IDLE disabled? (account settings, server settings, advanced)
- fails with compact disabled? (tools | options | advanced | network and disk)
- reliable network connection i.e. is not failing/is solid during your high CPU usage?
- what is the name and version of the imap server?
- maximum nunmber of messages in folder? (size is mentioned in bug 561272 comment 13, but not a message count)
- your list of addons?
- a profile url from https:/
- if it fails because of imap or autosync activity then a log of the activity will help https:/
Anything else you think is relevant.
Lastly, your bounty is not going to be an incentive in the BMO bug until you mention it there.
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #130 |
p.s. Bill's comment 35 is a very cool diagnostic. But PLEASE (anyone) do not do a repair on a folder without first copying both the folder and the .msf file to a safe location outside the profile
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #131 |
Please see:
https:/
for solution.
I.e. set
mail.db.max_open to 1000
From my POV this is resolved by Wayne Mery solution, and you can redeem the bounty for this bug (not sure what the process is as I never redeemed bounty. I am willing to assist if you have problem with redeem.).
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #132 |
(In reply to Miroslav Suchý from comment #87)
> Please see:
> https:/
> for solution.
>
> I.e. set
> mail.db.max_open to 1000
>
> From my POV this is resolved by Wayne Mery solution, and you can redeem the
> bounty for this bug (not sure what the process is as I never redeemed
> bounty. I am willing to assist if you have problem with redeem.).
It is pleasing that you have good results. But I do not think this bug should be closed until the reporter and/or a few others who have previously commented are able to confirm the problem is gone while version 38 with default settings.
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #133 |
Thunderbird 38 is currently in Fedora stable so I gave it try.
I tried to reset mail.db.max_open to default value. While the lags are not so big, after while I still found some. It was like 5 secons vs. 50 seconds as when I originally reported this issue.
After few hours I set mail.db.max_open back to 1000 and even those 5 secs lags are gone.
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #134 |
(In reply to Miroslav Suchý from comment #89)
> Thunderbird 38 is currently in Fedora stable so I gave it try.
> I tried to reset mail.db.max_open to default value. While the lags are not
> so big, after while I still found some. It was like 5 secons vs. 50 seconds
> as when I originally reported this issue.
> After few hours I set mail.db.max_open back to 1000 and even those 5 secs
> lags are gone.
Miroslav,
What happens if you set mail.db.max_open to something more modest, like 60?
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #135 |
Modest value works as well. As long as it is bigger then number of my folders (something between 100 and 200 in my case).
When I use smaller value than my number of folders (e.g. 60 I see performance decreese).
In Red Hat Bugzilla #493000, vseerror (vseerror-redhat-bugs) wrote : | #136 |
(In reply to Miroslav Suchý from comment #91)
> Modest value works as well. As long as it is bigger then number of my
> folders (something between 100 and 200 in my case).
> When I use smaller value than my number of folders (e.g. 60 I see
> performance decreese).
How many accounts do you have? pop or imap?
Do you use any server side filters or folder properties set to check for new mail (other than Inbox)?
What extensions are installed? (listed at help | troubleshooting)
Do you see the problem with Thunderbird started in safe mode?
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #137 |
(In reply to Wayne Mery (:wsmwk) from comment #92)
> How many accounts do you have? pop or imap?
2 IMAP accounts
1 news (UUTP)
1 RSS
> Do you use any server side filters or folder properties set to check for new
> mail (other than Inbox)?
Yes, I have 14 folders on server (beside those basics) and I have 14 servers sides filters which move mails to those folders.
All remaining folders are local and are handled by Thunderbird filters.
> What extensions are installed? (listed at help | troubleshooting)
Awesome LdapInfoShow
Enigmail
Lightning
MailRedirect
QuoteCollapse
> Do you see the problem with Thunderbird started in safe mode?
Yes.
Changed in thunderbird (Fedora): | |
importance: | Unknown → Medium |
status: | Unknown → Fix Released |
In Red Hat Bugzilla #493000, msuchy (msuchy-redhat-bugs) wrote : | #138 |
I believe that the bounty
https:/
still has not been picked up. Guys, that is $150 in limbo. Pick it up. This has been resolved and someone deserves the money.
Fedora 13 changed to end-of-life (EOL) status on 2011-06-25. Fedora 13 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.
If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.
Thank you for reporting this bug and we are sorry it could not be fixed.