[MASTER] Committing to urlclassifier3.sqlite causes excessive CPU usage and disk I/O

[Parag Warudkar]

Note: the follow up bug for this is bug #229745:
 #229745 - after fix for #215728 - Committing to urlclassifier3.sqlite still causes excessive CPU usage and disk I/O (the 2nd)

===

Browsing with Firefox 3.0b5 on Hardy (All updates applied) causes Xorg to use 50-60% CPU all the time.

This is not limited to a particular site or page - it happens any time when Firefox is rendering pages.

The high CPU usage makes browsing very jerky - switching tabs takes more time than it should, pages appear frozen for a brief moment.

===
From https://bugzilla.mozilla.org/show_bug.cgi?id=430530#c4:

After some testing, it looks like this gets pretty bad on linux.

Once we hit sqlite's cache size, sqlite starts hitting the disk pretty hard.
While the database is small this isn't a big deal (we don't hit the cache
limit), and once we're up-to-date it isn't as bad (we aren't adding as much
data).

But during the middle/end of the push to get the complete database, things take
way too long, and we're thrashing the disk the whole time.

We can reduce the amount of IO by increasing the cache size, and that'll help
some (it'll help a whole lot if we increase the cache size by enough to hold
the db). We might also need to look at throttling the update process a bit, so
that we don't slam the system all at once...

===

To reproduce:

1. enable browser.safebrowsing.malware.enabled + browser.safebrowsing.enabled
2. observe IO on urlclassifier3.sqlite while browsing the web

[Miguel Martinez (el-quark)]

This is also happening here. Firefox 3 on uptodate Hardy is also causing a lot of I/O, which is probably the eventual cause of the high CPU usage. Furthermore, there is also more people affected, as evidenced in the following forum thread:

http://ubuntuforums.org/showthread.php?t=759673

[liamdawe (liamdawe)]

I am also having the exact same problem doing "top" in terminal shows firefox using massive amounts of cpu like a person pointed out in the thread linked above!

[Andrea Garbarini (garba)]

same problem here, this is making firefox unusable for me, i hope the devs understand how serious this issue is

[unggnu (unggnu)]

I can confirm this with current Hardy. The cpu usage seems to mainly come from the high disc activity.

[Miguel Martinez (el-quark)]

Some say that disabling the xulrunner language packs does help. I can't confirm it right now, though. I will try it and see what happens.

[Miguel Martinez (el-quark)]

I have disabled the language addons (first only xulrunner and then both xulrunner and firefox) and the hard disk trashing still occurs. Please note that I have been running the firefox update that came this morning (April 22). However, epiphany-browser (using gecko 1.9) works just fine.

[Alexander Sack (asac)]

please submit info about this firefox package version you are exactly using. Further, please provide an example url that consumes that much CPU.

Also submit a complete list of extensions you are using. try to run firefox -safe-mode to see if disabling extensions helps.

[Miguel Martinez (el-quark)]

I'm using firefox version 3.0~b5+nobinonly-0ubuntu3 and xulrunner-1.9 version 1.9~b5+nobinonly-0ubuntu3. Extensions used are:
-Adblock Filterset.G updater 0.3.1.3
-Adblock Plus 0.7.5.4
-Flashblock 1.5.5
-Ubuntu Firefox Modifications 0.5

I need to wait some time for the HD access to occurr, and it has happened in http://www.phoronix.com/scan.php?page=home and at http://planet-f1.com/ I've just had it happen in google, with no other tabs opened and loaded directly after my home page. I was browsing with epiphany while this happened, so that may invalidate the results. Some users also report issues in ubuntuforums.com although I have been using epiphany for two days and can't comment.

I will try to use firefox in safe mode and report results as soon as possible.

[Miguel Martinez (el-quark)]

OK, I just ran firefox in safe mode and just left it at my home page http://www.phys.ufl.edu/fermisurface/ After a while, it started using CPU and using the hard disk, although memory usage according to htop didn't skyrocket. Completely closing firefox takes a while once the "thrashing" has started: more than it takes to close the window as if it were doing some I/O operations after that. The system goes back to normal as soon as it is closed.

It would be great if some other could confrim or deny what I'm seeing here.

[bendis (bendis)]

Hi,

I have exactly the same symptoms. It does not matter what page is loaded in firefox or whether the safe-mode is on or not, the cpu and disk trashing starts in a minute or two after firefox is launched. Firefox becomes unresponsive and closing it takes about ten seconds. Once closed, cpu and disk calm down.

This problem started to occur a few days ago...

My versions are:
firefox 3.0~b5+nobinonly-0ubuntu3
xulrunner-1.9 1.9~b5+nobinonly-0ubuntu3

[bendis (bendis)]

Well, I just found out what is firefox doing during this high cpu and disk activity - it does something nasty with the urlclassifier3.sqlite file (about 20MB in size) in may home firefox profile directory. The urlclassifier3.sqlite is being touched and the urlclassifier3.sqlite-journal file constantly grows.

I just googled around and found this: http://www.mozillazine.org/talkback.html?article=22714 (look for "Firefox 3 beta 1 memory hog on Windows").

Regards,
Bendis

[Nick B. (futurepilot)]

Yes, it does not matter what site you visit. The high CPU and disk trashing will happen almost immediately after launching Firefox 95% of the time and then periodically afterwards, even if Firefox is idle.

[Gumper (rlutzinger)]

Same problems as everyone else. Lots of HD activity and FF becomes unresponsive. This just started happening about 2 days ago for me.

[50sQuiff (aflitcroft)]

I have absolutely the same symptoms being described here too. This has started happening after a very recent update.

[unggnu (unggnu)]

Have you tried a new profile? Close Firefox, move .mozilla to a backup location and then restart the browser. Maybe temporary but it fixes the problem for me so bendis could be right.

[unggnu (unggnu)]

Btw. some information about urlclassifier3.sqlite http://forums.mozillazine.org/viewtopic.php?t=640383. In the standard configuration checking of "suspected forgery" is disabled in Firefox in Ubuntu.

[sebbouckaert (sebastiaan-bouckaert)]

Same happens here with Firefox 3.0 b5+nobinonly. And this started after updating Hardy on april 22th (CET). I also happen to use Nvidia-new driver and Compiz.

[bendis (bendis)]

to unggnu: I just deleted the urlclassifier file - Firefox created a new one (9KB in size) and everything seems fine now...

Bendis

[Bryce Harrington (bryce)]

From the comments (esp. #19) it seems this is not an xorg issue, so closing that component.

[Christian Wolf (christianwolf)]

If Firefox 3.0 b6 does not make it into the final release, I suggest to roll back to FF 3.0b4 and to skip b5.

I also had to stop using FF b5 on my Hardy systems since it is barely usable after a few minutes of browsing.

Together with the random scrollkeeper attacks on my HD, CPU and memory which started to show up again since a few days, this is a rather unpleasant desktop experience at the last day before release.

I would call this a typical example of Murphy's law..... ;-)

[unggnu (unggnu)]

@bendis
If you don't disable the forgery and attack site options the db will be updated gradual again.

I guess the beta5 of Firefox will stay for three years (LTS desktop support time) or are there plans to upgrade it with newer Hardy releases?

[getaceres (getaceres)]

The same happens to me. Since the last update (2 or 3 days ago) I get an unusable firefox that eats CPU constantly.

[Chris Coulson (chrisccoulson)]

Changing this from Incomplete to Confirmed, as I can only see one request for information here (Alexander), and it was answered by Miguel.

This is also happening on my system (same version of firefox and xulrunner as everyone else)

[Dcamp (dcamp)]

There's a thread at http://ubuntuforums.org/showthread.php?t=759673 complaining that some users are seeing painful amounts of disk IO during safebrowsing updates.

I'm not sure how widespread this is, but regardless I think we can help this a lot with some index and cache size tweaking. I'm working on getting some numbers and a patch together now.

[Reed Loden (reed)]

Heh, I _just_ mailed you about this, and then I look at my bugmail, and I see this bug was just filed. Hah.

[Reed Loden (reed)]

See also http://ubuntuforums.org/showthread.php?t=763773 and http://ubuntuforums.org/showthread.php?t=763742.

[Alexander Sack (asac)]

If you see this, please stop any firefox running, backup your urlclassifier3.sqlite from your .mozilla/firefox/... profile. then remove that file from your profile and restart firefox. Does this help?

[Alexander Sack (asac)]

btw, please keep the backup in case we need more info.

[Beltzner (beltzner)]

Dave, can you renom when you've got those numbers together to help us understand the impact/size of the problem?

[Alexander Sack (asac)]

no replies to my last question. setting to incomplete again.

[Chris Coulson (chrisccoulson)]

Alexander - I got rid of the file last night, and I didn't notice any more lock-ups. The file was re-created but only 9kB. I will need to use it a bit more tonight after work before I can conclude whether it has fixed the issue.

[Nick B. (futurepilot)]

I removed the urlclassifier3.sqlite file, but it did not help any.

[Chris Coulson (chrisccoulson)]

My urlclassifier3.sqlite file has grown back to 17.4MB this evening and the periodic high CPU load and disk I/O seems to have returned

[Dcamp (dcamp)]

After some testing, it looks like this gets pretty bad on linux.

Once we hit sqlite's cache size, sqlite starts hitting the disk pretty hard. While the database is small this isn't a big deal (we don't hit the cache limit), and once we're up-to-date it isn't as bad (we aren't adding as much data).

But during the middle/end of the push to get the complete database, things take way too long, and we're thrashing the disk the whole time.

We can reduce the amount of IO by increasing the cache size, and that'll help some (it'll help a whole lot if we increase the cache size by enough to hold the db). We might also need to look at throttling the update process a bit, so that we don't slam the system all at once...

I've sent mail to Dr. Hipp looking for suggestions, and any other suggestions would be appreciated.. :/

[Parag Warudkar (parag-warudkar)]

Alexander - your last question was "If you see this, please stop any firefox running, backup your
urlclassifier3.sqlite from your .mozilla/firefox/... profile. then
remove that file from your profile and restart firefox. Does this help?"

And it was answered 3 times - Chris Coulson (2 times - helped initially but file grew and had the same problem) and Nick B (1 time - did not help any.)

Setting it to Confirmed again.

[unggnu (unggnu)]

The file keeps growing to previous size if the phishing and attack site filter isn't disabled.

If the file can be compressed well maybe it is a good idea to upload it here so that the devs can retrace it. With 7z my 22 MB file is reduced to 5.7 MB. I don't know if it is too much for Launchpad but since I don't experience the problem atm I couldn't upload it. Theoretically only both options have to be enabled and Firefox restarted several times (Firefox needs some time to download the part) to gain the full size of the file again since it is downloaded in chunks.

[unggnu (unggnu)]

It seems to happen again with a fresh profile and both options enabled so I upload the file. If it is too big or not needed anymore just delete it. For testing copy both files to Firefox profile and wait some time ~1 minute I guess. It even happens with a blank page.

[f3a97 (f3a97)]

Hi,
      same problem here too.

I've previously opened a new bug, where I attached some iostat and Systemtap measuerements of FF3 (iotime and disktop).

Please see them here:

https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/221842

I can confirm too that this huge IO activity was not present some days ago, so it must be a recent Hardy change.

[Gleidson Lacerda (gleidsonlm)]

+1

Linux version 2.6.24-16-generic (buildd@palmer) (gcc version 4.2.3 (Ubuntu 4.2.3-2ubuntu7)) #1 SMP Thu Apr 10 13:23:42 UTC 2008
Mozilla/5.0 (X11; U; Linux i686; pt-BR; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5

[Michael Rooney (mrooney)]

After looking at upstream, it looks like they have a pretty good handle on this:

"After some testing, it looks like this gets pretty bad on linux.

Once we hit sqlite's cache size, sqlite starts hitting the disk pretty hard. While the database is small this isn't a big deal (we don't hit the cache limit), and once we're up-to-date it isn't as bad (we aren't adding as much data).

But during the middle/end of the push to get the complete database, things take way too long, and we're thrashing the disk the whole time.

We can reduce the amount of IO by increasing the cache size, and that'll help some (it'll help a whole lot if we increase the cache size by enough to hold the db). We might also need to look at throttling the update process a bit, so that we don't slam the system all at once..."

[jeroenl (jeroenl)]

I added some info in the duplicate bug #221760. The most important what I found out is this:

When I let Firefox run with the disk activity and the 100% CPU for approx. half an hour, all problems are gone. Disk activity stops and CPU goes back to a normal level. When restarting Firefox, it seems the problems are not coming back, but after 10 min or so, the activity starts again from the beginning. This time only for a few minutes. After that I didn't had any problems anymore (for 15 min.).

My urlclassifier3.sqlite is now 26,1 MB.

[Dcamp (dcamp)]

OK, so I split the current google database into 3 pieces (which are much bigger chunks than they're currently giving us now, and bit bigger than they're planning to start feeding us during an initial update) and fed them to the DB one piece at a time. This gives us ~800,000 urls to deal with each update (some are adds that will be applied, some are subs that will cause an add to be deleted, some are subs that will be saved to apply to a future add, and some are adds that will cause a previously-applied sub to delete).

On OSX and Vista and vista, it takes ~30-35 seconds to apply one of these chunks the entire process. On ubuntu, it takes 10 minutes of system-performance-degrading IO to deal with that. I'm sure it's worse on network filesystems (though we use the Local data dir on osx and windows).

At its largest size, (which is actually near the middle of the update, because the update tends to send us a bunch of subs-that-don't-have-adds-yet that fill up the database, then the adds that remove them come later) the database has roughly 35,000 1k pages. During one of these big initial updates all the pages will be touched quite a few times.

Once we've caught up to the state of the server, our updates are going to be much smaller - usually in the low hundreds or occasionally a thousand or two. This will touch many fewer pages in the database (I haven't measured this closely).

The only way I see to get this performing decently is to bump up the max page cache size to fit all the pages during updates. If we bump the max page cache to a pretty big size (say, 35-40 megs), we'll avoid hitting the disk until commit time for even the largest updates. We can blow away that cache after the update, so it would be a transient spike in memory during updates, relative to the size of the update.

This sucks, but it's all I can think of. Any thoughts or suggestions?

[Parag Warudkar (parag-warudkar)]

If Vista and OSX both can do the updates in 30-35 seconds as you quote - I am wondering if anything is wrong with Linux/kernel?

I originally reported this as a problem against Ubuntu Hardy with high CPU utilization while loading pages but the bug report got hijacked by disk thrashing bug as it apparently also consumes CPU while thrashing the disk - I have never had disk thrashing problem.

Is there a way to reproduce this easily? I have debug build on my machine so I can try to figure out what is going wrong.

[Sylvain Pasche (sylvain-pasche)]

The trouble with Linux may be in the way the underlying filesystems implement fsync() which I guess is called a lot during the trashing. See bug 421482 comment 11 or bug 421482 comment 12.

[jeroenl (jeroenl)]

Removing urlclassifier3.sqlite only works partly. It indeed stops the behaviour for a while, but today it started again, but the duration of the high disk/CPU activity takes not as long as the moment it all started (max 1 min.). The size of urlclassifier3.sqlite is now 29.7 MB.

[R (Chandra) Chandrasekhar (chandra)]

I can confirm all of the above, especially the hammering of the hard disk and firefox being on top of top, after upgrade from Gutsy to Hardy on Kubuntu.

My packages are:

dpkg -l firefox xulrunner
||/ Name Version Description
+++-==============-==============-============================================
ii firefox 3.0~b5+nobinon meta package for the popular mozilla web bro
un xulrunner <none> (no description available)

with several firefox add-ons.

My urlclassifier?.sqlite file sizes are:

4970496 2008-04-26 23:42 urlclassifier2.sqlite
23249920 2008-04-27 17:01 urlclassifier3.sqlite

I have now disabled firefox checking for attack sites and suspected forgery sites in Preferences -> Security.

Is there anything else to be done? I fear for my hard disk.

The peak I/O activity is so intense that I cannot move mouse focus from one firefox tab to another for two or three minutes. It is worth fixing this problem fast.

Thanks.

[marcw (marcw)]

Chandra, now that your two pref -> security checkboxes are cleared, just delete those 2 usrlclassifier* files and you should be good to go. They will be recreated next time you start Firefox but will not grow beyond 9k. At least that's my experience.

[Gearoid-murphy (gearoid-murphy)]

*** Bug 430649 has been marked as a duplicate of this bug. ***

[R (Chandra) Chandrasekhar (chandra)]

Thank you marcw.

I have done what you have suggested. Now urlclassifier2.sqlite is absent and urlclassifier3.sqlite is 9216 bytes as you have said.

Firefox is quiet and responsive now. I will post if there are any changes.

[António Lima (amrlima)]

+1 here with hardy up to date. It's unacceptable to release like this. Cpu usage is always at 50% and disc usage is very high.

[Iain Parris (ipv2)]

+1, hardy, up to date.
Workaround was successful (disabling phishing/attack detection, closing Firefox, deleting ~/.mozilla/firefox/[profile].default/urlclassifier*.sqlite , opening Firefox).

[jaiwithani (jaiwithani)]

+1, glad it's being fixed. Curious as to why this problem doesn't occur on other OS's. Same fix as everyone else - deleting urlclassifier3.sqlite

[Jon Wilson (j85wilson)]

+1, used same workaround as iain, successful so far. url3 was up to 38M, and url2 was up to 4.9M before I deleted them.

[Marius Gedminas (mgedmin)]

Created attachment 318072
strace output of firefox's fsyncs

Here's some data: In 11 minutes of light browsing (I visited perhaps 6 pages), firefox called fsync() 17 times on places.sqlite, 47 (!) times on places.sqlite-journal, and 12 times on other files (one is formhistory.sqlite, the other appear to have been closed already, so I don't know). Total time spent inside fsync system calls: 63 seconds. Longest nonresponsive period: 12.8 seconds.

I've attached the strace log with timestamps and durations. The file descriptors mentioned in it are:

32 /home/mg/.mozilla/firefox/default.a83/places.sqlite
40 /home/mg/.mozilla/firefox/default.a83/formhistory.sqlite
49 /home/mg/.mozilla/firefox/default.a83/places.sqlite-journal
53 54 65 70 already gone from /proc/.../fd/

Strangely enough I had't noticed excessive disk I/O or bouts of nonresponsiveness in Firefox 3.0 beta 4. The data here is collected from beta 5 (the build included in Ubuntu Hardy Heron), where the fsyncs make the browsing excessively painful.

[Dan Andreșan (danyer)]

@jaiwithani: please not that this is not a fix, it is a workaround. An important functionality of firefox was disabled.

Anyway, it's good that I can use firefox again and I'm confident that the developers will provide a fix to this problem.
No need to add more comments, let's all watch Update Manager notification icon :)

[les marques (lasmarkas)]

same problem for me , this morning .
yesterday afternoon, i enabled the phishing detection .

il let the attack site option enabled , and will look if it happens again

[Christophe Pradier (cpradier-gmail)]

Hi.
I see the bug is closed with a workaround but I would like to ask another question. For me, the bug has started to happen with Firefox 3.0b5 after the upgrade to Hardy. But it is also the first time I started Amarok, that I use permanently since, and that uses sqlite inside.

So I wanted to ask if some others of you were running sqlite programs at the same time they noticed the bug ?

[Blondel Roger (rblondel77-deactivatedaccount)]

je ne parle que le français
merci.
problème résolu
r. blondel

[Christophe Pradier (cpradier-gmail)]

Comme tout était en anglais, j'ai écrit en anglais. En français, ça donne ceci :

Salut,
Je vois que le bug a été fermé avec un workaround mais je voudrais poser une autre question. Pour moi, le bug a commencé à se produire avec Firefox 3.0b5 après une mise-à-jour vers Hardy. Mais c'est aussi la première fois que j'ai utilisé Amarok, que j'utilise en permanence depuis, et qui utilise sqlite à l'intérieur.

Donc je voulais demander si d'autres d'entre vous utilisaient en parallèle des programmes sqlite quand ils ont remarqué le bug ?

[jeroenl (jeroenl)]

@Christophe: No it has nothing to with other applications using sqlite, like Amarok. I use Amarok to, but with mysql and I also experienced this bug. I also don't see the bug is closed with a workaround. It is still open and I am sure it will be fixed. Also RC1 is around the corner (although it seems to be delayed) and will will hopefully also have a fix of this problem.

[Andrea Garbarini (garba)]

i've handed a hardy cd to a friend of mine and he's been complaining over the last few days about firefox being totally unusable, needless to say his conclusion was "linux sucks"

i know im abusing the bug report but i really can't understand how you could find reasonable to release hardy with such an outstanding problem affecting the most used application on a desktop system, this bug (or whatever you want it to be called) should have been regarded as a major showstopper for a good release, best regards, andre

[Beltzner (beltzner)]

Looks like this is more than just safebrowsing?

[petit-prince (petit-prince)]

The workaround worked for me as well, at least for now. This is a major show stopper, I actually had to use an alternative browser to find the workaround Alexander suggested.

[Scott Lewin (sclewin)]

+1. The workarounds seems to work fro me also, for now. Hopefully a fix will be out soon.

[Parag Warudkar (parag-warudkar)]

So why do we need to call fsync so many times on the safebrowsing and places/formhistory data in first place? Why are data integrity gurantees required for that sort of data? I mean no big deal if we did an update and the browser crashed - ff is not a database after all and this type of data is not like it is going to cause a lot of problems if it is lost.

I suspect that the problem is sqlite trying to be paranoid - there should be a way to tell it to do relaxed syncs - in a separate low CPU and IO priority (where supported) thread or something like that.

[John Vivirito (gnomefreak)]

Scott this is tagged as a blocker in Mozilla bug so they will fix it before rc1 is released or better yet they wont release RC1 until this is fixed.

[Benjamín Valero Espinosa (benjavalero)]

Well, it seems that removing the files 'urlclassifier3.sqlite' and 'urlclassifier3.sqlite-journal' all works fine.

[Michael Rooney (mrooney)]

Hi Andrea. Keep in mind when looking at a bug report that you are getting an incredibly biased view of the effect of the bug, as few people are going to post in a bug report saying to say the bug does not affect them. This bug does not affect all Firefox users in Hardy across the board. In some cases the only way to catch bugs like this is to release the package; if more people tested our Alphas and Betas perhaps more issues like this could get resolved earlier. I and many others have been using Hardy for months in Alpha and Beta and I had not heard of this bug until release, and it certainly does not apply to my install. Just something for you to think about that I hope is useful. Thanks for your comment, and thanks for using Ubuntu! As others have said, Mozilla is aware of this bug and it should be resolved in a timely manner.

[Sergei Turchaninov (sadistt0)]

same problem, high CPU usage and high I\O, system goes unusable, after closing FF3 all back to normal...

ATI X1300 Mobile, fglrx driver from 8.04 all updates are installed

[Jeffrey Baker (jwbaker)]

There may be a few different problems here. First problem may be sqlite vacuums. It looks to me like Firefox occasionally reads and writes the entire contents of large sqlite files (~50MB in my profile). Second, there seems to be very excessive preallocation or wasting of space. For example I just did an offline sqlite vacuum which reduced my profile from 50MB to 38MB. Then I started the browser and visited one site, and the sqlite files in my profile are up to 47MB. That's 9MB for one site visit?

The reason I suspect a vacuum issue is that I see this very regularly. Just as a rough guess, I get a 20-30 second hang on every 20 page loads.

I'm testing with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9b5) Gecko/2008041514 Firefox/3.0b5

[Jeffrey Baker (jwbaker)]

Created attachment 318333
Strace of Firefox 3.0b5 on Linux

Throwing in an additional strace. This is the result of strace -fte open,close,write,fsync. During this 150-second run, Firefox writes an incredible 34MB of crap (and then, of course, periodically fsyncs). The write storm starts at timestamp 00:02:36. File 80 is urlclassifier3.sqlite-journal and 44 is urlclassifier3.sqlite. Output to these files continues quite heavily until the browser shuts down at 00:03:20.

Contents of write() calls expurgated for privacy's sake.

[Martin Kaffanke (martin-kaffanke)]

sorry that I didn't find this bug. Would be good to tell the reporters to make a really specific subject, 9 duplicates on a short time, but 'high cpu usage' can really mean anything.

[Sylvain Pasche (sylvain-pasche)]

*** Bug 431094 has been marked as a duplicate of this bug. ***

[Bartek (tschew)]

113654 is most probably NOT a duplicate of this bug, i don't have any of the issues reported here and only get the problems when visiting that specific page

[Beltzner (beltzner)]

From today's triage call:
 - might be able to ship the DB w/Ubuntu?
 - might be due to cost of a sync?
 - will probably suck for network mounted profiles
 - karl thinks he's seen it in gentoo

Dave thinks that the easiest way to solve this is let the DB have more cache memory, but that will potentially affect our memory stats in the firstrun case.

Do we know any SQLLite gurus?

[Jruderman (jruderman)]

How about making firefox download a database *snapshot* incrementally? We know how to download files incrementally (see software update). With this approach, users won't be protected until they get the whole thing, but Firefox might use less memory and CPU as firefox downloads it the first time, and Firefox might even be able to download it more quickly.

[Dcamp (dcamp)]

(In reply to comment #16)
> How about making firefox download a database *snapshot* incrementally? We know
> how to download files incrementally (see software update). With this approach,
> users won't be protected until they get the whole thing, but Firefox might use
> less memory and CPU as firefox downloads it the first time, and Firefox might
> even be able to download it more quickly.

If we can figure this out on the server side, it would be reasonably easy to implement, and would almost certainly be a better user experience (downloading roughly the same amount of data, but no significant CPU/memory usage). So yeah, if we can figure out where to host these snapshots this would be a good solution.

[Dolske (dolske)]

I'd be a little wary of changing the basic way the DB is initially downloaded (or shipped). Tweaking the existing mechanism would seem lower-risk.

[Rickstockton (rickstockton)]

I'm uncertain that this is purely a Linux phenomenon-- comment 7 of bug # 431094 is from a Windows user, and a 'Lars Erik' talked about this within yesterday's "The Official Build of 20080429 is out" Thread over on Mozillazine. (I sent him a PM there, asking him to come over here and comment). Here's his post:

Lars-Erik
Joined: 30 Nov 2007
Nydalen, Oslo, Norway PostPosted: Mon 28th Apr 2008 8:20am

"Anyone else but me that have HUGE CPU usage and I/O.
Loading some pages cause my HDD to go nuts, my A/V to scan constantly and Minefield CPU usage go 100%!!"

He says he's got an A/V scanner going wild, and most Linux Desktop users don't even bother to have one-- so I'm suspicious that he might be on Windows.

[Supernova00 (supernova00)]

My windows builds sometimes takes forever to start up and the CPU useage is probably around 100% because everything is very slow to respond (switching windows to Firefox 2 or Thunderbird). I haven't checked to see if the CPU useage is at 100% though. How am I supposed to figure out if it is url-classifier downloading/updating during this?

[Rickstockton (rickstockton)]

I described my own experience in reply to Lars, here's what I said. BTW, I am on RPM-based Mandriva-2008-fall (2008.0) so it's not explusive to Ubuntu (Probably no one imagines that it was, but I'll document that fact right here.)
I said,

- - - quote form Mozillazine - - - -

"This seems to be a 100% Firefox-Created issue, although increased A/V activity may be a side effect of the FF misbehavior on your box.

"Many times when I start running FF3, my hard drive will busy for a long time doing something. Usually it's shortly after starting FF3, but sometimes it goes wild in the middle of a long session. CPU does go 100%-- and my system has no virus scanner or add-on malware monitor at all, modern Linux doesn't need one in my kind of usage. (The entire Firefox installation is within a USER account, and nothing has access anywhere in a system context.) Within Firefox, the panel becomes totally unresponsive. I can start other applications, because Linux is a bit more "fair" about allowing other applications (like my Window Manager) to get a peice of the box when I do something-- but inside Firefox, it's totally locked up and focused on this disk-thrashing task for many, many seconds.

Lars, are you using a really old and much-upgraded Profile? (I am, I've got about a zillion sets of "saved form" data and don't know how to migrate this info to a freshly-built profile.)

I did get rid of urlcalssifier2.sqlite a couple of weeks ago, but the current urlclassifier3.sqlite has already grown to considerable size. I don't know if FF might be churning the whole file, in tiny random-access reads, to "organize it". If so, that needs to be STOPPED. There's also a tiny chance that it's the new memory model doing some garbage collection, but I doubt this, because it happens right at startup a lot, before any memory bits have been allocated. Also, I have lots of RAM, and the Firefox task can get all the "real" memory it wants to have. Here's my current sqlite file sizes:

-rw-r--r-- 1 rick rick 1082368 2006-04-30 00:00 bookmarks_history.sqlite
-rw-r--r-- 1 rick rick 7168 2008-03-24 01:37 content-prefs.sqlite
-rw-r--r-- 1 rick rick 193536 2008-04-28 10:53 cookies.sqlite
-rw-r--r-- 1 rick rick 12288 2008-04-28 10:28 downloads.sqlite
-rw-r--r-- 1 rick rick 240640 2008-04-28 10:39 formhistory.sqlite
-rw-r--r-- 1 rick rick 5120 2007-12-12 12:16 permissions.sqlite
-rw-r--r-- 1 rick rick 4685824 2008-04-28 10:53 places.sqlite
-rw-r--r-- 1 rick rick 2048 2007-10-16 01:11 search.sqlite
-rw-r--r-- 1 rick rick 32229376 2008-04-28 10:40 urlclassifier3.sqlite
-rw-r--r-- 1 rick rick 2048 2008-04-18 09:37 webappsstore.sqlite

I'm currently running a 5-day old version,
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9pre) Gecko/2008042320 Minefield/3.0pre - Build ID: 2008042320

I've no idea what it's doing, and haven't opened a bug because I've no idea how to describe it adequately-- this was pretty much everything I know."

- - - - end mozillazine quote - - - - -

But my "Firefox locked up, disk thrashing" Time periods often exceed 20 seconds, and lots of people have less capable boxes than I do (disk has 8mb cache, lots of people still have just 2mb). I agree with comment 9...

[Rickstockton (rickstockton)]

(In reply to comment #5)
> ... Once we've caught up to the state of the server, our updates are going
> to be much smaller - usually in the low hundreds or occasionally a thousand
> or two. This will touch many fewer pages in the database (I haven't measured > this closely)....
>

Dave, I may be mis-understanding this part of your assessment. But, like SuperNova in comment 20, I run Firefox many times per day, hundreds of times per week-- and I am not experiencing small, incremental updates. Instead, it seems that I'm getting "tons-of-megabytes" disk I/O incidents, day after day after day-- even on days when I haven't downloaded a new Build, although it seems to me that the Databases in my profile shouldn't need to be completely overwritten even if I *did* doi a new Nightly every night.

[Jeffrey Baker (jwbaker)]

I have to agree with Rick Stockton. It would be a mistake to characterize this bug as some kind of ramp-up cost for new users only. I use Firefox continuously on my laptop and I also see this bug pretty much all day long. You can see from my strace that Firefox had written more than 30MB within a few minutes of starting, and I can reproduce that behavior at will.

From my very little experience with sqlite, my impression of this bug is that either we're doing full database writes when we think we're doing incremental updates, or we're doing full database vacuums when we wanted incremental vacuum.

Aside from that, what stands out to me in the strace log is that it looks like the sqlite database is initialized with 1KiB pages. It might be a lot more efficient if it used the same page size as my filesystem (4KiB). This could be queried before initializing the database, or if that's too complicated then 4KiB is a far better default on modern Linux systems anyway.

[PSa (psa000)]

I just wanted to let you know that the problem occurs in Win XP Pro SP2 Swedish as well and has been doing since beta 1 & beta 3 & beta 5.

It has now consumed 17 CPU minutes since I started Firefox about an hour ago, an mostly surfing this thread (Some googling to find the thread), and Yes It is reading/writing the urlclssifier files all the time.

(It has done 6 million read requests & 4 million write request, having read 6GB & written 4GB of data, in the last hour)

I will stop using this version going back to 2.14 for a while hoping that the problem will be solved.

[Gearoid-murphy (gearoid-murphy)]

as far as I can deduce from these comments, this bug seems to be associated with beta 5 alone, is this correct?, if so, what exactly is different about the underlying implementation between beta 4 and 5?

[Dcamp (dcamp)]

Created attachment 318863
increase the page size, let the cache size grow on linux

The attached patch lets the cache grow as big as it needs to (up to a limit of 100 megs, as a sanity check) on linux. The cache is freed when the update is done. It also fixes the page size to be 4096 (this is actually a regression from the schema versioning change awhile back).

To pick up the pagesize change we need to reset the DB, hence the IMPLEMENTATION_VERSION bump.

I'm not sure this is the best or final fix, but I'd like to get feedback on it, so I think we should put it in.

[revelationman (brianwmarto-gmail)]

I think the best thing is to go back to 7.10 and wait till they sort this crap out
I mean why in the heck would they place a beta browser in this beats the bloody life out of me

[Pjotr12345 (computertip)]

There is an easy workaround.

This workaround is successful:
1. Open Firefox 3
2. Disable phishing/attack detection in the security section of the Preferences
3. Close Firefox
4. Make hidden files visible (Locations - Personal folder - View - show hidden files)
5. Delete /home/user/.mozilla/firefox/blahblahblah.default/urlclassifier*.sqlite
6. Open Firefox.
7. There is no next step. You're done. :-)

Greetz, Pjotr.

[Mike Connor (mconnor)]

Comment on attachment 318863
increase the page size, let the cache size grow on linux

>+#ifdef XP_WIN
>+pref("urlclassifier.updatecachemax", -1);
>+#elifdef XP_MACOSX
>+pref("urlclassifier.updatecachemax", -1);
>+#else
>+pref("urlclassifier.updatecachemax", 100 * 1024 * 1024);
>+#endif

you just want to do ifdef MOZ_WIDGET_GTK2 here, I don't think we want this on OS/2 or beos, etc.

> // The implementation version is updated during development when we
> // want to change schema, or to recover from updating bugs. When an
> // implementation version change is detected, the database is scrapped
> // and we start over.
>-#define IMPLEMENTATION_VERSION 3
>+#define IMPLEMENTATION_VERSION 4

as an aside, this is really bloody convenient. :)

>+ if (gUpdateCacheSize > 0) {
>+ PRUint32 cachePages = gUpdateCacheSize / PAGE_SIZE;
>+ nsCAutoString cacheSizePragma("PRAGMA cache_size=");
>+ cacheSizePragma.AppendInt(cachePages);
>+ rv = mConnection->ExecuteSimpleSQL(cacheSizePragma);
>+ if (NS_FAILED(rv)) {
>+ mUpdateStatus = rv;
>+ return rv;
>+ }
>+ mGrewCache = PR_TRUE;
>+ }

>+ nsCAutoString cacheSizePragma("PRAGMA page_size=");
>+ cacheSizePragma.AppendInt(PAGE_SIZE);

ITYM pageSizePragma

looks good, r=me

[Beltzner (beltzner)]

Comment on attachment 318863
increase the page size, let the cache size grow on linux

a1.9=beltzner

[Dara Adib (daradib)]

revelationman: You can install the package firefox-2 in Ubuntu 8.04 if you want Firefox 2.

[Rickstockton (rickstockton)]

Thanks DC, MC, MB! I'll keep my eyes on tinderbox and start running the Linux Smoketest as soon as I see this Update in there. (Hoping there's a little bit of a testing window before Google's bug # 402469 changes are implemented, in case we need to figure out which change is to "get the blame"... if the combined effect is no-worky.)

[Dcamp (dcamp)]

Created attachment 318939
updated patch

Updated patch fixes review comments and pref file syntax (can't actually do multiplication in the pref file!).

[Reed Loden (reed)]

Checking in browser/app/profile/firefox.js;
/cvsroot/mozilla/browser/app/profile/firefox.js,v <-- firefox.js
new revision: 1.328; previous revision: 1.327
done
Checking in toolkit/components/url-classifier/src/nsUrlClassifierDBService.cpp;
/cvsroot/mozilla/toolkit/components/url-classifier/src/nsUrlClassifierDBService.cpp,v <-- nsUrlClassifierDBService.cpp
new revision: 1.74; previous revision: 1.73
done

[Reed Loden (reed)]

Comment on attachment 318939
updated patch

>+#ifdef MOZ_WIDGET_GTK2

I changed this to |#ifdef UNIX_BUT_NOT_MAC|.

[Steffen Wilberg (steffen-wilberg)]

> >+#ifdef MOZ_WIDGET_GTK2
>
> I changed this to |#ifdef UNIX_BUT_NOT_MAC|.
You didn't:
http://bonsai.mozilla.org/cvsview2.cgi?subdir=mozilla/browser/app/profile&command=DIFF_FRAMESET&file=firefox.js&rev1=1.327&rev2=1.328

[les marques (lasmarkas)]

flag "resolved fixed" on bugzilla :
https://bugzilla.mozilla.org/show_bug.cgi?id=430530

[Gavin Sharp (gavin-sharp)]

Good thing, too, because UNIX_BUT_NOT_MAC would have been the same as the original patch.

[Rickstockton (rickstockton)]

EEEK! Gavin, Steffen: I'm no expert at this, but Tinderbox shows Reed's final version as rev 1.329 (applied a minute AFTER the big update rev 1.328), and the bonsai diff shows this change applied exactly has he said:
http://bonsai.mozilla.org/cvsview2.cgi?subdir=mozilla/browser/app/profile&command=DIFF_FRAMESET&file=firefox.js&rev1=1.328&rev2=1.329

And rev 1.329 is the last, so it's in there as Reed says. Maybe it's a good thing, maybe not-- better ask Reed why he didn't agree with comment 26. But it looks like he made this change very, VERY intentionally.

- - - - -
I'd been waiting for this to land and try it out before the Google folk landed their changes for 402469, but I gave up at about 2AM. So I'm pulling the nightly right now, a little "end user's test report" (how long to update the file) coming in my next comment. My current profile is a very, very old one-- migrated all the way from about Mozilla 0.8, but it has been following FF3 for a long time. I'll leave the current urlclassifier3.sqlite intact upon restart.

And yeah, my profile *IS* getting backed up first ;)

[Rickstockton (rickstockton)]

shocking, great, absolutely awesome:

my urlclassifier3.sqlite file auto-magically changed in size from 9.8MB to 17.6MB, without ANY apparent hard drive thrashing at all. But I may not have been using Firefox actively at that moment-- I'll be sure to play again, more acrtively, opening tabs and typing into text boxes, when the "stale" timer expires again in a little less than 45 mins.

[Rickstockton (rickstockton)]

apology: as you can guess, my "active usage" involved reloading tabs-- and I collided with myself, #36 == #35. But I see something unexpected happened: I was hoping to see the update occur at 12:58 + 2700 seconds = 13:43 (that's why I starteed playing at 13:41, but instead it updated (adding another 200KB) earlier: at 13:31, per my local PC time.

So I missed it. grrrr. I'll try playing again, from 14:00 and continuing until I see the file updated.

THEN: I'll download FF3 nightly into a Windoze box which has either FF2 or no current Firefox at all, see what happens with that "migration" or creation of a new FF3 profile.

[Gavin Sharp (gavin-sharp)]

(In reply to comment #34)
> EEEK! Gavin, Steffen: I'm no expert at this, but Tinderbox shows Reed's final
> version as rev 1.329 (applied a minute AFTER the big update rev 1.328), and the
> bonsai diff shows this change applied exactly has he said:

"Eek" indeed. I undid that:
mozilla/browser/app/profile/firefox.js 1.330

[Rickstockton (rickstockton)]

OK, I caught it. And WOW, that was really hard to even notice-- right in the middle of reloading a bunch of tabs (including Acid3, which loads a bit slowly, and Tinderbox, which loads REALLY slow). I heard a quick disk I/O-- less than 1/2 second (I'll SWAG between 0.2 and 0.3 seconds). And it was done! added another 300K, now 20.1 MB.

slightly off-topic: how big is this file gonna get?

I'll proceed to start up a Windows-XP machine, install this morning's nightly, and report results.

[Rickstockton (rickstockton)]

Downloaded 2.0.0.14 and installed it, creating a new default profile. The upgraded to Tinderbox windows build. (Downloaded 14:21, which might have been too early for Gavin's update-- but it should certainly have no effect on Windows, which should have failed either test very decisively.)

Initial profile creation (under 2.x) took a while, but 3.0 Tinderbox showed no issues when used for only a few minutes. HOWEVER, "no issues" may have been only because I didn't keep it running for long enough- I've got a urlclassifierkey3.txt, but no urlclassifier3.sqlite file. Back to Windoze for a while...

[Launchpad Janitor (janitor)]

This bug was fixed in the package xulrunner-1.9 - 1.9~b5+nobinonly-0ubuntu4

---------------
xulrunner-1.9 (1.9~b5+nobinonly-0ubuntu4) intrepid; urgency=low

  * fix LP: #215728 - "excess disk IO when updating the url-classifier"; we
    increase cache size of sqlite database and better align page size.
    Cherry-picking and backporting upstream fix from bmo#430530;
    other changes vs. upstream: we move default pref for cache size from
    browser/ to toolkit/
    - add debian/patches/bz430530_attachment_318939.patch
    - update debian/patches/series
  * don't use gcc-4.2/g++-4.2 and dont depend on that package accordingly
    - update debian/rules
    - update debian/control

 -- Alexander Sack <email address hidden> Sat, 03 May 2008 01:08:17 +0200

[Rickstockton (rickstockton)]

PROBLEM ON WINDOWS (I think).

So I went back to Windows and ran for a long time (a full hour), aggressively switching between and reloading tabs during periods when I anticipated that FF3 might start updating the urlclassifier_x.sqlite file. I ran for over an hour, but

urlclassifier3.sqlite was never created.

urlclassifier2.sqlite was never updated (left unchanged from it's State with Firefox2).

I can't read code reliably, but I'm under the impression that phishing protection (using current Trunk code) should ALWAYS create a urlclassifier3.sqlite file, in the root of the profile, on all platforms. And update should have happened in no more than 45 minutes. (Although I gave it an entire hour, looking for something to happen to either of the urlclassifier_x files).

So I suspect we've introduced a bug-- somehow, a Windows machine doesn't set the timer correctly, or doesn't initiate the timer listener properly.

New bug? The Linux fix worked great.

[Steffen Wilberg (steffen-wilberg)]

Sorry, should've looked better before posting.

[unggnu (unggnu)]

Btw. could anyone of the kernel devs answer why a simple disk operation like this one uses 100% kernel cpu? I have made a bug report some time ago but the devs neglected it. It often happens on simple copy operation and similar things.

[Alexander Sack (asac)]

test package with cherry-picked patch from upstream available in intrepid. please test.

[Alexander Sack (asac)]

preparing and uploading hardy preview package of this cherry pick to mozillateam PPA: https://edge.launchpad.net/~mozillateam/+archive

The version that will be upped in an hour or so is: 1.9~b5+nobinonly-0ubuntu4~8.04.0mt1

Please test.

[Beltzner (beltzner)]

This will not block the final release of Firefox 3. Any patch will need unit tests in order to be approved.

[Beltzner (beltzner)]

(sorry for spam - finger slipped and hit my bookmarklet ;)

[Pjotr12345 (computertip)]

@Alexander Sack:
If I understand you correctly, you would like us to test a possible fix for this bug. If so, could you provide us with direct links to the packages in question (32-bit and 64-bit)? I can't find my way on the link page that you gave.

By the way: why do you think that xulrunner is the culprit? On my Intel Core 2 Duo machine, running 32 bit Ubuntu, the problem wasn't solved after I removed both xulrunner Add-ons (british and dutch). So how can this bug be related to xulrunner?

The only thing that helped was the workaround that I posted a few posts higher up in this thread. That succesful workaround suggests that the problem lies in Firefox 3 itself, not in the add-on xulrunner.

Greetz, Pjotr.

[Chris Coulson (chrisccoulson)]

Pjotr12345: The link is a PPA and contains the fixed xulrunner packages. You just add the PPA to your /etc/apt/sources.list and install the update through the package manager.

And to clarify, xulrunner isn't a Firefox add-on. xulrunner is the backend to Firefox - without it, it won't work at all.

[Pjotr12345 (computertip)]

@Alexander Sack and Chris Coulson:

I get this error message from the system when I "apt-get update" with the addition of the line "https://edge.launchpad.net/~mozillateam/+archive" in muy sources list:
@thuiscomputer:~$ sudo apt-get update
[sudo] password for ....:
E: Type 'https://edge.launchpad.net/~mozillateam/+archive' on line 48 in sources list /etc/apt/sources.list is unknown

Please make it a little easier for me to help you with testing. I consider myself to be an experienced Linux user, but this is over my head.

[Sander (sanderth)]

Pjotr: try to add the following line to your /etc/apt/sources.list:
deb http://ppa.launchpad.net/mozillateam/ubuntu hardy main

Good luck!

[Pjotr12345 (computertip)]

OK. Updates applied and testing started. I will report back in a day or so.

Bedankt, Sander! :-)

[Pjotr12345 (computertip)]

Intermediate report, in view of the critical importance of this bug (show stopper).

After more than an hour of intensive surfing, with the warning for attack sites turned on (default configuration): no more problems. :-)

However, urlclassifier3.sqlite has grown from 0,009 MB to 9,6 MB. Is this rate of increase normal? If so, how can the growth of this file be limited?

[Pjotr12345 (computertip)]

I think it can be considered as fixed. A couple of hours more of intensive browsing and no problems at all. :-)

Please roll out this update very quickly. It's extremely damaging for the reputation of Ubuntu, especially for an LTS-version.

One remark: urlclassifier3.sqlite has grown to 18,8 MB. This worries me: how much more will it grow? And how can I stop it from growing to a huge size, apart from disabling the attack website warnings?

[Sander (sanderth)]

Pjotr, that file contains information about malicious websites, and I think it's intended to become quite big. It however should stop growing at some point, when all information has been downloaded.

[Rickstockton (rickstockton)]

(In reply to comment #41)
> PROBLEM ON WINDOWS (I think).

NO PROBLEM EXISTS. I don't know why my experience was different yesterday, but I asked for 'nightly' Windows users at the Mozillazine "daily build" forum Thread to tell me what they saw on their boxen... and urlcalssifier3.sqlite *is* being updated properly, for both XP and Vista users (WildCatRay and LittleMutt, they're much more expert than I am at FireFox bug testing.) So we're all done here, and all done with 402469 :)

When RC1 is created, I'll update my wife's *PRODUCTION* FF2 system and double-check to see what occurs there. She's on Windows exclusively.

My file size is now 44.8MB

[Alexander Sack (asac)]

initiating SRU process.

[Alexander Sack (asac)]

parts of the upstream patch touch firefox 3, but for hardy we move the pref from browser/ to toolkit/ to prevent that both packages need to be updated.

[Martin Pitt (pitti)]

Accepted into hardy-proposed, please test and give feedback here.

[Alexander Sack (asac)]

Test instructions: Go to System -> Administration -> Software Sources and enable "hardy-proposed" in the "Updates" tab. Next, upgrade your system and once you got xulrunner-1.9 version 1.9~b5+nobinonly-0ubuntu4~8.04.0mt1, verify that this problem is gone.

Thanks for testing!

 - Alexander

[Daniel T Chen (crimsun)]

To test thoroughly, do we also need a pristine FF3.0b5 environment (e.g., new user, or along the lines of rm -r ~/.mozilla, so that an existing urlclassifier cache is not in play)?

[Rocco (rocco)]

This fix didn't fix the problem I'm seeing in https://bugs.launchpad.net/bugs/224320, which is marked as a dup of this bug.

Removed .mozilla and I still see the same problem as reported at 224320, after install of proposed xulrunner-1.9

[Martin Kaffanke (martin-kaffanke)]

Since upgrade to the xulrunner from hardy-proposed, my firefox crashes after watching videos on www.autsch.de and www.clipfish.de - I can watch one movie, the second crashes firefox.

[jeroenl (jeroenl)]

I didn't experienced any high disk I/O and high CPU since upgrading to the mt-version.

@Rocco and @Martin: Are you sure you closed/killed all firefox processes after upgrading?

@Martin: I can't confirm any crashes on those sites with those flash videos.

[Pjotr12345 (computertip)]

Still testing (over a day now) and still no problems. This update really works. Good job! :-)

[Alexander Sack (asac)]

Rocco, I unduped your bug. You are seeing something else as you are sure that you experience no high IO load.

[Alexander Sack (asac)]

Daniel, afaict you don't need to remove any existing urlclassifier database.

[R (Chandra) Chandrasekhar (chandra)]

I have added to /etc/apt/sources.list the following two lines:

deb http://ppa.launchpad.net/mozillateam/ubuntu hardy main
deb-src http://ppa.launchpad.net/mozillateam/ubuntu hardy main

The currently installed packages are:

firefox-3.0_3.0~b5+nobinonly-0ubuntu3_i386.deb
xulrunner-1.9_1.9~b5+nobinonly-0ubuntu4~8.04.0mt1_i386.deb

The previously encountered problems have not recurred.

I have one question, though. Having ticked both the "suspected attack site" and "suspected forgery site" boxes, I find that I now have only

urlclassifier3.sqlite

in my .mozilla/firefox/<blah> directory, and it is growing at a modest rate. But there is no

urlclassifier2.sqlite

which was one of the files I was advised to delete previously.

Does this have any significance?

Thanks.

[f3a97 (f3a97)]

Hi,
       I upgraded the xulrunner too, up to now the behaviour is ok!

Thanks for the fix.

When it will be available as a standard distribution package?

[Martin Pitt (pitti)]

stek79. The packages are currently in hardy-proposed and should be tested from there (not from the PPA). Once we collected enough test results, they will go into hardy-updates, where all users will get them automatically.

[Alexander Sack (asac)]

yes, please remove the PPA lines from your sources.list ... enable -proposed in System -> Administration -> Software sources ... and run

  sudo apt-get update; sudo apt-get install --reinstall xulrunner-1.9

Then test and submit your test-results.

Thanks!

[Colin Watson (cjwatson)]

This is all rather subjective, as I didn't time it beforehand, but Firefox feels a lot more responsive after this fix, particularly just after entering a URL and when switching tabs.

[Benjamín Valero Espinosa (benjavalero)]

I am using the proposed update and all works fine now (at least till this moment). By the way, I agree Colin, I feel Firefox more responsive. Good work!

[Daniel T Chen (crimsun)]

Using xulrunner-1.9 from hardy-proposed, verified that symptoms are no longer present using:
1) existing user with existing FF 2.0.0.14 profile (gutsy-security) migrated to FF 3.0b5 profile (hardy-proposed);
2) existing user with existing FF 3.0b5 profile (hardy-proposed);
3) new user with new FF 3.0b5 profile (hardy-proposed).

[Martin Pitt (pitti)]

For the record, I have used the new xulrunner for over a day here and didn't notice any regressions. I don't use any fancy extensions, though (just AdBlock plus), and the proprietary flash.

[Pjotr12345 (computertip)]

Today (10 hours ago), I repeated the testing with xulrunner in proposed, instead of xulrunner from upstream (after applying Alexander Sack's instruction: https://bugs.launchpad.net/ubuntu/+bug/215728/comments/94

Again, no problems after a day of intensive web surfing.

*Please* don't wait any longer with spreading this update through the normal channels. This bug is so bad, that reparation is extremely urgent.

[Chris Coulson (chrisccoulson)]

I can't test the version in hardy-proposed, because I can find a mirror that's up-to-date enough, and the connection to Main is patchy at best

[Samuel Lidén Borell (samuellb)]

I can confirm that this update doesn't introduce any regressions (the urlclassifier files are updated, web sites I use to visit work, etc). I've tested on a computer (and Firefox profile) upgraded from Ubuntu 5.04->5.10->6.06->8.04. I did some quick testing with a fresh profile and it seemed to work well too.

[Moshewe (orelweinstock)]

Same here.

[Jruderman (jruderman)]

*** Bug 432444 has been marked as a duplicate of this bug. ***

[Gabriel (gabriel-nobody)]

same problem , same behaviour here. Laptop goes unusable.
I went back to firefox 2 for now.

[R (Chandra) Chandrasekhar (chandra)]

No problems, but two questions.

Before the workaround, I had two files:

4970496 2008-04-26 23:42 urlclassifier2.sqlite
23249920 2008-04-27 17:01 urlclassifier3.sqlite

Now that I have installed xulrunner-1.9 version 1.9~b5+nobinonly-0ubuntu4~8.04.0mt1 from hardy-proposed, I do not have the file

urlclassifier2.sqlite

I only have the file urlclassifier3.sqlite so:

33275904 2008-05-07 10:25 urlclassifier3.sqlite

I have enabled both suspected attack sites and suspected forgery sites.

1. Is this absence of urlclassifier2.sqlite normal behaviour?

2. What does it imply?

Thanks.

[Martin Pitt (pitti)]

Verification looks good so far, marking verification-done. I'll copy this once Alex answers to the previous question and it is not a problem.

[Alexander Sack (asac)]

urlclassifier2.sqlite is the old filename and is not used anymore.

The schema/filename policy reads like this:

// The database filename is updated when there is an incompatible
// schema change and we expect both implementations to continue
// accessing the same database (such as between stable versions of the
// platform).
#define DATABASE_FILENAME "urlclassifier3.sqlite"

// The implementation version is updated during development when we
// want to change schema, or to recover from updating bugs. When an
// implementation version change is detected, the database is scrapped
// and we start over.
#define IMPLEMENTATION_VERSION 4

This update only bumped IMPLEMENTATION_VERSION.

[Martin Pitt (pitti)]

Copied to hardy-updates.

[dave87 (david-roth)]

Same Problem with my laptop here. Extremely hard-disk usage with firefox 3 Beta5 and hardy heron makes laptop very very slow.
not very nice ...

[Dan Andreșan (danyer)]

dave87, look at the post above yours (posted 11 hours ago).

Fix was released, use System|Administration|Update Manager to download the latest updates. It will be two updates xulrunner-something.

Enjoy,
Dan.

[Tormod Volden (tormodvolden)]

I installed the 1.9~b5+nobinonly-0ubuntu4~8.04.0mt1 from comment 69, but still had this problem. I have now reinstalled from hardy-updates. I see the version name is the same, but can I expect it to work better now?

[Alexander Sack (asac)]

On Wed, May 07, 2008 at 08:45:30PM -0000, Tormod Volden wrote:
> I installed the 1.9~b5+nobinonly-0ubuntu4~8.04.0mt1 from comment 69, but
> still had this problem. I have now reinstalled from hardy-updates. I see
> the version name is the same, but can I expect it to work better now?
>

No, you are most likely seeing an unrelated issue (most likely a
rendering operation or javascript consuming high CPU ... or flash).

 - Alexander

[Tormod Volden (tormodvolden)]

I should have been more clear: I mostly saw high i/o load, not CPU usage, and the .sqlite file was ~30MB. This came several days after installing your package (and deleting the old .sqlite files). Now things run fine with the official update, but I wonder if it can kick in again after some time.

[kulight (kulight)]

I'm also still getting excessive disk I/O after the update.
fire-fox grinds the disk while surfing

[f3a97 (f3a97)]

Are you sure it is Firefox to cause that IO load? Have you measured it? If you want I have a Systemtap script useful to find the IO bound process.

With the proposed fix I've never experienced the issue again.

[kulight (kulight)]

The disk I/O is not as bad or as frequent as before. But it's still there, i have no idea how to measure it or where it comes from if you have a script or instructions on how to do this (measuring) i'll be happy to do that and report.

[unggnu (unggnu)]

I can confirm this. The main problem is gone but it seems partly shifted. Every half an hour or something like that Firefox and nearly the whole system freezes (at least all disk tasks) for around ten seconds. My laptop hard disk isn't so fast so I guess Firefox syncs the the database everytime when it happens. It is now btw. 50 MB. If I disable the options the problem disappears.
The kernel should better be able to scale/queue intensive hard disk ios so not one task is able to halt the whole system (Bug #131094).

[f3a97 (f3a97)]

Hi guys,
     two useful methods to measure IO by process are the following, based on the great SystemTap linux kernel tool.

The disktop.stp script, which can be found attached here, measure the first N processes odered by IO activity - a 'top' for disk resource, we could say. I've found it on the net, it seems to be made by Oracle and it is the porting to SystemTap of the equivalent Dtrace script.

Once found the culprit process, we can also see the response time of each IO request that the process make with the iotime.stp script, which is provided with the systemtap package (/usr/share/doc/systemtap/examples/iotime.stp)

I hope you can find them useful. Let us know if firefox is actually the process that hurts your performance. I've noticed that in Ubuntu there are also other very high disk demanding tool, for example the scrollkeeper-update or the slocate db updater.

I think it should be paid more attention to the disk resource, often its impact on performance is overlooked but actually it is important. If the culprit process is a background tool, you can lower its IO priority with ionice - thanks to the linux fair CFQ scheduler.

Let us know what you find!

[Nick B. (futurepilot)]

Yes, there is still some heavy disk I/O going on. It's no where near as bad as it was before the update. This patch is a HUGE improvement, but it still happens from time to time causing Firefox to become unresponsive for a few seconds.

[Tormod Volden (tormodvolden)]

The easiest way to see which processes are doing i/o is to use iotop, there are packages in my PPA.

[f3a97 (f3a97)]

Hi,
   actually I've just measured some firefox heavy IO activity, but as Nick reported, it is a matter of seconds and in my case it doesn't make the whole system irresponsive as it was before the patch.

Luckily I had disktop running, so here is the output. I can say that the the slowdown that I felt was probably due to the large write of 5.5MB in 5 seconds you can see at 18:28:31 - some random IO activity I guess.

Sun May 11 18:28:11 2008 , Average: 657Kb/sec, Read: 2642Kb, Write: 647Kb

     UID PID PPID CMD DEVICE T BYTES
    1000 5880 1 firefox sda3 R 2605104
    1000 5880 1 firefox sda3 W 646660
    1000 5875 1 transmission sdb2 R 98304
    1000 5875 1 transmission sdb2 W 16384
    1000 5807 1 multiload-apple sda3 R 2049

Sun May 11 18:28:16 2008 , Average:1031Kb/sec, Read: 2965Kb, Write: 2194Kb

     UID PID PPID CMD DEVICE T BYTES
    1000 5880 1 firefox sda3 R 2936832
    1000 5880 1 firefox sda3 W 2246792
    1000 5875 1 transmission sdb2 R 98304
    1000 5807 1 multiload-apple sda3 R 1366

Sun May 11 18:28:21 2008 , Average: 856Kb/sec, Read: 2842Kb, Write: 1438Kb

     UID PID PPID CMD DEVICE T BYTES
    1000 5880 1 firefox sda3 R 2826240
    1000 5880 1 firefox sda3 W 1440504
    1000 5875 1 transmission sdb2 R 81920
    1000 5875 1 transmission sdb2 W 32768
    1000 5807 1 multiload-apple sda3 R 2049

Sun May 11 18:28:26 2008 , Average: 607Kb/sec, Read: 2089Kb, Write: 948Kb

     UID PID PPID CMD DEVICE T BYTES
    1000 5880 1 firefox sda3 R 2023424
    1000 5880 1 firefox sda3 W 971588
    1000 5875 1 transmission sdb2 R 114688
    1000 5807 1 multiload-apple sda3 R 1366
       0 5331 1 preload sda3 W 131

Sun May 11 18:28:31 2008 , Average:1144Kb/sec, Read: 254Kb, Write: 5468Kb

     UID PID PPID CMD DEVICE T BYTES
    1000 5880 1 firefox sda3 W 5599737
    1000 5880 1 firefox sda3 R 131204
    1000 5875 1 transmission sdb2 R 114688
    1000 6157 5970 iostat sda3 R 11652
    1000 5807 1 multiload-apple sda3 R 2049
    1000 6157 5970 ba...

[Pjotr12345 (computertip)]

I can confirm both the improvement because of the patch, and the remaining hugely diminished problem.

Oh well, I have simply disabled the warning for attack websites. Who cares anyway, all they attack is Windows machines..... :-)

[Alexander Sack (asac)]

ok, i have cut out a new bug for the issues left:

https://bugs.edge.launchpad.net/ubuntu/+source/firefox/+bug/229745:
 #229745 - after fix for #215728 - Committing to urlclassifier3.sqlite still causes excessive CPU usage and disk I/O (the 2nd)

if you want to follow there, feel free to subscribe to that bug.

[Tormod Volden (tormodvolden)]

(Just write bug #229745 and you get a correct bug link)