With Bindwood installed, Firefox is completely unresponsive

Bug #443121 reported by Anton Kraus on 2009-10-05
286
This bug affects 55 people
Affects Status Importance Assigned to Milestone
Bindwood
High
Zachery Bir
bindwood (Ubuntu)
Medium
Zachery Bir
Karmic
Medium
Zachery Bir

Bug Description

Fix can be found in Bindwood PPA: https://answers.edge.launchpad.net/bindwood/+faq/859

Binary package hint: bindwood

I can reliably reproduce that, with Bindwood installed, Firefox does not react to any input.
Clicks on the menus lead to no reaction. All I can do is close the browser window.
Upon removing Bindwood, everything works fine again.

During these freezes, the processes firefox and beam.smp consume all available CPU.

When started in a terminal, Firefox shows no output. There's also nothing in the system log.

My bookmarks are arranged within multiple folders and subfolders. Could this be a problem?

ProblemType: Bug
Architecture: i386
Date: Mon Oct 5 15:54:57 2009
DistroRelease: Ubuntu 9.10
NonfreeKernelModules: nvidia
Package: bindwood (not installed)
ProcEnviron:
 LANGUAGE=en_GB.UTF-8
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-10.35-generic
SourcePackage: bindwood
Uname: Linux 2.6.31-10-generic i686

Related branches

Anton Kraus (done) wrote :
Zachery Bir (urbanape) wrote :

Anton,

Thanks for taking the time to file a bug. We've added some diagnostic debugging that can be enabled if FF is run with BINDWOOD_DEBUG in the process' environment. The easiest way is to run from the terminal (like you tried):

  $ BINDWOOD_DEBUG=1 firefox &

This will output a lot of debugging messages to the error console. The default FF error console only allows you to copy one message at a time, so you could install the Console2 extension ( https://addons.mozilla.org/en-US/firefox/addon/1815 ) to enable copying multiple lines.

Bindwood iterates over all your bookmarks, pushing each to your desktop couchdb (after first checking that it's not already present in the db). During testing and development, it's likely that we didn't provide our own instances with enough bookmark data to really stress test the setup. Off hand, do you have an idea of the number of bookmarks you have?

In the future, Bindwood will do a much better job after the first run, of keeping its network traffic to a bare minimum. As it stands now, it's a little too chatty.

Changed in bindwood (Ubuntu):
assignee: nobody → Zachery Bir (urbanape)
status: New → Confirmed
Changed in bindwood:
status: New → Triaged
Anton Kraus (done) wrote :

Thanks for the quick reply!

The amount of bookmarks indeed seems to be part of the problem, yet I also discovered something else.

First some information:
I have 78 bookmarks in total, in a hierarchy that is two subfolders deep. The actual json-bookmark file exported by Firefox is 33KB in size.
My Internet connection is not very fast (max. 15KB/s upload).
I waited 15 minutes to see whether the initial sync would eventually complete, but this was not the case and Firefox was still unusable after all the time.

So I deleted all bookmarks. With nothing to sync, the browser now was usable from the start.
Then I restored my bookmarks, by loading a backup file from within Firefox.
Interestingly, this time the browser remained usable, beam.smp and firefox did not consume 100% CPU, and the sync began, at a rate of about 1 bookmark per minute.
There were no errors and warnings in the error console, only regular messages about the ongoing transfer. It would probably have completed successfully.

But out of curiosity I did not wait that long. Instead I restarted Firefox, with the result that, once again, CPU load spiked and the restarted browser froze.

In short:
-Start Firefox+Bindwood with existing bookmarks --> freeze, high CPU load
-Start Firefox+Bindwood without bookmarks, restore them from file --> regular sync begins

I also tried to get debug information according to your instructions. But with the browser frozen I was unable to access the error console.

tags: added: amd64
Anton Kraus (done) wrote :

This bug has disappeared after deleting all items labeled "Desktop Couch user authentication" in seahorse.
Firefox now responds to input. :-)

Well, my experience is that if you wait long enough (tens of minutes) firefox will eventually be responsive again. Of course, it's completely unsatisfactory to have to wait tens of minutes after starting a browser for it to be useful.

Any thoughts on why it's so slow to start up?

Anton Kraus (done) wrote :

A bit more information: I couldn't reproduce this bug on another machine with a relatively fresh Karmic install and the same set of bookmarks.

The PC with the freeze bug has been running development releases since Hardy, so maybe something broke along the way.

Kev (ukev) wrote :

Hi,

I could reproduce this bug with a fresh karmic beta install + all updates.
But on my computer it consumes not only cpu, but also all memory (8 Gig) and take the swap, after 3 Gig of swap I killed firefox.
After disabling the bindwood plugin via the safe mode feature, firefox works again.
I have 150-200 bookmarks in 25-30 folders with maximum hierarchy depth of 4.

My internet connection has 100 MBit/s upload.

Kev (ukev) wrote :

I've got a few more informations:

I saved my bookmarks as backup, deleted all my bookmarks in firefox, activated bindwood and restarted firefox. After that bindwood worked, I created one new bookmark and it was synced correct to the desktop couch. After that I imported my backup and it took only 4 seconds to sync all my normal bookmarks to the desktop couch.
It works this way. It created 230 documents in the bookmarks database.

But after I restarted firefox, the browser consumes all cpu and memory again. So the problem is not the performance of bindwood to the desktop couch and to much data alone, the issue has to do more with the way firefox starts up and launchs it.

How can I access the firefox error console if firefox is unresponsive?

Ken VanDine (ken-vandine) wrote :

Bindwood is syncing live bookmarks, which spams desktopcouch on every startup with lots of duplicate entries.

Ken VanDine (ken-vandine) wrote :
Download full text (4.1 KiB)

Here is an excerpt of the raw content from my desktopcouch, these come from the default live bookmark in firefox.

{"id":"1c6d34c81fcb0048575fe523faa32558","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"204830b6140fea0a9f2122f10d194ec8","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"23aad0bf9adbba87fc79d16a0b6aefc3","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"2739083632b8a75477d56bffa4614796","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"404c789f7c65d7cea56e8c4fdf8f128a","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"5a98387397942df68f478dd4a52ed994","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"5fec432a3a41703bf2851aa804d1e6b1","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"749cb9fdd035bd5cf7876dbe7bed5601","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"9986ac59a3b71a061a522e28ea75175a","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"ba51772d47e29ddf3eecba2d971d885b","key":"'Bracelets' useless in arthritis","value":"news.bbc.co.uk/go..."},
{"id":"c9ff5b3c60c48c5ed01ace38482ac25b","key":"'Crash for cash' scam man jailed","value":"news.bbc.co.uk/go..."},
{"id":"ccc3bf62c55e74a8ee79e50d0e45176f","key":"'Crash for cash' scam man jailed","value":"news.bbc.co.uk/go..."},
{"id":"676d5cd1d63d45d624351592860fef85","key":"'Extremists hijack' military name","value":"news.bbc.co.uk/go..."},
{"id":"6a59843851a2019c2700d25d11023c58","key":"'Extremists hijack' military name","value":"news.bbc.co.uk/go..."},
{"id":"6dba534fd64064af38c1c27052c8eb3d","key":"'Extremists hijack' military name","value":"news.bbc.co.uk/go..."},
{"id":"d18dc1f55d21c63820926f753a6a2c5c","key":"'Extremists hijack' military name","value":"news.bbc.co.uk/go..."},
{"id":"dbc5a8f70ede4a7a5a051c0149aef632","key":"'Extremists hijack' military name","value":"news.bbc.co.uk/go..."},
{"id":"30de68d5a47be4fbd72c7c7f1b47bf9a","key":"'Giant' orb web spider discovered","value":"news.bbc.co.uk/go..."},
{"id":"7a9170ae5853ea94823aa095a4e90da0","key":"'Giant' orb web spider discovered","value":"news.bbc.co.uk/go..."},
{"id":"86b7d246d322233d86af4f74074419a8","key":"'Giant' orb web spider discovered","value":"news.bbc.co.uk/go..."},
{"id":"4cb52e26e4693d2858d60264ea516adf","key":"'Hoodies down' call for colleges","value":"news.bbc.co.uk/go..."},
{"id":"d58d343f43242594521211807a9b14f0","key":"'Hoodies down' call for colleges","value":"news.bbc.co.uk/go..."},
{"id":"01f6996b06ac859a9c5e15614114de7d","key":"'Quick test' for airport liquids","value":"news.bbc.co.uk/go..."},
{"id":"04a64e18bf83386a6d4ceee5251c1131","key":"'Quick test' for airport liquids","value":"news.bbc.co.uk/go..."},
{"id":"0e02ac6c983bf000f8bb7b8c4a5f5cd3","key":"'Quick test' for airport liquids","value":"news.bbc.co.uk/go..."},
{"id":"42c41ab69ad13c7730e009be3f0476fa","key":"'Quick test' for airport liquids","value":"news.bbc.co.uk/go..."},
{"id":"4ebd93ba67fae6836426d0a4b7b083df","key":"'Quick test' for airport liquids","valu...

Read more...

Zachery Bir (urbanape) wrote :

I think I see what the problem is and how to correct it. There are means to avoid livemarks folders from the folder iteration, and we haven't been doing that. I'm currently testing a solution to this now, and hope to have a fix shortly.

Changed in bindwood:
status: Triaged → In Progress
Changed in bindwood (Ubuntu):
status: Confirmed → In Progress
Changed in bindwood:
importance: Undecided → Critical
assignee: nobody → Zachery Bir (urbanape)

This is important show case for CouchDB, and will be a popular feature I think. We should move the fix into Karmic if it is feasible.

Changed in bindwood (Ubuntu Karmic):
importance: Undecided → Medium
milestone: none → ubuntu-9.10
Elliot Murphy (statik) on 2009-10-21
tags: added: ubuntuone-karmic
Jorge Castro (jorge) wrote :

I've deleted my live bookmarks folder to see if that stops the grinding but as soon as I launch firefox it seems that bindwood repopulates the browser immediately.

Jorge O. Castro wrote:
> I've deleted my live bookmarks folder to see if that stops the grinding
> but as soon as I launch firefox it seems that bindwood repopulates the
> browser immediately.

Yes, we saw that during testing, too. The branch that fixes this
behavior is almost through review. After it's released there are two
ways forward for those suffering from this:

   1) Move aside your bookmarks Couch database (you'll need to make sure
that desktop couch is stopped), and let Bindwood recreate the bookmarks
database from scratch - it will avoid syncing the livemarks if present)

   2) Manually delete the documents from Couch (preferably after
disabling Bindwood temporarily). Another bug that was fixed in the
branch under testing was that new bookmarks created by Firefox were
stamped as coming from the toolbarFolder by default, even if they were
created (say) in the livemarks folder. Then, when Bindwood pulls those
bookmarks and has to recreate them, they'd end up in the toolbar. Not at
all what anyone really wants.

Sorry for the trouble, folks. It's about to get a whole lot better.

Zac

Zachery Bir (urbanape) on 2009-10-22
Changed in bindwood:
status: In Progress → Fix Committed
Alexander Sack (asac) wrote :

Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading bindwood_0.4.2-0ubuntu1.dsc: done.
  Uploading bindwood_0.4.2.orig.tar.gz: done.
  Uploading bindwood_0.4.2-0ubuntu1.diff.gz: done.
  Uploading bindwood_0.4.2-0ubuntu1_source.changes: done.
Successfully uploaded packages.

Changed in bindwood (Ubuntu Karmic):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package bindwood - 0.4.2-0ubuntu1

---------------
bindwood (0.4.2-0ubuntu1) karmic; urgency=low

  * new bug fix release 0.4.2
    - fix LP: #459068 - Create a proper folder for storing Couchdb/UbuntuOne book
      marks
    - fix LP: #443121 - With Bindwood installed, Firefox is completely unresponsive

 -- Alexander Sack <email address hidden> Mon, 26 Oct 2009 17:26:22 +0100

Changed in bindwood (Ubuntu Karmic):
status: Fix Committed → Fix Released
Peter Harley (pjrharley) wrote :

I'm not convinced that this is fixed. I'm running a fresh install (as of this morning) of karmic rc (bindwood 0.4.2-0ubuntu2), and having bindwood installed means firefox is unresponsive for the first minute or so that it is open. I have ~100 bookmarks.

I'll see if I can attach some debugging output later.

Zachery Bir (urbanape) wrote :

mister_pink wrote:
> I'm not convinced that this is fixed. I'm running a fresh install (as of
> this morning) of karmic rc (bindwood 0.4.2-0ubuntu2), and having
> bindwood installed means firefox is unresponsive for the first minute or
> so that it is open. I have ~100 bookmarks.
>
> I'll see if I can attach some debugging output later.

In its current incarnation, there will be a linear lag at startup,
depending on the number of bookmarks stored in Firefox. At startup, we
iterate over all the local bookmarks and push them to CouchDB if they're
not already present. This is a known limitation, but isn't going to land
before Karmic. We'll be pushing updates to our PPA in the meanwhile
between Karmic and Lucid releases, so please stay tuned if Bindwood is
something you plan on using.

Thanks,

Zac

Jeremy Nickurak (nickurak) wrote :

Yeah, still seeing a big lag on firefox startup.

Is there a fix for this in a PPA that we should be testing? Or should we just disable bindwood until a fix is available?

Phil Krämer (man0riax) wrote :

Same problem with my bookmarks.
Number of bookmarks: 314
Number of newsfeeds: 8
Number of folders: 13
Firefox just freezes after I launch it with bindwood enabled.

Zachery Bir (urbanape) wrote :

Jeremy: I'm working on a new branch that will likely impose a stiff lag penalty on first launch, but will drastically reduce the lag after initial sync. As soon as I can, I'll upload a test package to my PPA, and let everyone here know.

Zachery Bir (urbanape) wrote :

And by "impose a stiff lag penalty", I mean a linear lag penalty based on the number of bookmarks in your profile. Those with more bookmarks will notice a longer lag at first startup, but things should be much faster later on.

Jeremy Nickurak (nickurak) wrote :

So what exactly is the work around? And why is this marked "Fixed" if people are still affected by it?

I see people talking about removing a "Live Bookmarks" folder, but I don't see that anywhere. And I see stuff about removing couchdb stuff, but I don't know anything about that. A detailed explaination of what to do would help those who show up here because they're affected by this.

Zachery: This still seems like a bad solution. Why isn't this just done in the background, so that firefox remains responsive while the bookmark sync is happening?

Zachery Bir (urbanape) wrote :

Jeremy Nickurak wrote:
> So what exactly is the work around? And why is this marked "Fixed" if
> people are still affected by it?

The workaround right now, if it's really crippling, is to disable the
Bindwood extension. It was marked Fixed because the initial bug was
about the unresponsiveness and bloat caused by the livemarks.

> I see people talking about removing a "Live Bookmarks" folder, but I
> don't see that anywhere. And I see stuff about removing couchdb stuff,
> but I don't know anything about that. A detailed explaination of what to
> do would help those who show up here because they're affected by this.

> Zachery: This still seems like a bad solution. Why isn't this just done
> in the background, so that firefox remains responsive while the bookmark
> sync is happening?

The simplest explanation is that I didn't want to dive into dealing with
threads until I saw a need. In retrospect, it was a naive decision.

I agree with you: there shouldn't be any responsiveness penalty for the
user, regardless of bookmarks corpus size. So I'm now exploring
Firefox's thread manager to make all bookmarks collection and network
operations happen on a background thread.

Thanks,

Zac

Zachery Bir (urbanape) wrote :

I've reopened this bug and marked it "in progress"

Changed in bindwood:
status: Fix Committed → In Progress
Brendan_P (brendan-p) wrote :

Hi Zac,

Thanks for the update and the work on this. Look forward to testing to the updated package.

All the best
Brendan

Lastest Karmic, fully Updated (19:30 UK time, 11/11/2009), fresh install of bindwood through synaptic, kills Firefox on both PCs (unresponsive; 100% CPU load). json file is 86k - accumulated debris of years of bookmarking, I admit, but they're all in neat folders by subject, it's not unreasonable... and backup/restore/copy-between-PCs of that json file locally takes seconds, so I don't really understand what the problem is (or for that matter why it's taken two hours for Ubuntu One to upload about 10M of files, which again should only take seconds ... what's going on?)

tags: added: common
Phil Krämer (man0riax) wrote :

Any news here?

iapx86 (jlettvin) wrote :

I learned a lot about Chrome, Arora, and Seamonkey while trying to figure out what froze my FF.
Just found this thread 10 minutes ago and finally got back up and running in FF.

I have thousands of bookmarks. No hope for me with bindwood until this is fixed.

Wrote a python script to be launched by System > Preferences > Preferred Applications > Mail Reader > Custom
to enable other browsers to do what FF does with Edit > Preferences > Applications > mailto
and will share it with any interested parties.

Zachery Bir (urbanape) wrote :

Hi, all.

I've got some news for everyone affected by this. While the updated package isn't yet in Universe, you can get an updated package from my PPA:

  ppa:urbanape/bindwood-exp

The updated version should be:

  $ apt-cache policy bindwood
  bindwood:
    Installed: 0.5-0ubuntu1~ppa1
    Candidate: 0.5-0ubuntu1~ppa1
    Version table:
   *** 0.5-0ubuntu1~ppa1 0
          500 http://ppa.launchpad.net karmic/main Packages
          100 /var/lib/dpkg/status
       0.4.2-0ubuntu2 0
          500 http://us.archive.ubuntu.com karmic/universe Packages

This update doesn't solve moving the work to a background thread, but it does alleviate the vast majority of work we do at (subsequent) startup and while the current session is running.

Please give it a try. On the first startup after installing it, Firefox will still take a short time to become responsive, but after the initial sync, it should be far more usable. I'm still working on making all this happen in the background, as it should, but this should get you all back to being productive with FF.

Thanks for your patience,

Zac

Zachery Bir (urbanape) wrote :

I haven't seen any response to this, but I've gotten good feedback from internal testers. I'll be pushing an SRU with this behavior while I work on the threaded version.

merovius (merovius) wrote :

Sorry I failed to communicate, but I have been running this since you posted it on 11/23 and have had no issues at all. Thumbs up.

Brendan_P (brendan-p) wrote :

Hi Zac,

Installed the new version as requested and all seems fine. As you say, the 1st startup after install is slow but after that all seems ok.

Thanks again.

All the best
Brendan

rincewind (joerg-matysiak) wrote :

Works fine on my two machines running karmic 64bit :)

Zac, thank you for fixing this!

TimH1 (tim-h1) wrote :

Zac,

This now works for me. Thanks for fixing it.

Tim

Jason L. Froebe (jason-froebe) wrote :

installed 0.5-0ubuntu1~ppa1 on karmic 32bit and it just ate up cpu/memory/swap until I ran out :(

philinux (philcb) wrote :

This is not fixed on 64 bit Karmic as of 2010-01-13
The Bindwood addon instantly freezes firefox and cause huge memory leak. All memory gets used.

Binary package hint: bindwood

It also uses half of my swap partition. Screen shot attached.

I disabled all other firefox addons and it still causes the massive memory leak. System becomes unresponsive firefox greys out.
Have to use killall firefox to regain control. Memory then returns to normal.

I had to use ubuntu-bug without bindwood installed in order to submit a bug report.

ProblemType: Bug
Architecture: amd64
Date: Tue Jan 12 16:07:14 2010
DistroRelease: Ubuntu 9.10
InstallationMedia: Ubuntu 9.10 "Karmic Koala" - Release Candidate amd64 (20091020.3)
NonfreeKernelModules: nvidia
Package: bindwood (not installed)
ProcEnviron:
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-17.54-generic
SourcePackage: bindwood
Uname: Linux 2.6.31-17-generic x86_64

See 506496 for ubuntu-bug attachements.

Brendan_P (brendan-p) wrote :

Hi,

Must say I'm not seeing this and I'm on amd64 too...

Not sure what to suggest, just works for me.

Cheers
B

Elliot Murphy (statik) wrote :

This is an important bug, and Zac is fixing it for Lucid. It's not a round-the-clock bug that needs to be marked as critical though, the workaround is to uninstall bindwood.

Changed in bindwood:
importance: Critical → High
description: updated
Zachery Bir (urbanape) on 2010-04-05
tags: added: desktop+ u1-lucid
tags: added: package
tags: added: u1-karmic
removed: u1-lucid ubuntuone-karmic
Martin G Miller (mgmiller) wrote :

I just updated my system to lucid 64 bit. I have been using Ubuntu One for backups since Karmic without problems. As soon as i installed and enabled the bindwood extension, FF 3.6 became unresponsive. I do have over a 100 bookmarks, including some live ones. As long as i disable the bindwood extension, FF 3.6 is very quick. With it enabled, it's not usable.

Botond Szász (boteeka) wrote :

The same is happening to me as described by Martin G Miller (#40)

Ray Wang (raywang) wrote :

Hey there, I was also affected by this bug. I installed bindwood, xul-ext-bindwood 1.0.4-0ubuntu1
i686, lucid with same error which was described by the reporter.

Anders Wallenquist (aw) wrote :

My .local/share/desktop-couch/bookmarks.couch-file are 4GB large (400 bookmarks), I have the same issue at five machines, all lucid. Firefox are unresponsive and turns grey from time to time.

xul-ext-bindwood 1.0.4-ubuntu1

Experienced no unresponsiveness. But today all my bookmarks are gone. My bookmarks.couch file is 324,0 MB! I had merely 50 bookmarks. Surely they've got "compressed" into that file. Anyone lost his/her bookmarks too? What are the chances for recovery?

miki971 (mclzapemtanks) wrote :

I disabled Bindwood 1.0.4 and my Firefox is running great again. I didn't really need it installed anyway.
Thank you!

psypher (psypher246) wrote :

My bookmarks.couch file is 8GB, what the hell?

psypher (psypher246) wrote :

promptly uninstalling bindwood now, i would rather copy and paste than waste 8GB of disk space on something that doesn't really work

Tiparega (tiparega) wrote :

I have seen something that could be interesting, maybe you already know:
Opening ~/.local/share/desktop-couch/couchdb.html you could see a manager of the couchdb, the problem seems to be that FF records a lot of versions for each bookmark (maybe each time you access) and can be solved using "Compact", deleting all old versions and "deleted" bookmarks, now it's 0.8MB.
This is only a temporal solution.

But you can also configure the max size of the database (in "configuration", at the right), for me (Lucid 64, FF 3.6 updated) it's by default at 4GB, the same size I have read here. I change it to 10MB and I will report if it works.
It's the max_document_size, there is another one, max_attachment_chunk_size, that I don't really know what is it, but it was also at 4GB and I changed it to 10MB too.

Maybe if you psypher haven't delete it (or even if you do, maybe) can check the size for you.

Tuomas Heino (iheino+ub) wrote :

NoScript's backup-settings-as-a-bookmark feature seems to cause excessive couchdb growth as well.

About 20 bookmarks and that noscript-settings on Maverick resulted in 245 "documents" in bookmarks taking 9 megabytes of space, with sequence number at 1172 in mere couple days. At times Firefox seemed to be slowly accessing couchdb on every keypress on form fields where javascript was blocked by NoScript.

John O'Brien (jdobrien) on 2012-03-15
Changed in bindwood:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Related questions