Ubuntu

gwibber_messages couchdb views use more disk space than needed

Reported by Eric Casteleijn on 2010-03-15
110
This bug affects 36 people
Affects Status Importance Assigned to Milestone
Gwibber
High
Unassigned
gwibber (Debian)
Invalid
Undecided
Unassigned
gwibber (Ubuntu)
High
Unassigned
Lucid
High
Unassigned

Bug Description

A lot of the views in the gwibber_messages couchdb database emit doc as the value. This is unnecessary, as any view can me made to return the document in its rows by calling it with the parameter include_docs=true, and I believe it wastes a lot of disk space, as the views are stored in their entirety. On my machine, with somewhere around 50K messages in gwibber, the database (after compaction) takes 78MB, whereas the views for it take over 1GB.

I have started work on a branch that fixes this, which should be pretty small, but it's a little more work than I expected, absent any kind of tests. I'll try to add tests for the changes I make at least.

(Another space saver is to use much shorter document ids. It looks like gwibber generates its own ids, so I'd suggest encoding the uuids to base64 to use less bytes or something. Obviously I'm polluting my own bug here, I just wanted to make note of it somewhere in case I forget to follow up on it.)

Changed in gwibber:
assignee: nobody → Eric Casteleijn (thisfred)
Ryan Paul (segphault) on 2010-03-15
Changed in gwibber:
milestone: none → 2.30.0
importance: Undecided → High
milestone: 2.30.0 → none
status: New → Confirmed
Kamus (kamus) on 2010-03-16
Changed in gwibber (Ubuntu):
status: New → Triaged
Changed in gwibber (Ubuntu):
importance: Undecided → High
Changed in gwibber:
status: Confirmed → Fix Committed
Omer Akram (om26er) wrote :

This bug is fix committed from 2010-03-18 and gwibber 2.29.94 was released on 2010-04-01 is this bug fixed?

Patrick Haverkamp (alphakamp) wrote :

I have gwibber 2.30.0.1-0ubuntu3 and I still have this issue.

scottme (poplarmark) wrote :

Definitely still a problem. I have 2.30.0.1 on Ubuntu 10.4 and the .view file in desktop-couch is 3.7GB!!! No excuse for that.

kunami (mastah-killa) wrote :

This bug affects me , too. I have two .view files, for a total of 7GB!

Changed in gwibber (Ubuntu Lucid):
assignee: nobody → Mikhail Turov (groldster)
assignee: Mikhail Turov (groldster) → nobody
Changed in gwibber (Debian):
status: New → Invalid
Eric Casteleijn (thisfred) wrote :

Now that gwibber moved away from couchdb, this is won't fix, or invalid I guess.

Changed in gwibber:
status: Fix Committed → Invalid
Changed in gwibber (Ubuntu):
status: Triaged → Invalid
Tom Goh (tomgohj) wrote :

hmm, what is the fixed version for Ubuntu 10.04 LTS

I guess I am going to do an apt-get purge gwibber because nothing ever gets fixed and gets pushed around.

lucid users can safely use this ppa https://launchpad.net/~*gwibber*-*team*
/+archive/*ppa which fixes this bug including a ton of other bugs too.*

On Fri, Dec 3, 2010 at 3:52 PM, Tom Goh <email address hidden> wrote:

> hmm, what is the fixed version for Ubuntu 10.04 LTS
>
> I guess I am going to do an apt-get purge gwibber because nothing ever
> gets fixed and gets pushed around.
>
> --
> You received this bug notification because you are subscribed to
> Gwibber.
> https://bugs.launchpad.net/bugs/539275
>
> Title:
> gwibber_messages couchdb views use more disk space than needed
>
> Status in Gwibber:
> Invalid
> Status in “gwibber” package in Ubuntu:
> Invalid
> Status in “gwibber” source package in Lucid:
> Triaged
> Status in “gwibber” package in Debian:
> Invalid
>
> Bug description:
> A lot of the views in the gwibber_messages couchdb database emit doc as
> the value. This is unnecessary, as any view can me made to return the
> document in its rows by calling it with the parameter include_docs=true, and
> I believe it wastes a lot of disk space, as the views are stored in their
> entirety. On my machine, with somewhere around 50K messages in gwibber, the
> database (after compaction) takes 78MB, whereas the views for it take over
> 1GB.
>
> I have started work on a branch that fixes this, which should be pretty
> small, but it's a little more work than I expected, absent any kind of
> tests. I'll try to add tests for the changes I make at least.
>
> (Another space saver is to use much shorter document ids. It looks like
> gwibber generates its own ids, so I'd suggest encoding the uuids to base64
> to use less bytes or something. Obviously I'm polluting my own bug here, I
> just wanted to make note of it somewhere in case I forget to follow up on
> it.)
>
>
>

Changed in gwibber:
assignee: Eric Casteleijn (thisfred) → nobody
James Cuzella (trinitronx) wrote :

Ran baobab today and found that gwibber was using about 30GB of space in my home dir!

~$ ls -sh ~/.local/share/desktop-couch/.gwibber_messages_design/
total 26G
 13M 0526a2ae236cc99c2b84d33f1be6cea1.view 2.7G 2f3267703246f5e02533e59714915b7d.view 24G 8873a749eadc0dee755b5fdfdc59c0a1.view

Is this really necessary? How long does gwibber save data? Is there a way to turn down the history saving?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers