gwibber_messages couchdb views use more disk space than needed

[Eric Casteleijn]

A lot of the views in the gwibber_messages couchdb database emit doc as the value. This is unnecessary, as any view can me made to return the document in its rows by calling it with the parameter include_docs=true, and I believe it wastes a lot of disk space, as the views are stored in their entirety. On my machine, with somewhere around 50K messages in gwibber, the database (after compaction) takes 78MB, whereas the views for it take over 1GB.

I have started work on a branch that fixes this, which should be pretty small, but it's a little more work than I expected, absent any kind of tests. I'll try to add tests for the changes I make at least.

(Another space saver is to use much shorter document ids. It looks like gwibber generates its own ids, so I'd suggest encoding the uuids to base64 to use less bytes or something. Obviously I'm polluting my own bug here, I just wanted to make note of it somewhere in case I forget to follow up on it.)

[Omer Akram (om26er)]

This bug is fix committed from 2010-03-18 and gwibber 2.29.94 was released on 2010-04-01 is this bug fixed?

[Patrick Haverkamp (alphakamp)]

I have gwibber 2.30.0.1-0ubuntu3 and I still have this issue.

[scottme (poplarmark)]

Definitely still a problem. I have 2.30.0.1 on Ubuntu 10.4 and the .view file in desktop-couch is 3.7GB!!! No excuse for that.

[kunami (mastah-killa)]

This bug affects me , too. I have two .view files, for a total of 7GB!

[Eric Casteleijn (thisfred)]

Now that gwibber moved away from couchdb, this is won't fix, or invalid I guess.

[Tom Goh (tomgohj)]

hmm, what is the fixed version for Ubuntu 10.04 LTS

I guess I am going to do an apt-get purge gwibber because nothing ever gets fixed and gets pushed around.

[Omer Akram (om26er)]

lucid users can safely use this ppa https://launchpad.net/~*gwibber*-*team*
/+archive/*ppa which fixes this bug including a ton of other bugs too.*

On Fri, Dec 3, 2010 at 3:52 PM, Tom Goh <email address hidden> wrote:

> hmm, what is the fixed version for Ubuntu 10.04 LTS
>
> I guess I am going to do an apt-get purge gwibber because nothing ever
> gets fixed and gets pushed around.
>
> --
> You received this bug notification because you are subscribed to
> Gwibber.
> https://bugs.launchpad.net/bugs/539275
>
> Title:
> gwibber_messages couchdb views use more disk space than needed
>
> Status in Gwibber:
> Invalid
> Status in “gwibber” package in Ubuntu:
> Invalid
> Status in “gwibber” source package in Lucid:
> Triaged
> Status in “gwibber” package in Debian:
> Invalid
>
> Bug description:
> A lot of the views in the gwibber_messages couchdb database emit doc as
> the value. This is unnecessary, as any view can me made to return the
> document in its rows by calling it with the parameter include_docs=true, and
> I believe it wastes a lot of disk space, as the views are stored in their
> entirety. On my machine, with somewhere around 50K messages in gwibber, the
> database (after compaction) takes 78MB, whereas the views for it take over
> 1GB.
>
> I have started work on a branch that fixes this, which should be pretty
> small, but it's a little more work than I expected, absent any kind of
> tests. I'll try to add tests for the changes I make at least.
>
> (Another space saver is to use much shorter document ids. It looks like
> gwibber generates its own ids, so I'd suggest encoding the uuids to base64
> to use less bytes or something. Obviously I'm polluting my own bug here, I
> just wanted to make note of it somewhere in case I forget to follow up on
> it.)
>
>
>

[Omer Akram (om26er)]

https://launchpad.net/~gwibber-team/+archive/ppa

[James Cuzella (trinitronx)]

Ran baobab today and found that gwibber was using about 30GB of space in my home dir!

~$ ls -sh ~/.local/share/desktop-couch/.gwibber_messages_design/
total 26G
 13M 0526a2ae236cc99c2b84d33f1be6cea1.view 2.7G 2f3267703246f5e02533e59714915b7d.view 24G 8873a749eadc0dee755b5fdfdc59c0a1.view

Is this really necessary? How long does gwibber save data? Is there a way to turn down the history saving?

[Rolf Leggewie (r0lf)]

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".