Ontv uses lots of CPU and memory when updating tv listings

[Wouter Stomp]

Binary package hint: ontv

Ontv uses a lot of CPU and memory when it updates the tv listings. According to the system monitor, it uses up to 60% cpu, severely slowing down the system and up to 200mb of memory. Even worse it keeps using 200mb of memory after it has finished updating.

[Johan Svedberg (johan-svedberg-com)]

Just wanted to comment on how this works. When updating listings two shell (which are not part of OnTV) scripts are executed, the configured grabber and tv_sort. I think these are the ones responsible for the increased cpu usage, especially tv_sort. It would be nice if we could get rid of that one because it's really only used to fill in missing stop_times in the grabbed xmltv file. We should be able to do this ourself when loading the data. I don't have time to look into this but patches are of course welcome. :-)

About the memory. Are you looking at resident memory? Because that's really what's interesting in this case. It's true that the applet uses a lot of memory but that's because it's keeping all listings in memory. If you want to bring it down you could use the --days option for the grabber. Did some quick measurements, when starting up it's using 49mb, after one update, 88mb, second update 99mb. I think the biggest reason for the increase is Python which is known for being bad at releasing memory.

[Wouter Stomp (wouterstomp-deactivatedaccount)]

How do I look at resident memory? I just fired up the gnome system monitor and got my data from that.

It would be really nice if it could use less resources, because on my 2ghz 512mb pc it brings down everything to a halt when updating. Eliminitaing the need for tv-sort would be great. Also could these scripts be run with a lower priority, so they won't take all cpu resources when I am doing other things? I think that wouldn't be hard to implement. And why does it keep all listings in memory? I would think it only needs to have the current and upcoming programs in memory (and maybe one or two after that) and then read from disk when those are finished.

[Johan Svedberg (johan-svedberg-com)]

In "top" it's the RES field. I think the might be a setting in g-s-m for this.

Running the scripts as niced processes is a good idea and should be easy, I'll look into it. The listings are needed for the search program feature, and there are plans on adding a listing browser as well. The right way to do this is probably to use some light weight database for the listings.

[Cesare Tirabassi (norsetto)]

Looks like bug 475348 to me.

[Gustaf (g-rantila)]

Let me just share my case. I'm on AMD64 (and therefor I am supposed to accept programs swallowing huge amounts of memory).

Now, this is the line from 'top', it's the third biggest memory hog on my machine (running Gnome, Fx3, OOo, Icedove etc):
9154 gustaf 20 0 551m 182m 13m S 0 4.6 2:52.34 /usr/lib/ontv/o

So out of 551 megs (which forces other apps to swap and make the system lazy and require a dozen seconds delay to responds to virtual desktop switching after a few hours of non-usage), 182 megs are directly being used for OnTV. This is outrageous.
And just the fact that ontv has actually allocated _half a gigabyte_ is absolutely I-don't-know-what... WTF?!

Just some quick math: Let's say I care about (have registered for) 10 channels. They have 50 programs per day, and I have a few days (say 3) in beforehand. This means 10 x 50 x 3 = 1500 tv programs. Let's say they have in general, about 500 bytes of info (title + description, and this is probably far over reality). This would mean .75 megs. Let me repeat myself, ontv eats 182 megs. Ok, there are some channel logos and some libraries, but come on! The fact that OnTV stores _everything_ isn't a good enough explanation to why it has to just swallow such huge amounts of ram.

The lunatics doesn't end here; When I haven't right-clicked the OnTV applet icon for a while, doing so causes the system to completely freeze for several seconds (I see lots of CPU usage on my load applet).
This program really acts like some enterprise data center super program, when in fact all it does is present a little TV info.

My suggestion:

When reading new shows (updating), convert the data into an SQLite database. Each minute, read the current (and +1) shows so that the GUI can quickly show it. Searching and other operations can easily be converted into SQLite queries. In other words, almost no programs will be resident (or even allocated) in memory.
This should drastically lower (if not minimize) the memory usage, and if this isn't enough, then perhaps someone will be kind enough to move this from python into something less memory hogging.

It's a shame that such a good little app is one of the worst hoggers on the desktop.

[Johan Svedberg (johan-svedberg-com)]

Sounds great. I'm not working on OnTV anymore, but feel free to implement it and I'll commit the patch.

[Olof Kindgren (flamingolof)]

This is planned for a future release