me-tv unusually slow, often freezes

[spybotor]

Binary package hint: me-tv

Hi,
on Ubuntu 9.04 Beta (Jaunty) me-tv is very slow (compared to previous versions), the gui often freezes so i have to kill it. If i'm using xine_ui directly i don't have such problems, but channel locking is also slow.
System Specs:
Pentium M 2GHz
1 Gb RAM
WideView WT-220U PenType Receiver

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I can't reproduce this issue on any of my 3 computers. The least spec'ed one is my Media Centre which runs Me TV constantly:

Debian Lenny - 32 bit
AMD Athlon 3200 (single core)
On-board GeForce 6150
nVidia drivers
1 GB RAM

It really behaves no differently to any other version of Me TV.

Try changing the xine video output driver in the preferences dialog.

[spybotor (backdoor-netshadow)]

HI, thx for the advice!
But i already tried that - without changes.
I think this problem is related to the driver and/or xine.

dmesg says:
[ 15.797847] dvb-usb: found a 'WideView WT-220U PenType Receiver (Typhoon/Freecom)' in warm state.
[ 15.798009] dvb-usb: will use the device's hardware PID filter (table count: 15).
[ 15.799016] DVB: registering new adapter (WideView WT-220U PenType Receiver (Typhoon/Freecom))
[ 15.799119] DVB: registering adapter 0 frontend 0 (WideView USB DVB-T)...
[ 15.799422] input: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1d.7/usb1/1-6/input/input12
[ 15.825433] dvb-usb: schedule remote query interval to 300 msecs.
[ 15.825440] dvb-usb: WideView WT-220U PenType Receiver (Typhoon/Freecom) successfully initialized and connected.
[ 18.136012] dvb-usb: recv bulk message failed: -110

I dont really know what to do to pin down the cause of this. The bulk message error doenst seem to be that important as xine-ui plays all my dvb streams without problems. Only weird thing about xine is that it always starts with volume set to zero (and thats means me-tv is muted anyway because you cant control the volume from inside it), but thats a xine specific problem.
Could it be that the video driver (nvidia) has something to do with it?

I will try to enable RenderAccel and post results...

[spybotor (backdoor-netshadow)]

Hi again,
i found out that my me-tv turns gray when updating its EPG data.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Please try 0.8.0 [0] and tell me if that works any better for you. If not, you can revert. Also, be advised of this bug [1], it's easy to work around.

[0] https://launchpad.net/~me-tv-development/+archive/ppa
[1] https://bugs.edge.launchpad.net/me-tv/+bug/353796

[spybotor (backdoor-netshadow)]

The .deb download section seems to be broken

Sometimes i receaved messages like this:
04.04.2009 16:30:02: Saving channel 145 (WDR Köln)
04.04.2009 16:30:02: Exception: Lesen der Daten vom Demuxer fehlgeschlagen: Value too large for defined data type
04.04.2009 16:30:02: Exception in EPG thread: Lesen der Daten vom Demuxer fehlgeschlagen: Value too large for defined data type
04.04.2009 16:30:02: Updating EPG

I just compiled the 0.8.1. It doenst help though :-/
Me-Tv turns gray while receaving many mesages of this kind:
04.04.2009 17:33:30: Ignoring EPG event 291 (*something*), too old

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I think that I am now seeing a merge with this bug [0]. I will try to solve this bug first because you have compiled from source which gives us more options. I'm attaching a file to replace src/gtk_epg_widget.cc with. All it does is add extra debugging messages to try to isolate where the all time is being taken. Please attach the output from `me-tv -v`. I'm not sure this will expose anything but it's a start.

Launchpad was not accessible for a few hours, not sure why. Hopefully it should work now.

"Value too large ..." means that your computer isn't keeping up with the events being sent. It's not fatal.
"Ignoring EPG event ..." is also normal, it means that the EPG event that the broadcaster sent was for an event in the past.

[0] https://bugs.launchpad.net/ubuntu/+source/me-tv/+bug/353798

[spybotor (backdoor-netshadow)]

Hello,
ive logged everything until i had to kill me-tv. Before i first could do anything in me-tv i had to wait several seconds (i'd say about 20). Is there a possibility to disable EPG for testing purposes?
regards

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Looking at the log I don't think that it is the EPG.

To disable the EPG, comment out line 535 in main_window.cc

Also, while you're at it and willing to try a few things, let's disable the EPG thread by commenting out line 115 of stream_thread.cc.

Let's see if we can isolate it.

[spybotor (backdoor-netshadow)]

Hi,
thanks! I tried it an until now i didnt run into ui-freezes yet. Only some channels (like Pro7 and N24) need a little bit more time for channel locking and are turning the window gray for ~2 seconds, but this is due to signal quality i guess.
(BTW im using gstreamer engine as xine is muted, but channel locking times arent affected by that)

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Which one of those two fixes made the difference? Then we can dig a little deeper to find out the exact cause.

So did you add the configure option to the build to enable GStreamer?

[spybotor (backdoor-netshadow)]

Yes i did (also xine-lib which doenst work for me because of a audio stream error),

i uncommented line 535 in main_window.cc (epg updates ON) an it runs good. So i assume the prohibition of the EPG thread did the job.

[spybotor (backdoor-netshadow)]

Now i tried it the other way around with the same results. I'll try it without changes to make sure whether it is the engine option which fixes it.

[spybotor (backdoor-netshadow)]

Okay.... It runs relativly stable with both lines enabled. The window turned gray after some clicking on the ui. Now i disabled compiz and and didnt run into freezes yet.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Thanks for trying options. Like I said, I cannot replicate this issue. Keen to know your results.

[spybotor (backdoor-netshadow)]

Yeah, i think this might have something to do with xine as giving the sub-thread a better nice value improves stability. I'll also try mplayer and vlc options. Thanks for your help and go on with the good work, i really like me-tv.

[VladimirCZ (vlabla)]

I am very sorry to say that the same happens to me with the now released version 0.8.6 and even worse. This version even when properly closed leaves behind some running process with comment in wait channel of System monitor "jbd2_log_wait_commit". This also disables other start of me-tv resulting in a message that the database is locked.

OS Ubuntu 64-bit 9.04 fully updated.
(Note: the old version from Ubuntu repo worked up to today's upgrade without a problem).

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I cannot reproduce this. Is there anyone with this issue that can debug it?

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I've added 2 options to 0.8.7:

--disable-epg
Stops the rendering of the EPG event buttons on the UI.

--disable-epg-thread
Disable the EPG thread. Me TV will stop collecting EPG events.

Please try these options and let me know if they help at all.

[VladimirCZ (vlabla)]

I am still using the version 0.8.6 and after some experimenting it works well.

The problem with freezing for a long time happened after installation at first start when I imported a config file containing 25 channels and the database was being filled with the EPG data. Then the EPG and Sqlite caused the me-tv being irresponsive and grayed. After a minute or so about I killed me-tv. Then I had to purge the ~/.me-tv/ folder etc.

As Michael Lamothe was indicating he could not reproduce this behaviour and he is the developer I decided to install libsqlite3-dev (development) package and started the me-tv again. This time I let it be grayed for a few minutes. Then I minimized and unminimize it. After that the me-tv gui returned to colour mode and since then I can use it. It is hard to say what helped: the development package or subsequent long waiting for the initial database filling ...

When I started the me-tv first time today it grayed for about 10 seconds (updating the database I guess) and since then it works without a problem.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I really don't think that libsqlite2-dev will help. The actions that you have described are king of normal, 10 seconds is a bit slow but maybe you have more events.

Are these the same symptoms that the original poster had?

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Of course I mean libsqlite3-dev ;)

[spybotor (backdoor-netshadow)]

> Are these the same symptoms that the original poster had?

Yes i had the same symptoms. After being away for a week me-tv took a lot more time to start than under daily usage.

[marijus (marijus73)]

michael,
you probably cant reproduce the problem because lenny is using libsqlite3-0 version 3.5.9-6 while jaunty is using version 3.6.10-1.

i remember to have read somewhere that this two versions behave different in some ways... that is also a reason why liferea feedreader (1.4.x up to 1.5.15) doesnt work well in jaunty... (actually same symptoms while updating the feeds)

and im pretty sure this is a duplicate of #353798 (or the other way around)

[marijus (marijus73)]

btw in 0.8.7 --disable-epg-thread obviously resolves the problem...

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I've just installed Jaunty on 2 computers with 2 different DVB devices and tried it without issue. Admittedly there was a spot where the UI did not respond for 5-10 seconds, but that is to be expected and should be a one-off. Now, I would still like to remove this issue because I think I will solve another 2 outstanding bugs at the same time.

The major database access functions are:

1. EPG Thread (write)
2. EPG UI (read)
3. Scheduled recording (write)
4. Scheduled recording check (read)

I can only assume that the issue is contention between 1 and 2. Now, I would like to propose and implement a solution if people are prepared to build and test it from source, preferably the bzr repository. Let me know if you are interested.

Let me explain what I think the issue is, the UI is locking up while it tries to update the EPG UI because it happens within a GDK lock (blocking all other UI updates). The EPG UI is taking a long time to draw because it's waiting for EPG events to be written, so the whole UI locks up.

My solution, make the EPG UI DB call fail silently if the database is busy and let it try again a second later, it's not critical if the events don't appear immediately when updating the UI.

Thanks,

Michael

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Actually, the simple 4 database function scenario that I describe isn't correct, but the same improvements should apply.

[spybotor (backdoor-netshadow)]

Sounds good, i want to give it a try. Which libsqlite version should be installed?

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

The version that came with Jaunty. I'm testing this on a Jaunty machine so it is best that we use the same version.

[spybotor (backdoor-netshadow)]

0.8.7: The epg database seems to be filled faster at the beginning while the interface remains accessible. But after a short time (and switching the channel) the gui locks and takes too much cpu-time (my whole system got locked one time) for some minutes (until the database ist filled). This could be related to libsqlite. But as you dont have this issue this could also be affected by bad reception or a combination of both (because earlier versions of me-tv didnt have any gui-locks at the same location).

[spybotor (backdoor-netshadow)]

PS:
After the database is filled everything runs smoothly

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Ok, I DO have this issue on mt laptop and it IS sqlite on Jaunty, Thanks for your patience, I can now replicate this issue. It takes about 500 ms to save a single record to the database. If there are 100 events and many event texts then we have an issue. Still pondering the best way to get around this.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I've just checked in a version that saves the EPG data in the EPG event thread rather than in the main GTK thread. To test, please remove the ~/.me-tv/me-tv.db and run Me TV. Scott is usually pretty quick at packaging the pre-release versions in his PPA [0]. If you can't be bothered building it yourself you might find that he has a version available shortly.

I need to know if this works and I'm on the right track so please report back your findings.

Thanks,

Michael

[0] https://launchpad.net/~vk7hse/+archive/scott-vk7hse-jaunty

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Also, if you know what you are doing you might want to try a later version of libsqlite3 from http://packages.debian.org/sid/libsqlite3-0. I think that it might still install on Jaunty.

[spybotor (backdoor-netshadow)]

0.8.7 with 406_405.diff:

The first channel plays but after switching channel I had to kill it after 20 mins while getting constantly these two messages:

28.04.2009 20:47:21: No output connected
28.04.2009 20:47:21: Writing PAT/PMT header

I also installed the newer libsqlite from sid but i didnt notice any difference

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Those 2 messages are normal when there is nothing receiving the stream i.e. the xine player is not running. Does it give you any indication as to why the player failed?

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Yep, I can see that now. Working on it.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Ok, sorry about that. I think that I've fixed that issue in revision 409 of the development branch. Please try that, no promises, I don't have time to do a full regression test.

Also, I think that I might have subtitles working. If you are able, please test that too.

Thanks,

Michael

[spybotor (backdoor-netshadow)]

Ok installed revision 409:

Well, unfortunately that didnt change much for me.
BTW - I receive 45 stations over 9 networks. This is surely a reason for long response times.

now i receive new messages like these:

29.04.2009 23:29:01: *** 'channel' REPLACE
29.04.2009 23:29:01: *** 'channel' STEPPING
29.04.2009 23:29:01: *** 'channel' REPLACE
29.04.2009 23:29:01: *** 'channel' STEPPING
29.04.2009 23:29:01: *** 'channel' REPLACE
29.04.2009 23:29:01: *** 'channel' STEPPING

and of course even more of that kind:

29.04.2009 23:31:02: Ignoring EPG event 53777 (Frühreport), too old
29.04.2009 23:31:02: Ignoring EPG event 53778 (Frühreport Wirtschaft), too old

if i dont disable epg-thread the ui will lock (but at least i can continue watching)

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Please try revision 413.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

I've just released 0.8.8 which should hopefully help with this issue. To recap, this is an issue with the libsqlite3 included in Jaunty. All EPG events/scheduled recording persistence is done in a new separate thread (Save Thread). The save thread is called every minute and on certain actions in the application. If you have data that needs to be saved when Me TV exits the application will remain running until the data is saved (see --verbose if you want to see this in action).

[ourasi (ourasi)]

Thank you, maybe it's little better now.
Tested 417 for half an hour.

I noticed, (with --verbose) that Me-tv over again
writes same "too old" items, is it OK?

Here is my trial to make scheduled recording, it allways makes same
Exception after I must kill Me-tv.

05/02/2009 13:05:00: Now: 1241258700
05/02/2009 13:05:00:
=============================================================================================
05/02/2009 13:05:00: #ID | Start Time | Duration | Record | Channel |
Device | Description
05/02/2009 13:05:00:
=============================================================================================
05/02/2009 13:05:00: 1 | 1241258700 | 3360 | true | YLE TV1 |
/dev/dvb/adapter1/frontend0 | Ykkösdokumentti: Näköismetsä
05/02/2009 13:05:00: Changing channel for scheduled recording
05/02/2009 13:05:00: Exception: Channel '0' not found
05/02/2009 13:05:01: Writing PAT/PMT header
05/02/2009 13:05:01: Ignoring EPG event 824 (Tv-uutiset), too old
05/02/2009 13:05:01: Ignoring EPG event 824 (Tv-uutiset), too old
05/02/2009 13:05:02: Ignoring EPG event 824 (Tv-uutiset), too old
05/02/2009 13:05:02: Ignoring EPG event 824 (Tv-uutiset), too old
05/02/2009 13:05:03: Writing PAT/PMT header

Off topic
Its nearly summer here now

Two http://en.wikipedia.org/wiki/Osprey cameras here in Finland:
http://www.natureit.net/site/saaksikamera.php?cam=seili-wide&refresh=15
http://kotinetti.suomi.net/saaksi/index.php

Going shopping now
Esa

On Sat, May 2, 2009 at 12:16 PM, Michael Lamothe
<email address hidden>wrote:

> I've just released 0.8.8 which should hopefully help with this issue.
> To recap, this is an issue with the libsqlite3 included in Jaunty. All
> EPG events/scheduled recording persistence is done in a new separate
> thread (Save Thread). The save thread is called every minute and on
> certain actions in the application. If you have data that needs to be
> saved when Me TV exits the application will remain running until the
> data is saved (see --verbose if you want to see this in action).
>
>
> ** Changed in: me-tv
> Status: Fix Committed => Fix Released
>
> --
> me-tv unusually slow, often freezes
> https://bugs.launchpad.net/bugs/351510
> You received this bug notification because you are subscribed to Me TV.
>
> Status in Me TV, it's TV for me computer: Fix Released
> Status in “me-tv” source package in Ubuntu: Confirmed
>
> Bug description:
> Binary package hint: me-tv
>
> Hi,
> on Ubuntu 9.04 Beta (Jaunty) me-tv is very slow (compared to previous
> versions), the gui often freezes so i have to kill it. If i'm using xine_ui
> directly i don't have such problems, but channel locking is also slow.
> System Specs:
> Pentium M 2GHz
> 1 Gb RAM
> WideView WT-220U PenType Receiver
>

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Me TV 0.8.8 is now based on revision 418. I do not get the error that you have, please try with 0.8.8 and let me know if the issue persists.

Thanks,

Michael

[VladimirCZ (vlabla)]

I have installed the version 0.8.9 under Ubuntu 9.04 64-bit and it looks that all freezing is over. So far no problems have re-appeared. Michael, thank you for your exceptionally fast and effective response.

[Michael Lamothe (lamothe-deactivatedaccount-deactivatedaccount)]

Thanks for your comments. 0.8.8-0.8.9 are still a bit unstable after the many changes required to fix this issue. Specifically, I've noticed that adding a second scheduled recording clobbers the first one, similarly with channel edits. Please test the application and let me know if you see any other issues.

[spybotor (backdoor-netshadow)]

Thank you very much!

[Launchpad Janitor (janitor)]

This bug was fixed in the package me-tv - 0.9.4-0ubuntu1

---------------
me-tv (0.9.4-0ubuntu1) karmic; urgency=low

  * New upstream release (LP: #379706)
    - Fix to stop EPG update crashing the application
      after faulty save (LP: #72872)
    - Fixed spin buttons from GtkBuilder conversion (LP: #382197)
    - Fixed icon on application popup menu (LP: #379685)
    - Fixed compiling of me-tv-0.8.12 fails on Fedora 11 (LP: #377020)
    - Fixed Failed to lock to channel at boot (LP: #377050)
    - Increased timeout to 5 seconds again (LP: #371165)
    - Fixed me-tv unusually slow, often freezes (LP: #351510)
    - Fixed channel persistence (LP: #361514)
    - Fix for forward slashes in description (LP: #359710)
    - Fixed Must create .me-tv directory manually (LP: #353796)
    - Fixed audio stream can't be changed (LP: #350402)
  * debian/control:
    - Removed dependency libxine1-ffmpeg, libxine1-x
    - Removed libglademm-2.4-dev Build-Depends.

 -- Scott Evans <email address hidden> Sat, 27 Jun 2009 00:46:22 +1000