incorrect usage of MLT API

[Gleb Smirnoff]

Running OpenShot 1.4.0 linked to mlt-0.7.6 on FreeBSD 10. This works fine when source files for example are DV dumps. However, trying to use AVI file produced by Nikon D90 is causing a crash.

Backtrace and some additional info:

(gdb) bt
#0 mlt_properties_get_double (self=0x0, name=0x80c0da573 "_speed")
at mlt_properties.c:484
#1 0x000000080beaed58 in Mlt::Producer::get_speed (this=0x81bce8100)
at MltProducer.cpp:149
#2 0x000000080beaedcd in Mlt::Producer::pause (this=0x81bce8100)
at MltProducer.cpp:133
#3 0x000000080bc6435b in frame_get_image ()
from /usr/local/lib/python2.6/site-packages/_mlt.so
#4 0x000000000047d28d in PyEval_EvalFrameEx ()
#5 0x000000000047e134 in PyEval_EvalFrameEx ()
#6 0x000000000047e134 in PyEval_EvalFrameEx ()
#7 0x000000000047e134 in PyEval_EvalFrameEx ()
#8 0x000000000047e134 in PyEval_EvalFrameEx ()
#9 0x000000000047e134 in PyEval_EvalFrameEx ()
#10 0x000000000047eae1 in PyEval_EvalCodeEx ()
#11 0x00000000004c824d in PyClassMethod_New ()
#12 0x0000000000417ecd in PyObject_Call ()
#13 0x000000000041d86d in PyClass_IsSubclass ()
#14 0x0000000000417ecd in PyObject_Call ()
#15 0x0000000000477eb6 in PyEval_CallObjectWithKeywords ()
#16 0x0000000803ae46b6 in init_gobject ()
from /usr/local/lib/python2.6/site-packages/gobject/_gobject.so
#17 0x000000080202554f in g_closure_invoke ()
from /usr/local/lib/libgobject-2.0.so.0
#18 0x000000080203c521 in g_signal_handlers_block_matched ()
from /usr/local/lib/libgobject-2.0.so.0
#19 0x000000080203e0b2 in g_signal_emit_valist ()
from /usr/local/lib/libgobject-2.0.so.0
#20 0x000000080203e688 in g_signal_emit_by_name ()
from /usr/local/lib/libgobject-2.0.so.0
#21 0x000000080e4adf00 in propagate_event ()
from /usr/local/lib/libgoocanvas.so.3
#22 0x000000080e4ae191 in emit_pointer_event ()
from /usr/local/lib/libgoocanvas.so.3
#23 0x00000008042d50cf in gtk_marshal_BOOLEAN__VOID ()
from /usr/local/lib/libgtk-x11-2.0.so.0
#24 0x000000080202554f in g_closure_invoke ()
from /usr/local/lib/libgobject-2.0.so.0
#25 0x000000080203c6eb in g_signal_handlers_block_matched ()
from /usr/local/lib/libgobject-2.0.so.0
#26 0x000000080203e0b2 in g_signal_emit_valist ()
from /usr/local/lib/libgobject-2.0.so.0
#27 0x000000080203e7c3 in g_signal_emit ()
from /usr/local/lib/libgobject-2.0.so.0
#28 0x00000008043e48b5 in gtk_widget_style_attach ()
from /usr/local/lib/libgtk-x11-2.0.so.0
#29 0x00000008042cdd29 in gtk_propagate_event ()
from /usr/local/lib/libgtk-x11-2.0.so.0
#30 0x00000008042cee57 in gtk_main_do_event ()
from /usr/local/lib/libgtk-x11-2.0.so.0
#31 0x0000000804805bac in gdk_add_client_message_filter ()
from /usr/local/lib/libgdk-x11-2.0.so.0
#32 0x00000008024a37f3 in g_main_context_dispatch ()
from /usr/local/lib/libglib-2.0.so.0
#33 0x00000008024a7812 in g_main_context_prepare ()
from /usr/local/lib/libglib-2.0.so.0
#34 0x00000008024a7c05 in g_main_loop_run ()
from /usr/local/lib/libglib-2.0.so.0
#35 0x00000008042cf1f3 in gtk_main () from /usr/local/lib/libgtk-x11-2.0.so.0
#36 0x0000000803e99dbd in init_gtk ()
from /usr/local/lib/python2.6/site-packages/gtk-2.0/gtk/_gtk.so
#37 0x000000000047c914 in PyEval_EvalFrameEx ()
#38 0x000000000047e134 in PyEval_EvalFrameEx ()
#39 0x000000000047e134 in PyEval_EvalFrameEx ()
#40 0x000000000047eae1 in PyEval_EvalCodeEx ()
#41 0x000000000047ebc2 in PyEval_EvalCode ()
#42 0x0000000000498462 in Py_CompileString ()
#43 0x0000000000498536 in PyRun_FileExFlags ()
#44 0x00000000004999bf in PyRun_SimpleFileExFlags ()
#45 0x0000000000413dd3 in Py_Main ()
#46 0x00000000004131ea in main ()
(gdb) fr 0
#0 mlt_properties_get_double (self=0x0, name=0x80c0da573 "_speed")
at mlt_properties.c:484
484 property_list *list = self->local;
(gdb) fr 1
#1 0x000000080beaed58 in Mlt::Producer::get_speed (this=0x81bce8100)
at MltProducer.cpp:149
149 return mlt_producer_get_speed( get_producer( ) );
(gdb) fr 2
#2 0x000000080beaedcd in Mlt::Producer::pause (this=0x81bce8100)
at MltProducer.cpp:133
133 if ( get_speed() != 0 )
(gdb) fr 3
#3 0x000000080bc6435b in frame_get_image ()
from /usr/local/lib/python2.6/site-packages/_mlt.so

I can put an example AVI on the http somewhere if it would help.

Initially I've filed this report in the MLT bug repository, but MLT developer noticed that invalid mlt.Producer was passed to API. He thinks that problem should be fixed on the side of OpenShot:

https://sourceforge.net/tracker/index.php?func=detail&aid=3441193&group_id=96039&atid=613414

[Olivier Girard (eolinwen)]

Hi,
I don't know how Dan do that.
Effectively, it will be interesting to have an sample of this file somewhere. It will be helpful.
Thanks.

[Gleb Smirnoff (glebius)]

Here is sample file:

http://glebius.int.ru/tmp/DSC_8560.AVI

[Olivier Girard (eolinwen)]

Thanks a lot. We'll take a look soon as we 'll have the time.

[Andy Finch (fincha)]

Hi, we have tried your supplied file and can't reproduce the crash. We also took a look at Dan's comments on your MLT bug report. Openshot already uses the is_valid() method to check the producer object.

[Gleb Smirnoff (glebius)]

  Hi, Andy!

On Sun, Nov 27, 2011 at 09:24:55PM -0000, Andy Finch wrote:
A> Hi, we have tried your supplied file and can't reproduce the crash. We
A> also took a look at Dan's comments on your MLT bug report. Openshot
A> already uses the is_valid() method to check the producer object.
A>
A> ** Changed in: openshot
A> Milestone: 1.4.1 => None
A>
A> ** Changed in: openshot
A> Status: New => Invalid

I'd ask you not to close the bug, but help me to investage it further.

I hope, that you believe me that I really experience this bug, and I
didn't type an imaginary backtrace by hand :) So marking my report as
"invalid" kinda offends me :)

So, the problem might be on my side. I'm pretty sure it'll be reproducible
on any FreeBSD with same package set as mine, and OpenShot 1.4.0. It might
be reproducible on Linuxes with exactly same set of versions of dependecies.
So closing it is not a perfect idea.

What can I do to understand it? What additional info can I obtain from backtrace?

--
Totus tuus, Glebius.

[Gleb Smirnoff (glebius)]

On Sun, Nov 27, 2011 at 09:24:55PM -0000, Andy Finch wrote:
A> Hi, we have tried your supplied file and can't reproduce the crash. We
A> also took a look at Dan's comments on your MLT bug report. Openshot
A> already uses the is_valid() method to check the producer object.

Actually Openshot doesn't use is_valid() after obtaining new produceer,
in the thumbnail.py line 287:

                #get the frame
                self.p = self.p.cut(frame, frame)

--
Totus tuus, Glebius.

[Andy Finch (fincha)]

At what point do you get the crash? Is it during the file import or playback or some other operation?

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 12:58:15PM -0000, Andy Finch wrote:
A> At what point do you get the crash? Is it during the file import or
A> playback or some other operation?

Import goes successfully and Openshot draws a thumbnail with first
frame. Then I drag clip to the timeline and any attempt to move the
red cursor within timeline causes crash.

--
Totus tuus, Glebius.

[Olivier Girard (eolinwen)]

Just a thought.

Could you see the clip in the tree project without a crash ?
Select the clip, right click, Option preview file.
Thanks.

[Olivier Girard (eolinwen)]

Another one thought, I'm thinking that you have already check certainly and it is just more to know if you have the same dependencies on BSD system : could you tell us if you have all this dependencies : http://openshotusers.com/forum/viewtopic.php?f=12&t=1060
Thanks.

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 02:16:35PM -0000, Cenwen wrote:
C> Just a thought.
C>
C> Could you see the clip in the tree project without a crash ?
C> Select the clip, right click, Option preview file.
C> Thanks.

No, it crashes. End of backtrace looks quite alike:

(gdb) bt
#0 mlt_properties_get_double (self=0x0, name=0x80cd0f573 "_speed") at mlt_properties.c:484
#1 0x000000080cae3d58 in Mlt::Producer::get_speed (this=0x81ba3ad60) at MltProducer.cpp:149
#2 0x000000080cae3dcd in Mlt::Producer::pause (this=0x81ba3ad60) at MltProducer.cpp:133
#3 0x000000080c88a154 in _wrap_Producer_pause (args=0x81b8fac90) at mlt_wrap.cxx:11733
#4 0x0000000000483904 in PyEval_EvalFrameEx ()
#5 0x0000000000483e65 in PyEval_EvalFrameEx ()
#6 0x0000000000483e65 in PyEval_EvalFrameEx ()
#7 0x0000000000483e65 in PyEval_EvalFrameEx ()
#8 0x0000000000485580 in PyEval_EvalCodeEx ()
#9 0x00000000004d4ae9 in PyClassMethod_New ()
#10 0x0000000000418c2d in PyObject_Call ()

--
Totus tuus, Glebius.

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 02:23:08PM -0000, Cenwen wrote:
C> Another one thought, I'm thinking that you have already check certainly and it is just more to know if you have the same dependencies on BSD system : could you tell us if you have all this dependencies : http://openshotusers.com/forum/viewtopic.php?f=12&t=1060

I've got the following packages:

x264-0.116.2076
ffmpeg-0.7.7,1 (including /usr/local/lib/libavformat.so.52.110.0)
python-2.7,2
py27-xdg-0.19
py27-gtk-2.24.0
py27-mlt-0.7.6
py27-goocanvas-0.14.1_3
py27-imaging-1.1.7_1
goocanvas-0.15_3 (including /usr/local/lib/libgoocanvas.so.3)
mlt-0.7.6 (including /usr/local/lib/libmlt++.so.3, /usr/local/lib/libmlt.so.4, /usr/local/bin/melt, and including all files from Linux mlt-data package)
sox-14.3.2_1
frei0r-plugins-1.3
sdl-1.2.14_2,2 (including /usr/local/lib/libSDL-1.2.so.11)
librsvg2-2.34.1
py27-httplib2-0.7.1
fontconfig-2.8.0_1,1

MISSING:
libgoocanvas-common

If libgoocanvas-common is just what is described here:

http://packages.debian.org/lenny/all/libgoocanvas-common/filelist

, then I don't think it is important.

--
Totus tuus, Glebius.

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 02:16:35PM -0000, Cenwen wrote:
C> Could you see the clip in the tree project without a crash ?
C> Select the clip, right click, Option preview file.

More information on this. I have tried this sequence using KOI8-R
locale ( LANG=ru_RU.KOI8-R ). With this locale the preview doesn't
crash, it just displays "INVALID" with a large font in the preview
window.

However, putting clip to the timeline and trying to move cursor
produces same crash as before.

--
Totus tuus, Glebius.

[Olivier Girard (eolinwen)]

And what gives you this ticket ?
https://answers.launchpad.net/openshot/+faq/983

[Olivier Girard (eolinwen)]

Have you reconfigure your locales ?
Could you have the last stable version of MLT (today the 0.7.7) ?

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 05:40:42PM -0000, Cenwen wrote:
C> Have you reconfigure your locales ?

I usually use ru_RU.KOI8-R locale. I have also tried running with
ru_RU.UTF-8. The crash reported is the same under both locales.

However the video preview scenario, that you asked me to tru, works
different.

C> Could you have the last stable version of MLT (today the 0.7.7) ?

I believe 0.7.7 is not released, yet. The site says 0.7.6 is latest
one: http://www.mltframework.org/twiki/bin/view/MLT/

--
Totus tuus, Glebius.

[Gleb Smirnoff (glebius)]

On Mon, Nov 28, 2011 at 05:38:25PM -0000, Cenwen wrote:
C> And what gives you this ticket ?
C> https://answers.launchpad.net/openshot/+faq/983

For the video file DSC_8560.AVI, that I have posted as an
example:

1) ffplay works, seek works.
2) melt works, frame incrementing/decrementing works.

--
Totus tuus, Glebius.

[Olivier Girard (eolinwen)]

Hum for me it seems to come from MLT. (The fact that you can't move the cursor without having a crash).

Could you get back the last development version of MLT ?
I am agree that 's not funny and can be complicated and it is without guarantee.
Under if you can wait another stable version.
It is working with the 0.7.7.

[Gleb Smirnoff (glebius)]

On Fri, Dec 02, 2011 at 05:59:36PM -0000, Cenwen wrote:
C> Hum for me it seems to come from MLT. (The fact that you can't move the
C> cursor without having a crash).
C>
C> Could you get back the last development version of MLT ?
C> I am agree that 's not funny and can be complicated and it is without guarantee.
C> Under if you can wait another stable version.
C> It is working with the 0.7.7.

I'll try as soon as I have time.

Btw, here is comment from Dan of MLT:

  Comment By: Dan Dennedy (ddennedy)
  Date: 2011-12-01 12:56

  Message:
  I see openshot/classes/video.py is using Producers without checking if
  is_valid(). However, there is validation check in thumbnail.py, but I do
  not know if the code in thumbnail is always executed before that in
  video.py.

And as I also noticed earlier, in the thumbnail.py producer is checked
once, but then it is redefined and not checked.

--
Totus tuus, Glebius.

[Gleb Smirnoff (glebius)]

Okay, there several places where MLT Producer isn't checked for being valid and used later:

thumbnail.py
cli_render.py
video.py

My crash is related to video.py.

Attached patch _tries_ to fix the problem. However, in the video.py case my fix is insufficient - although I return and don't use producer immediately, I still get a crash later:

(gdb) bt
#0 mlt_properties_get (self=0x0, name=0x80cd0f57a "eof")
    at mlt_properties.c:484
#1 0x000000080cd03b0f in mlt_producer_seek (self=0x0, position=0)
    at mlt_producer.c:296
#2 0x000000080cae3f7e in Mlt::Producer::seek (this=0x81b9ab0a0, position=0)
    at MltProducer.cpp:111
#3 0x000000080c88a5d4 in _wrap_Producer_seek (args=0x81ba24e18)
    at mlt_wrap.cxx:11636
#4 0x0000000000483ba8 in PyEval_EvalFrameEx ()
....

From a quick glance I don't see a proper path to return failure to upper functions: load_profile, load_xml, etc... These functions are expect to succeed always. So, I suppose, later the object is called again and MLT code is entered.

I don't have enough time and python skills to write a more thorough patch, that would touch more code :(

[Gleb Smirnoff (glebius)]

Finally I have found what causes the crash. The absolute path to files from my camera contains 8-bit, but not UTF-8 characters. These are in KOI8-R encoding. If I move files to other location, that contains 7-bit ascii chars only, everything is okay.

I didn't yet understand where does the problem reside: in mlt or in openshot. Anyway, I won't ask neither Openshot nor MLT developers to fix that since nowadays everyone is moving to UTF-8 and everything else is considered obsoleted. But, the crash isn't a correct way to report problem :) You don't expect every user is capable to run gdb and dig out what's going on, so crash definitely needs to be converted to some error banner, or at least error message to stdout.

Thanks!

[Olivier Girard (eolinwen)]

Hi,

Normal, Openshot use the standard UTF-8. But I was thinking that the uft-8 should read all the others standards without any problems. .................
Exactly (about the habitabilities of the user).
The both (mlt and Openshot) are using UTF-8 like you have said previously, it is the standard.
I put your patch in the next milestone which will be applied after verification.
Thanks a lot.
Olivier.

[Gleb Smirnoff (glebius)]

Thanks, but be aware that patch to video.py is not sufficient to fix crash, it just marks a place where something better should be worked out, than my patch.

[Jonathan Thomas (jonoomph)]

Thanks for the patch. I tested it, and it seemed to work fine. I know this patch does not fix the crash, but since I don't have time to work on this further, I think this patch is as good as it's going to get for a while. Thanks for your help!