OSError: [Errno 24] Too many open files

[Michael Salmons]

    Using OpenShot 1.3-alpha-1 latest branch, while trying to view edits in the timeline (sliding the bar) the preview screen goes white, the the program stops working, the only term output is OSError: [Errno 24] Too many open files: '~/.openshot/queue'

    Running the program straight from .py haven't compiled it, if that makes a difference.

[Andy Finch (fincha)]

Can't reproduce. Is there a particular type of file you were using (format?)?

[IRIE Shinsuke (irie)]

Same for me. (r443)

Steps to reproduce the problem:

1. Generate an animated title and put it into a track
2. Play the movie several times

Then, I get an error like below:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
  File "/usr/lib/pymodules/python2.6/openshot/classes/lock.py", line 165, in run
OSError: [Errno 24] Too many open files: '/home/irie/.openshot/queue'
...

After that, I can no longer do playback or any file operations.

Though lots of files are opened when playing the animated title, these seem not to be closed.

[Jonathan Thomas (jonoomph)]

I was not able to reproduce this, but I did find some code in the lock.py that opens a file every 1 second, and never closes it. So... that is not good, and it's now fixed. I also found a few other files that were not being closed. Please test again, and see if you can still reproduce it with these changes.

[IRIE Shinsuke (irie)]

@Jonathan
Thanks, but the error still occurs.

I think your changes take no effect because a file object should automatically close when its reference count hits zero. For example,

Code 1:

import os
f=[]
for name in os.listdir("."):
    f.append(open(name))

Code 2:

import os
for name in os.listdir("."):
    f=open(name)

Code 1 can cause the error but code 2 doesn't.

[Jonathan Thomas (jonoomph)]

Can someone add the exact steps involved in reproducing this. For example, which animated title, how many animated titles, which clips on the timeline, and how you are playing the video (i.e. preview or timeline)? Thanks!

[IRIE Shinsuke (irie)]

Under UNIX system, maximum number of open file descriptors is 1024 by default.

Therefore, the problem should be reproduced by playing "Snow" 2 times, on the time line. This means loading 900*2=1800 frames and exceeding the limit.

[Jonathan Thomas (jonoomph)]

IRIE,
I am using the latest trunk version of MLT on a Ubuntu 10.04 system, running the latest version of OpenShot (of course), and I can not reproduce this error. I created 2 different image sequences of snow, added the 2 clips to my timeline, back to back, and have been playing them repeatability for the past 20 minutes. It works without any issues.

Can you give this a try. Run these commands, and see if you notice anything of significance.

Step 1) Run $ lsof | grep "openshot" > ~/Desktop/openfiles_before.txt
Step 2) Run openshot, and reproduce the "too many files" error
Step 3) While openshot is experiencing the issue, open up a new terminal, and run:
   lsof | grep "openshot" > ~/Desktop/openfiles_after.txt

My only thought was perhaps some library or MLT itself has already fixed this issue, and you are using one of those older libraries. Of course, that is just a theory. =)

Thanks,
-Jonathan

[IRIE Shinsuke (irie)]

Probably this problem is MLT's bug. I made an image sequence containing 1800 PNG images and executed melt command as:

$ melt TitleFileName%04d.png

Then, the screen went white at the frame 1024.

I use libmlt2 0.5.6+git20100727-1.

[Jonathan Thomas (jonoomph)]

Yeah, that seems pretty conclusive that the bug is coming from MLT. I would feel better if we confirmed that it's fixed in newer versions of MLT. So, hopefully someone (other than my 1 test) can confirm this on a newer version of MLT. It might also be related to the version of FFmpeg, as MLT uses that quite heavily.

[IRIE Shinsuke (irie)]

I upgraded MLT to 0.6.0, which is included in Ubuntu 11.04, and confirmed the problem no longer happens.

[Jonathan Thomas (jonoomph)]

Marking this ticket as invalid, as it was discovered the bug was really in the MLT library and has already been fixed in the latest version. Thanks!

[Roger Binns (ubuntu-rogerbinns)]

I'm hitting something similar presumably with the same cause. The blender snow with ~900 images seems to do the trick. I get the traceback when trying to open the animated titles menu again.

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/openshot/windows/MainGTK.py", line 1659, in on_mnu3dTitle_activate
  File "/usr/lib/pymodules/python2.6/openshot/windows/BlenderGenerator.py", line 34, in __init__
  File "/usr/lib/pymodules/python2.6/openshot/windows/SimpleGtkBuilderApp.py", line 32, in __init__
glib.GError: Failed to open file '/usr/lib/pymodules/python2.6/openshot/windows/ui/BlenderGenerator.ui': Too many open files

Ubuntu 10.10 AMD64 OpenShot 1.3.0 from PPA.