I noticed a subtle issue that we may just want to ignore…. at least, I cannot fix it now.
If the subprocess calls an index helper app that is not installed, and gives the subprocess error, then one file handler is leaked.
I noticed this because I increased the number of files to be created from samplegen, and I had wv missing. This triggered the issue. The samplegen does not run to the end, but gives the error you see below. If I installed wv, the error went away so apparently the link happens then the subprocess gets the exception.
During normal operation however you don't create so many files in one tx, _plus_ it is expected that you have the helper apps installed, so this is unlikely to occur…. however if on some server the helper is missing, this could break samplegen surprisingly.
2012-04-21 16:51:11,861 ERROR karl.content.models.adapters: Error converting file /var/folders/1g/ht7yds5x72jgn7ylgb50rf_80000gn/T/tmpPTROQD
Traceback (most recent call last):
File "/Users/ree/work/osi/bc/ux2-grid2/src/karl/karl/content/models/adapters.py", line 116, in _extract_file_data
mimetype=context.mimetype)
File "/Users/ree/work/osi/bc/ux2-grid2/src/karl/karl/utilities/converters/pdf.py", line 34, in convert
return self.execute('pdftotext -enc UTF-8 "%s" -' % filename), 'utf-8'
File "/Users/ree/work/osi/bc/ux2-grid2/src/karl/karl/utilities/converters/baseconverter.py", line 54, in execute
close_fds=True)
File "/Users/ree/work/python/python3/parts/opt/lib/python2.7/subprocess.py", line 672, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
File "/Users/ree/work/python/python3/parts/opt/lib/python2.7/subprocess.py", line 1035, in _get_handles
p2cread, p2cwrite = os.pipe()
OSError: [Errno 24] Too many open files
I'll let Chris determine priority and scheduling based on severity.