We think this is due to this code:
proc = subprocess.Popen("import_package.py %s" % self.package, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, preexec_fn=subprocess_setup) proc.stdin.close() while proc.poll() is None: if self.should_stop.isSet(): os.kill(proc.pid, signal.SIGTERM) time.sleep(1) output = proc.stdout.read()
if the subprocess writes lots of output then it will buffer and hang, and we won't consume until it ends, causing a deadlock.
We can't simply read() within the loop, as we don't want to block, so using select() is probably the quick fix for this.
It's another plus for using LP's job system/ampoule/celery/something though.
Thanks,
James
We think this is due to this code:
proc = subprocess. Popen(" import_ package. py %s" % self.package,
shell= True, stdout= subprocess. PIPE,
stderr= subprocess. STDOUT, stdin=subproces s.PIPE,
preexec_ fn=subprocess_ setup)
proc.stdin. close() stop.isSet( ):
os.kill( proc.pid, signal.SIGTERM)
time. sleep(1)
while proc.poll() is None:
if self.should_
output = proc.stdout.read()
if the subprocess writes lots of output then it will buffer and hang, and we won't consume
until it ends, causing a deadlock.
We can't simply read() within the loop, as we don't want to block, so using select() is
probably the quick fix for this.
It's another plus for using LP's job system/ ampoule/ celery/ something though.
Thanks,
James