Job revocation (CTRL-C) will stop also other jobs

[Daniele Viganò]

When CTRL-C is pressed OpenQuake master will send a revoke command to all the workers. This is causing an automatic restart of celery which will lead to a failure on other jobs running on the same workers.

[Daniele Viganò (daniele-vigano)]

Probably we are using 'terminate=True' and/or with the wrong SIGNAL.
http://docs.celeryproject.org/en/2.4/userguide/workers.html#revoking-tasks

[Michele Simionato (michele-simionato)]

Yes, we are using terminate=True, see ./openquake/engine/supervising/supervisor.py

[Daniele Viganò (daniele-vigano)]

As the default sign sent is TERM we are soft-killing the celery processes. This will break communication between RabbitMQ/AMPQ and celery causing a calculation failure.

[Michele Simionato (michele-simionato)]

Fixed in https://github.com/gem/oq-engine/pull/1303 (but we would need a better solution)