W/o this patch, rabbitmqctl commands issued by rabbitmq RA
could outlast the given timeout for action notify (60 seconds).
That may bring rabbitmq cluster to a state then join_cluster
and start_app commands would never ended as well as the
cluster reassemble process.
The solution is:
* Use limited interval for rabbitmqctl commands which depends on the
OCF action in progress. Looks like start and notify actions require
the most of the time to be finished.
* Evaluate interval as (timeout / 6 + 5).
So, for notify 180 sec timeout it would be 180/6+5=35 sec
to wait before any command issued by action notify would be killed
* Adjust timeout values for failure-timeout, start and notify 60->180.
That is required in order to provide enough time for the
commands to complete consequently w/o being killed by timeout.
* Fix start/shutdown timeout evaluation then recieved a negative value for
small timeouts (timeout=60s would result in -4 value)
Reviewed: https:/ /review. openstack. org/140092 /git.openstack. org/cgit/ stackforge/ fuel-library/ commit/ ?id=d18ccea5791 aae9e9003f62f9e 4ba0ddbce684cf
Committed: https:/
Submitter: Jenkins
Branch: master
commit d18ccea5791aae9 e9003f62f9e4ba0 ddbce684cf
Author: Bogdan Dobrelya <email address hidden>
Date: Mon Dec 8 17:48:08 2014 +0100
Wrap OCF rabbitmq commands to timeout -KILL
W/o this patch, rabbitmqctl commands issued by rabbitmq RA
could outlast the given timeout for action notify (60 seconds).
That may bring rabbitmq cluster to a state then join_cluster
and start_app commands would never ended as well as the
cluster reassemble process.
The solution is:
* Use limited interval for rabbitmqctl commands which depends on the
OCF action in progress. Looks like start and notify actions require
the most of the time to be finished.
* Evaluate interval as (timeout / 6 + 5).
So, for notify 180 sec timeout it would be 180/6+5=35 sec
to wait before any command issued by action notify would be killed
* Adjust timeout values for failure-timeout, start and notify 60->180.
That is required in order to provide enough time for the
commands to complete consequently w/o being killed by timeout.
* Fix start/shutdown timeout evaluation then recieved a negative value for
small timeouts (timeout=60s would result in -4 value)
Related-bug: #1396946
Change-Id: I33a0390b311646 266522ddd0f4e8c 75d762afe30
Signed-off-by: Bogdan Dobrelya <email address hidden>