Comment 20 for bug 1373569

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

@Tyler, the suggested fix affects blocking rule logic and should be changed as appropriate:
By design, an iptables blocking rule should persist from the moment then start_rmq_server_app() starts and unless it exits.

Putting unblock in the middle of this function, as my initial patch suggests, is a poor workaround breaking the block/unblock implementation.

The proper one, perhaps, would be to put unblock right into the start, just before to apply the blocking rule as well.

I analyzed the logs attached and found the "moment of truth":
2014-09-24T01:42:30.328672 node-69 ./node-69.lax3.ubiquity.io/rabbitmq-server.log:2014-09-24T01:42:30.328672+01:00 info: INFO: p_rabbitmq-server: start_rmq_server_app(): begin.
(action failed by timeout, there is no "end." event, hence, iptables blocking rule was not removed!)
2014-09-24T01:45:48.730049 node-69 ./node-69.lax3.ubiquity.io/rabbitmq-server.log:2014-09-24T01:45:48.730049+01:00 info: INFO: p_rabbitmq-server: start_rmq_server_app(): begin.
(2nd blocking rule added...)

If we apply the suggested solution - put "safe" unblock prior to the blocking, that would resolve the idempotency issue.
There is an another way - make block and unblock calls idempotent by checking if the rule already exists, but I guess it would complicate the OCF script logic even more and make it more error prone... Anyway, the "safe unblock" solutions works as well.