Comment 10 for bug 1544973

Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :

I figured it is a problem in our tests. 'crm configure show' on the affected env shows the following rules for RabbitMQ:

location location-p_rabbitmq-server-1 p_rabbitmq-server \
        rule $role=master -inf: #uname ne node-2.test.domain.local
location master_p_rabbitmq-server-on-node-1.test.domain.local master_p_rabbitmq-server 100: node-1.test.domain.local
location master_p_rabbitmq-server-on-node-2.test.domain.local master_p_rabbitmq-server 100: node-2.test.domain.local
location master_p_rabbitmq-server-on-node-3.test.domain.local master_p_rabbitmq-server 100: node-3.test.domain.local

Pay attention to the top rule, it didn't come from deployment or Pacemaker itself. It was set in the tests here:
https://github.com/openstack/fuel-qa/blob/7c70499309ec4882480963eed5bbd9d9975a6a8b/fuelweb_test/tests/tests_strength/test_failover_base.py#L1365-L1368

Just a little below the rule is deleted and seems like it generally works. But this time two duplicate rules were created with names location-p_rabbitmq-server and location-p_rabbitmq-server-1. While the first one was successfully deleted, the second remained and caused the tests to fail.

Attached are 5 cib files demonstrating the changes in the rules. In cib-49.raw location-p_rabbitmq-server was added. Then in cib-50.raw location-p_rabbitmq-server-1 was added. In cib-51.raw the first rule was deleted but the second remained.