OpenStack services can't find rabbitmq queues
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mirantis OpenStack |
Fix Released
|
High
|
MOS Oslo | ||
5.1.x |
Invalid
|
High
|
MOS Maintenance | ||
6.0.x |
Invalid
|
High
|
MOS Maintenance | ||
6.1.x |
Invalid
|
High
|
MOS Maintenance | ||
7.0.x |
Fix Released
|
High
|
MOS Oslo |
Bug Description
During create_
from rally.log: http://
from haproxy:
<134>Jul 7 17:33:26 node-13 haproxy[16037]: 172.16.44.13:55788 [07/Jul/
from nova-all.log on node-13: http://
The same errors on node-8 at the time: http://
from rabbitmq.log on node-8: http://
from rabbitmq.log on node-13: http://
from rabbitmq.log on node-15 (no messages at the time): http://
Similar from cinder-all on node-13: http://
Similar from cinder-all on node-8: http://
Similar from cinder-all on node-15: http://
pacemaker on node-8:
Jul 07 17:31:27 [26773] node-8.domain.tld pengine: info: master_color: Promoting p_rabbitmq-server:2 (Master node-13.domain.tld)
Jul 07 17:31:27 [26773] node-8.domain.tld pengine: info: master_color: master_
...
Jul 07 17:32:05 [26773] node-8.domain.tld pengine: info: master_color: Promoting p_rabbitmq-server:2 (Master node-13.domain.tld)
Jul 07 17:32:05 [26773] node-8.domain.tld pengine: info: master_color: master_
changing rabbit master:
Jul 07 11:47:21 [26773] node-8.domain.tld pengine: info: master_color: Promoting p_rabbitmq-server:0 (Master node-8.domain.tld)
Jul 07 11:47:21 [26773] node-8.domain.tld pengine: info: master_color: master_
Jul 07 11:48:27 [26773] node-8.domain.tld pengine: info: master_color: Promoting p_rabbitmq-server:2 (Slave node-13.domain.tld)
atop at 07 17:33 on node-8: http://
atop at 07 17:33 on node-13: http://
atop at 07 17:33 on node-15: http://
Diagnostic Snapshot is here: http://
on each controller node during the tests we kept rabbitmq statistics by http:// paste.openstack .org/show/ 355156/
This info+logs are attached