Thanks Andrew, I see you routed the porblem in ceph area. Do you think is someone starting the investigation? In my opinion ceph mon is active on the remainig 2 nodes , but in any case ceph is not accessible. [root@node-15 ceph]# ceph -s 2014-01-13 09:02:26.467554 7f5574237700 0 -- :/1002881 >> 192.168.121.3:6789/0 pipe(0x7f556800a480 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f556800a6e0).fault 2014-01-13 09:02:29.468596 7f5574136700 0 -- :/1002881 >> 192.168.121.3:6789/0 pipe(0x7f556800ad80 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f5568002f50).fault [root@node-15 ceph]# ps aux | grep ceph root 2223 0.2 0.8 787920 143404 ? Ssl Jan10 11:45 /usr/bin/ceph-osd -i 13 --pid-file /var/run/ceph/osd.13.pid -c /etc/ceph/ceph.conf root 3780 0.2 0.4 713348 71992 ? Ssl Jan10 11:28 /usr/bin/ceph-osd -i 16 --pid-file /var/run/ceph/osd.16.pid -c /etc/ceph/ceph.conf root 6541 0.2 1.3 867784 216584 ? Ssl Jan10 12:38 /usr/bin/ceph-osd -i 17 --pid-file /var/run/ceph/osd.17.pid -c /etc/ceph/ceph.conf root 41060 0.0 0.0 103240 884 pts/0 S+ 08:56 0:00 grep ceph root 42717 0.0 0.2 253716 49184 ? Sl Jan10 0:58 /usr/bin/ceph-mon -i node-15 --pid-file /var/run/ceph/mon.node-15.pid -c /etc/ceph/ceph.conf root 42870 0.3 0.8 791888 147336 ? Ssl Jan10 12:57 /usr/bin/ceph-osd -i 11 --pid-file /var/run/ceph/osd.11.pid -c /etc/ceph/ceph.conf root 44785 0.3 0.9 802136 150416 ? Ssl Jan10 14:32 /usr/bin/ceph-osd -i 8 --pid-file /var/run/ceph/osd.8.pid -c /etc/ceph/ceph.conf root 47618 0.2 1.0 806588 166588 ? Ssl Jan10 12:25 /usr/bin/ceph-osd -i 15 --pid-file /var/run/ceph/osd.15.pid -c /etc/ceph/ceph.conf According to the following output the system in unsusable: DHCP and L3 agent do not works: [root@node-15 ceph]# crm status Last updated: Mon Jan 13 08:41:13 2014 Last change: Fri Jan 10 16:38:49 2014 via crmd on node-14.prisma Stack: openais Current DC: node-15.prisma - partition with quorum Version: 1.1.8-1.el6-1f8858c 3 Nodes configured, 3 expected votes 17 Resources configured. Online: [ node-14.prisma node-15.prisma ] OFFLINE: [ node-13.prisma ] vip__management_old (ocf::heartbeat:IPaddr2): Started node-14.prisma vip__public_old (ocf::heartbeat:IPaddr2): Started node-15.prisma Clone Set: clone_p_haproxy [p_haproxy] Started: [ node-14.prisma node-15.prisma ] Stopped: [ p_haproxy:2 ] Clone Set: clone_p_mysql [p_mysql] Started: [ node-14.prisma node-15.prisma ] Stopped: [ p_mysql:2 ] Clone Set: clone_p_neutron-openvswitch-agent [p_neutron-openvswitch-agent] Started: [ node-14.prisma node-15.prisma ] Stopped: [ p_neutron-openvswitch-agent:2 ] Clone Set: clone_p_neutron-metadata-agent [p_neutron-metadata-agent] Started: [ node-14.prisma node-15.prisma ] Stopped: [ p_neutron-metadata-agent:2 ] p_neutron-dhcp-agent (ocf::mirantis:neutron-agent-dhcp): Started node-15.prisma (unmanaged) FAILED p_neutron-l3-agent (ocf::mirantis:neutron-agent-l3): Started node-14.prisma (unmanaged) FAILED openstack-heat-engine (ocf::mirantis:openstack-heat-engine): Started node-14.prisma