Ceph env fails

[Andrey Korolyov]

Simple env, three OSD and one controller

ceph -s
  cluster bff99f2e-4daf-4653-9c30-22db84d55a7a
   health HEALTH_OK
   monmap e1: 1 mons at {node-4=192.168.0.2:6789/0}, election epoch 2, quorum 0 node-4
   osdmap e11: 2 osds: 2 up, 2 in
    pgmap v40: 492 pgs: 492 active+clean; 0 bytes data, 4169 MB used, 41260 MB / 45429 MB avail
   mdsmap e1: 0/0/1 up

third OSD node failed, check the logs

seemingly it`s our logic problem (no third osd, rerun of ceph-deploy)

[Andrew Woodward (xarses)]

From node-7 /root/ceph.log

2013-12-17 11:23:14,726 [ceph_deploy.cli][INFO ] Invoked (1.2.7): /usr/bin/ceph-deploy --overwrite-conf config pull node-4
2013-12-17 11:23:14,726 [ceph_deploy.config][DEBUG ] Checking node-4 for /etc/ceph/ceph.conf
2013-12-17 11:23:14,727 [ceph_deploy.sudo_pushy][DEBUG ] will use a remote connection without sudo
2013-12-17 11:23:17,733 [ceph_deploy.config][ERROR ] Unable to pull /etc/ceph/ceph.conf from node-4
2013-12-17 11:23:17,733 [ceph_deploy][ERROR ] GenericError: Failed to fetch config from 1 hosts

Implies broken network or failed ssh keys

[Dmitry Borodaenko (angdraug)]

Management network was misconfigured on node-7: it wasn't able to connect to the controller (node-4) at 192.168.0.2, so it failed to pull ceph.conf, blocking the rest of ceph-osd deployment.

[Dmitry Borodaenko (angdraug)]

node-7 was patched differently from other nodes: admin network is patched to eth2 instead of eth0, and OVS br-mgmt and br-storage bridges were assigned to eth1 instead of eth2.

[Mike Scherbakov (mihgen)]

Andrey,
your title is not descriptive enough. Please provide better title so we can understand what the bug is about reading just title.

[Andrey Korolyov (xdeller)]

It`s all about non-strict networking checking in Fuel. May be it`s necessary to check connectivity just after Deploy button was hit and throw a warning. Of course new modal window should be introduced.

[Andrew Woodward (xarses)]

I created https://bugs.launchpad.net/fuel/+bug/1261940 to help prevent this kind of issue.