Comment 6 for bug 1314514

Hello! I've reproduced this issue on 6.1 build:

[root@fuel ~]# fuel --fuel-version
api: '1.0'
astute_sha: d81ff53c2f467151ecde120d3a4d284e3b5b3dfc
auth_required: true
build_id: 2015-02-23_22-54-44
build_number: '140'
feature_groups:
- mirantis
fuellib_sha: 8b79d47ef41bff293210d2a7b1bb02843f70948d
fuelmain_sha: b975019fabdb429c1869047df18dd792d2163ecc
nailgun_sha: 3616ae9df4ac3e088157bb94f73743a521f76f1a
ostf_sha: 1a0b2c6618fac098473c2ed5a9af11d3a886a3bb
production: docker
python-fuelclient_sha: 5657dbf06fddb74adb61e9668eb579a1c57d8af8
release: '6.1'
release_versions:
  2014.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: d81ff53c2f467151ecde120d3a4d284e3b5b3dfc
      build_id: 2015-02-23_22-54-44
      build_number: '140'
      feature_groups:
      - mirantis
      fuellib_sha: 8b79d47ef41bff293210d2a7b1bb02843f70948d
      fuelmain_sha: b975019fabdb429c1869047df18dd792d2163ecc
      nailgun_sha: 3616ae9df4ac3e088157bb94f73743a521f76f1a
      ostf_sha: 1a0b2c6618fac098473c2ed5a9af11d3a886a3bb
      production: docker
      python-fuelclient_sha: 5657dbf06fddb74adb61e9668eb579a1c57d8af8
      release: '6.1'

Baremetal, Centos, HA, Neutron-gre,Ceilometer,Ceph-all, Debug, nova-quotas, 6.1_140
Controllers:3 Computes:47

Deployment has been failed.

From astute log:
[424] Task '{"priority"=>1100, "type"=>"puppet", "uids"=>["1"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/openstack-controller/openstack-controller.pp", "timeout"=>3600, "cwd"=>"/"}}' on node 1 valid, but failed

From puppet log of node-1 (controller node):
2015-02-25 08:00:35 ERR
 (/Stage[main]/Ceilometer::Db/Exec[ceilometer-dbsync]) Failed to call refresh: ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf returned 1 instead of one of [0]

If I try execute this command:
[root@node-1 ~]# ceilometer-dbsync --debug --config-file=/etc/ceilometer/ceilometer.conf
2015-02-25 09:48:25.283 23642 DEBUG ceilometer.storage [-] looking for 'mongodb' driver in 'ceilometer.metering.storage' get_connection /usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py:96
2015-02-25 09:48:25.360 23642 INFO ceilometer.storage.mongo.utils [-] Connecting to mongodb on [('192.168.0.3', 27017), ('192.168.0.46', 27017), ('192.168.0.51', 27017)]
2015-02-25 09:48:25.365 23642 CRITICAL ceilometer [-] ConfigurationError: No suitable hosts found
2015-02-25 09:48:25.365 23642 TRACE ceilometer Traceback (most recent call last):
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/bin/ceilometer-dbsync", line 10, in <module>
2015-02-25 09:48:25.365 23642 TRACE ceilometer sys.exit(dbsync())
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/cmd/storage.py", line 32, in dbsync
2015-02-25 09:48:25.365 23642 TRACE ceilometer storage.get_connection_from_config(cfg.CONF, 'metering').upgrade()
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 86, in get_connection_from_config
2015-02-25 09:48:25.365 23642 TRACE ceilometer return get_connection(url, namespace)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 98, in get_connection
2015-02-25 09:48:25.365 23642 TRACE ceilometer return mgr.driver(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 395, in __init__
2015-02-25 09:48:25.365 23642 TRACE ceilometer self.conn = self.CONNECTION_POOL.connect(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/mongo/utils.py", line 248, in connect
2015-02-25 09:48:25.365 23642 TRACE ceilometer client = self._mongo_connect(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/mongo/utils.py", line 259, in _mongo_connect
2015-02-25 09:48:25.365 23642 TRACE ceilometer replicaSet=cfg.CONF.database.mongodb_replica_set))
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib64/python2.6/site-packages/pymongo/mongo_replica_set_client.py", line 704, in __init__
2015-02-25 09:48:25.365 23642 TRACE ceilometer self.refresh()
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib64/python2.6/site-packages/pymongo/mongo_replica_set_client.py", line 1104, in refresh
2015-02-25 09:48:25.365 23642 TRACE ceilometer raise ConfigurationError('No suitable hosts found')
2015-02-25 09:48:25.365 23642 TRACE ceilometer ConfigurationError: No suitable hosts found
2015-02-25 09:48:25.365 23642 TRACE ceilometer

from ceilometer-dbsync.log:
2015-02-25 09:48:25.283 23642 DEBUG ceilometer.storage [-] looking for 'mongodb' driver in 'ceilometer.metering.storage' get_connection /usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py:96
2015-02-25 09:48:25.360 23642 INFO ceilometer.storage.mongo.utils [-] Connecting to mongodb on [('192.168.0.3', 27017), ('192.168.0.46', 27017), ('192.168.0.51', 27017)]
2015-02-25 09:48:25.365 23642 CRITICAL ceilometer [-] ConfigurationError: No suitable hosts found
2015-02-25 09:48:25.365 23642 TRACE ceilometer Traceback (most recent call last):
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/bin/ceilometer-dbsync", line 10, in <module>
2015-02-25 09:48:25.365 23642 TRACE ceilometer sys.exit(dbsync())
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/cmd/storage.py", line 32, in dbsync
2015-02-25 09:48:25.365 23642 TRACE ceilometer storage.get_connection_from_config(cfg.CONF, 'metering').upgrade()
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 86, in get_connection_from_config
2015-02-25 09:48:25.365 23642 TRACE ceilometer return get_connection(url, namespace)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 98, in get_connection
2015-02-25 09:48:25.365 23642 TRACE ceilometer return mgr.driver(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 395, in __init__
2015-02-25 09:48:25.365 23642 TRACE ceilometer self.conn = self.CONNECTION_POOL.connect(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/mongo/utils.py", line 248, in connect
2015-02-25 09:48:25.365 23642 TRACE ceilometer client = self._mongo_connect(url)
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib/python2.6/site-packages/ceilometer/storage/mongo/utils.py", line 259, in _mongo_connect
2015-02-25 09:48:25.365 23642 TRACE ceilometer replicaSet=cfg.CONF.database.mongodb_replica_set))
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib64/python2.6/site-packages/pymongo/mongo_replica_set_client.py", line 704, in __init__
2015-02-25 09:48:25.365 23642 TRACE ceilometer self.refresh()
2015-02-25 09:48:25.365 23642 TRACE ceilometer File "/usr/lib64/python2.6/site-packages/pymongo/mongo_replica_set_client.py", line 1104, in refresh
2015-02-25 09:48:25.365 23642 TRACE ceilometer raise ConfigurationError('No suitable hosts found')
2015-02-25 09:48:25.365 23642 TRACE ceilometer ConfigurationError: No suitable hosts found

telnet to each controller are ok
[root@node-1 ~]# telnet 192.168.0.3 27017
Trying 192.168.0.3...
Connected to 192.168.0.3.
Escape character is '^]'.

[root@node-1 ~]# telnet 192.168.0.46 27017
Trying 192.168.0.46...
Connected to 192.168.0.46.
Escape character is '^]'.

[root@node-1 ~]# telnet 192.168.0.51 27017
Trying 192.168.0.51...
Connected to 192.168.0.51.
Escape character is '^]'.

Rabbit has been installed only on first controller at the moment and it's ok:
[root@node-1 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@node-1' ...
[{nodes,[{disc,['rabbit@node-1']}]},
 {running_nodes,['rabbit@node-1']},
 {cluster_name,<<"<email address hidden>">>},
 {partitions,[]}]
...done.

[root@node-1 ~]# grep mongo /etc/ceilometer/ceilometer.conf
connection=mongodb://ceilometer:WoJqXRek@192.168.0.3,192.168.0.46,192.168.0.51/ceilometer
mongodb_replica_set=ceilometer

mongo is running on each controller and listen:

[root@node-1 ~]# ps aux | grep mongo
root 11220 0.0 0.0 6380 700 pts/0 S+ 09:56 0:00 grep mongo
mongodb 25272 0.3 0.1 781300 44536 ? Sl 07:49 0:29 /usr/bin/mongod --quiet -f /etc/mongodb.conf run

[root@node-1 ~]# netstat -antp | grep 27017
tcp 0 0 192.168.0.3:27017 0.0.0.0:* LISTEN 25272/mongod
tcp 0 0 127.0.0.1:27017 0.0.0.0:* LISTEN 25272/mongod