db sync for ceilometer fails when mongo role is with cinder

[Tatyanka]

{"build_id": "2014-04-22_01-00-26", "mirantis": "yes", "build_number": "121", "nailgun_sha": "978d70312250b72af9d29390f13ce7df8907c05d", "production": "prod", "ostf_sha": "7d0ea20e1c450a5f571d91bb03141b12c301184b", "fuelmain_sha": "8a9c73e7deaa8e6d8cf37deaa4f8c3314a7e0dbc", "astute_sha": "421a9f28080f8d10b4d006ce06af8568cce40b9b", "release": "5.0", "fuellib_sha": "28d82c796081978ef937853dafbe489a251d8103"}

Steps:
1.Centos with Neutron CRE with ceilometer, sahara and murano, simole mode
2. 1 controller + 1 compute + mongo with cinder on dedicated node
3. Start deployment
4. Run ostf

Expected result:
ostf tests passed

Actual:
ceilometer tests failed
Errors in puppet that ceilometer can not sync db
manual execution of sync lead to the next error:
[root@node-4 ~]# ceilometer-dbsync --debug --config-file=/etc/ceilometer/ceilometer.conf
ceilometer-ceilometer.storage.pymongo_base INFO: Connecting to mongodb on [('10.108.2.4', 27017)]
ceilometer-ceilometer.storage.pymongo_base INFO: Connecting to mongodb on [('10.108.2.4', 27017)]
ceilometer-ceilometer CRITICAL: ConnectionFailure: could not connect to 10.108.2.4:27017: timed out
Traceback (most recent call last):
  File "/usr/bin/ceilometer-dbsync", line 10, in <module>
    sys.exit(storage_dbsync())
  File "/usr/lib/python2.6/site-packages/ceilometer/cli.py", line 112, in storage_dbsync
    storage.get_connection(cfg.CONF).upgrade()
  File "/usr/lib/python2.6/site-packages/ceilometer/storage/__init__.py", line 87, in get_connection
    return get_engine(conf).get_connection(conf)
  File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 82, in get_connection
    return Connection(conf)
  File "/usr/lib/python2.6/site-packages/ceilometer/storage/impl_mongodb.py", line 428, in __init__
    self.conn = self.CONNECTION_POOL.connect(url)
  File "/usr/lib/python2.6/site-packages/ceilometer/storage/pymongo_base.py", line 124, in connect
    safe=True)
  File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 337, in __init__
    raise ConnectionFailure(str(e))
ConnectionFailure: could not connect to 10.108.2.4:27017: timed out

[root@nailgun ~]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online
---|--------|------------------|---------|------------|-------------------|---------------|---------------|-------
4 | ready | Untitled (61:15) | 2 | 10.108.0.3 | e6:66:45:1a:f7:45 | controller | | True
5 | ready | Untitled (e2:fe) | 2 | 10.108.0.4 | 0a:e5:0d:d3:4c:48 | compute | | True
6 | ready | Untitled (1c:75) | 2 | 10.108.0.5 | 72:23:6c:3e:1c:4d | cinder, mongo | | True

[Dmitry Ilyin (idv1985)]

It looks very strange because I dont't see how these two could be related... This bug should be verified and investigated.

[Vladimir Kuklin (vkuklin)]

looks like connectivity issue between nodes - we need a reproducer

[Tatyanka (tatyana-leontovich)]

Vova I'm in progress to reproduce on 133 iso, so back to you later with updates

[Tatyanka (tatyana-leontovich)]

issue reproduce on 133 iso. Btw with connectivity between nodes is fine, according nova tests passed(I believe if there is not connectivity instance creation should fail too)

[Dmitry Ilyin (idv1985)]

Mongo was not installed at all because there was no deployment defined for primary-mongo role in simple mode.
This patch https://review.openstack.org/#/c/89665/4 seems to have fixed this bug.