Fuel 9.0 (task-based deployment is enabled by default there)
Environment:
- 3 controllers + mongo + cinder
- 1 compute node
- Ceilometer enabled
On in fact secondary Mongo node (node-2, IP 10.109.11.7) Puppet claims about this error:
/Stage[main]/Mongodb::Replset/Mongodb_replset[ceilometer] (err): Could not evaluate: rs.initiate() failed for replicaset ceilometer: not authorized on admin to execute command { replSetInitiate: { _id: "ceilometer", members: [ { _id: 0.0, host: "10.109.11.6:27017" }, { _id: 1.0, host: "10.109.11.7:27017" } ] } }
/etc/puppet/modules/mongodb/lib/puppet/provider/mongodb_replset/mongodb.rb:265:in `set_members'
/etc/puppet/modules/mongodb/lib/puppet/provider/mongodb_replset/mongodb.rb:62:in `flush'
...
2016-02-24 03:07:27 +0000 Puppet (warning): Impossible to get users info from slave
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb_user[admin] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb_user[admin] (warning): Skipping because of failed dependencies
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb configuring ceilometer database] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb configuring ceilometer database] (warning): Skipping because of failed dependencies
2016-02-24 03:07:27 +0000 Puppet (warning): Impossible to get databases from slave
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb::Db[ceilometer]/Mongodb_database[ceilometer] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb::Db[ceilometer]/Mongodb_database[ceilometer] (warning): Skipping because of failed dependencies
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb::Db[ceilometer]/Mongodb_user[ceilometer] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Mongodb::Db[ceilometer]/Mongodb_user[ceilometer] (warning): Skipping because of failed dependencies
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb finished] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:07:27 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb finished] (warning): Skipping because of failed dependencies
(see full puppet.log in attachment)
node-2 rs.status():
MongoDB shell version: 2.6.10
connecting to: test
ceilometer:SECONDARY> rs.status()
{
"set" : "ceilometer",
"date" : ISODate("2016-02-24T04:08:55Z"),
"myState" : 2,
"syncingTo" : "10.109.11.6:27017",
"members" : [
{
"_id" : 0,
"name" : "10.109.11.6:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3690,
"optime" : Timestamp(1456283242, 1),
"optimeDate" : ISODate("2016-02-24T03:07:22Z"),
"lastHeartbeat" : ISODate("2016-02-24T04:08:54Z"),
"lastHeartbeatRecv" : ISODate("2016-02-24T04:08:54Z"),
"pingMs" : 1,
"electionTime" : Timestamp(1456283250, 1),
"electionDate" : ISODate("2016-02-24T03:07:30Z")
},
{
"_id" : 1,
"name" : "10.109.11.7:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 3703,
"optime" : Timestamp(1456283242, 1),
"optimeDate" : ISODate("2016-02-24T03:07:22Z"),
"self" : true
}
],
"ok" : 1
}
The another secondary mongo node (node-1, IP 10.109.11.4) contains this error:
2016-02-24 03:40:22 +0000 Scope(Class[main]) (notice): MODULAR: mongo.pp
2016-02-24 03:40:25 +0000 Puppet (notice): Compiled catalog for node-1.test.domain.local in environment production in 3.18 seconds
2016-02-24 03:40:33 +0000 Puppet (notice): MongoDB params: 127.0.0.110.109.11.4
2016-02-24 03:40:33 +0000 /Stage[main]/Openstack::Mongo/Notify[MongoDB params: 127.0.0.110.109.11.4]/message (notice): defined 'message' as 'MongoDB params: 127.0.0.110
.109.11.4'
2016-02-24 03:40:33 +0000 /Stage[main]/Mongodb::Replset/Mongodb_replset[ceilometer]/ensure (notice): created
2016-02-24 03:40:33 +0000 Puppet (warning): Host 10.109.11.6:27017 is available, but you are unauthorized because of authentication is enabled: true
2016-02-24 03:40:34 +0000 Puppet (warning): Host 10.109.11.7:27017 is available, but you are unauthorized because of authentication is enabled: true
2016-02-24 03:40:34 +0000 Puppet (warning): Host 10.109.11.6:27017 is available, but you are unauthorized because of authentication is enabled: true
2016-02-24 03:40:34 +0000 Puppet (warning): Host 10.109.11.7:27017 is available, but you are unauthorized because of authentication is enabled: true
2016-02-24 03:40:34 +0000 /Stage[main]/Mongodb::Replset/Mongodb_replset[ceilometer] (err): Could not evaluate: rs.initiate() failed for replicaset ceilometer: couldn't
initiate : member 10.109.11.6:27017 is already initiated
/etc/puppet/modules/mongodb/lib/puppet/provider/mongodb_replset/mongodb.rb:265:in `set_members'
/etc/puppet/modules/mongodb/lib/puppet/provider/mongodb_replset/mongodb.rb:62:in `flush'
...
2016-02-24 03:40:34 +0000 Puppet (warning): Impossible to get users info from slave
2016-02-24 03:40:34 +0000 /Stage[main]/Openstack::Mongo/Mongodb_user[admin] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:40:34 +0000 /Stage[main]/Openstack::Mongo/Mongodb_user[admin] (warning): Skipping because of failed dependencies
2016-02-24 03:40:34 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb configuring ceilometer database] (notice): Dependency Mongodb_replset[ceilometer] has failures: true
2016-02-24 03:40:34 +0000 /Stage[main]/Openstack::Mongo/Notify[mongodb configuring ceilometer database] (warning): Skipping because of failed dependencies
node-1 (ip 10.109.11.4) rs.status():
> rs.status()
{
"startupStatus" : 3,
"info" : "run rs.initiate(...) if not yet done for the set",
"ok" : 0,
"errmsg" : "can't get local.system.replset config from self or any seed (EMPTYCONFIG)"
}
At the same time there is no mongo-related errors in PRIMARY mongo node (node-3, IP 10.109.11.6) puppet.log
node-3 rs.status():
ceilometer:PRIMARY> rs.status()
{
"set" : "ceilometer",
"date" : ISODate("2016-02-24T04:10:19Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "10.109.11.6:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 3793,
"optime" : Timestamp(1456283242, 1),
"optimeDate" : ISODate("2016-02-24T03:07:22Z"),
"electionTime" : Timestamp(1456283250, 1),
"electionDate" : ISODate("2016-02-24T03:07:30Z"),
"self" : true
},
{
"_id" : 1,
"name" : "10.109.11.7:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 3777,
"optime" : Timestamp(1456283242, 1),
"optimeDate" : ISODate("2016-02-24T03:07:22Z"),
"lastHeartbeat" : ISODate("2016-02-24T04:10:18Z"),
"lastHeartbeatRecv" : ISODate("2016-02-24T04:10:18Z"),
"pingMs" : 1,
"syncingTo" : "10.109.11.6:27017"
}
],
"ok" : 1
}
NOTE: this bug is not reproduced if task-based deployment engine is disabled
possible a duplicate of https:/