[library] Controller addition failed: ceph osd pool create images 512 returned 1 instead of one of [0]

[Artem Panchenko]

api: '1.0'
astute_sha: 8e1db3926b2320b30b23d7a772122521b0d96166
auth_required: true
build_id: 2014-08-18_11-13-09
build_number: '449'
feature_groups:
- mirantis
fuellib_sha: 2c9ad4aec9f3b6fc060cb5a394733607f07063c1
fuelmain_sha: 08f04775dcfadd8f5b438a31c63e81f29276b7d3
nailgun_sha: bc9e377dbe010732bc2ba47161ed9d433998e07b
ostf_sha: d2a894d228c1f3c22595a77f04b1e00d09d8e463
production: docker
release: '5.1'

Here is the part of Puppet logs:

http://paste.openstack.org/show/97324/

I guess that issue was caused by misconfiguration of Ceph keys on new controller:

[root@node-4 ~]# ceph osd pool create images 512; echo $?
2014-08-19 14:01:12.453900 7feaa0942700 0 librados: client.admin authentication error (1) Operation not permitted
Error connecting to cluster: PermissionError
1

[root@node-6 ~]# scp /etc/ceph/ceph.client.admin.keyring node-4:/etc/ceph/
ceph.client.admin.keyring 100% 63 0.1KB/s 00:00
[root@node-6 ~]# scp /var/lib/ceph/bootstrap-osd/ceph.keyring node-4:/var/lib/ceph/bootstrap-osd
ceph.keyring 100% 71 0.1KB/s 00:00

[root@node-4 ~]# ceph osd pool create images 512; echo $?
pool 'images' already exists
0

Unfortunately, I can't attach diagnostic snapshot to the bug because it's too big, but I uploaded full puppet log from failed node. Also, I can provide someone, who can figure out the cause of the issue, with access to the environment.