Deployment fails with 3 Ceph mon
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
kolla-ansible |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Overview
Hi
I'm trying to deploy OpenStack with three controllers nodes. But I'm facing an issue went kolla bootstrap Ceph container. The deployment fails and OSD are not correctly created
Steps to reproduce
Deployment with three controllers nodes (ceph enabled)
Actual results
I have found those logs :
-> on compute
*ceph-
monclient(hunting): authenticate timed out after 300
librados: client.admin authentication error (110) Connection timed out
* docker ps -a :
aa48da67a3c7 private-
6d12c93c668b private-
100e9f6c436b private-
6f82eb8ae898 private-
6b909d87ced2 private-
819914f446fb private-
* docker logs bootstrap_osd_3
INFO:main:Loading config file at /var/lib/
INFO:main:
INFO:main:Kolla config strategy set to: COPY_ALWAYS
INFO:main:Copying service configuration files
INFO:main:Copying /var/lib/
INFO:main:Setting permission for /etc/ceph/ceph.conf
INFO:main:Copying /var/lib/
INFO:main:Setting permission for /etc/ceph/
INFO:main:Writing out command to execute
Error connecting to cluster: TimedOut
-> On controller ceph-mon.
2017-12-28 11:09:21.206387 7f116aeba700 0 log_channel(
2017-12-28 11:09:21.206487 7f116aeba700 1 mon.X.X.
2017-12-28 11:09:36.237530 7f116aeba700 0 log_channel(
2017-12-28 11:09:36.237600 7f116aeba700 1 mon.X.X.
2017-12-28 11:09:51.276675 7f116aeba700 0 log_channel(
2017-12-28 11:09:51.276774 7f116aeba700 1 mon.X.X.
2017-12-28 11:10:06.313344 7f116aeba700 0 log_channel(
2017-12-28 11:10:06.313438 7f116aeba700 1 mon.X.X.
2017-12-28 11:10:18.817119 7f116b6bb700 0 mon.X.X.
2017-12-28 11:10:21.350803 7f116aeba700 0 log_channel(
2017-12-28 11:10:21.350913 7f116aeba700 1 mon.X.X.
2017-12-28 11:10:36.385019 7f116aeba700 0 log_channel(
2017-12-28 11:10:36.385101 7f116aeba700 1 mon.X.X.
2017-12-28 11:10:51.424834 7f116aeba700 0 log_channel(
2017-12-28 11:10:51.424952 7f116aeba700 1 mon.X.X.
2017-12-28 11:11:06.459091 7f116aeba700 0 log_channel(
2017-12-28 11:11:06.459160 7f116aeba700 1 mon.X.X.
2017-12-28 11:11:18.817242 7f116b6bb700 0 mon.X.X.
2017-12-28 11:11:21.499690 7f116aeba700 0 log_channel(
2017-12-28 11:11:21.499799 7f116aeba700 1 mon.X.X.
2017-12-28 11:11:36.534253 7f116aeba700 0 log_channel(
2017-12-28 11:11:36.534364 7f116aeba700 1 mon.X.X.
2017-12-28 11:11:51.577507 7f116aeba700 0 log_channel(
2017-12-28 11:11:51.577589 7f116aeba700 1 mon.X.X.
2017-12-28 11:12:06.613963 7f116aeba700 0 log_channel(
2017-12-28 11:12:06.614036 7f116aeba700 1 mon.X.X.
2017-12-28 11:12:18.817438 7f116b6bb700 0 mon.X.X.
2017-12-28 11:12:21.654153 7f116aeba700 0 log_channel(
2017-12-28 11:12:21.654244 7f116aeba700 1 mon.X.X.
2017-12-28 11:12:36.696281 7f116aeba700 0 log_channel(
2017-12-28 11:12:36.696370 7f116aeba700 1 mon.X.X.
2017-12-28 11:12:51.736531 7f116aeba700 0 log_channel(
2017-12-28 11:12:51.736625 7f116aeba700 1 mon.X.X.
2017-12-28 11:13:06.771501 7f116aeba700 0 log_channel(
2017-12-28 11:13:06.771587 7f116aeba700 1 mon.X.X.
2017-12-28 11:13:18.817649 7f116b6bb700 0 mon.X.X.
2017-12-28 11:13:21.810869 7f116aeba700 0 log_channel(
2017-12-28 11:13:21.810984 7f116aeba700 1 mon.X.X.
2017-12-28 11:13:36.851396 7f116aeba700 0 log_channel(
2017-12-28 11:13:36.851495 7f116aeba700 1 mon.X.X.
2017-12-28 11:13:51.890575 7f116aeba700 0 log_channel(
2017-12-28 11:13:51.890661 7f116aeba700 1 mon.X.X.
2017-12-28 11:14:06.930553 7f116aeba700 0 log_channel(
2017-12-28 11:14:06.930642 7f116aeba700 1 mon.X.X.
2017-12-28 11:14:18.817921 7f116b6bb700 0 mon.X.X.
2017-12-28 11:14:21.971071 7f116aeba700 0 log_channel(
2017-12-28 11:14:21.971178 7f116aeba700 1 mon.X.X.
2017-12-28 11:14:37.012441 7f116aeba700 0 log_channel(
2017-12-28 11:14:37.012516 7f116aeba700 1 mon.X.X.
2017-12-28 11:14:52.057103 7f116aeba700 0 log_channel(
2017-12-28 11:14:52.057181 7f116aeba700 1 mon.X.X.
2017-12-28 11:15:07.092539 7f116aeba700 0 log_channel(
2017-12-28 11:15:07.092627 7f116aeba700 1 mon.X.X.
2017-12-28 11:15:18.818098 7f116b6bb700 0 mon.X.X.
2017-12-28 11:15:22.124296 7f116aeba700 0 log_channel(
2017-12-28 11:15:22.124368 7f116aeba700 1 mon.X.X.
2017-12-28 11:15:37.162572 7f116aeba700 0 log_channel(
2017-12-28 11:15:37.162664 7f116aeba700 1 mon.X.X.
-> on Kayobe node (during kayobe overcloud service deploy )
fatal: [ctrl03 -> X.X.X.X]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "ceph_mon", "ceph", "auth", "get-or-create", "client.glance", "mon", "allow r", "osd", "allow class-read object_prefix rbd_children, allow rwx pool=images, allow rwx pool=images-
PS : I've hound similar issue here https:/
Thank you in advance for your help
Changed in kolla-ansible: | |
status: | New → Won't Fix |
Hi,
I've found a workaround, but I still not know why my change makes it work.
In fact, I'm using a deployment tools calls Kayobe which enable me to easily deploy OpenStack.
An inventory file is generated automatically, and the deployment is based on it.
Then went I change the node order (list of node) the deployment work fine, but if i change it no deployment no longer works.
I'm thinking that Kolla try to bootstrap Ceph cluster using predefined order (in ceph.conf, with the list of node) which is not the real order (the one provided by my inventory file).
Somebody has an explanation
Regards
Dirane TAFEN