Can't Configure Ceph with kolla-ansible

[Sabbir Sakib]

Description: The OpenStack ceph installation is failing with following error. Without ceph installation works just fine.

OS - CentOS-7.4

Error :

TASK [ceph : Copying over config.json files for services] ***********************************************************************************************************************************************************************************
ok: [oscontroller01.xyz.com] => (item={u'group': u'ceph-mon', u'name': u'ceph-mon'})
changed: [oshyp01.xyz.com] => (item={u'group': u'ceph-mon', u'name': u'ceph-mon'})
changed: [oshyp02.xyz.com] => (item={u'group': u'ceph-mon', u'name': u'ceph-mon'})
ok: [oshyp01.xyz.com] => (item={u'group': u'ceph-osd', u'name': u'ceph-osd'})
ok: [oscontroller01.xyz.com] => (item={u'group': u'ceph-osd', u'name': u'ceph-osd'})
ok: [oshyp02.xyz.com] => (item={u'group': u'ceph-osd', u'name': u'ceph-osd'})
changed: [oshyp01.xyz.com] => (item={u'group': u'ceph-rgw', u'name': u'ceph-rgw'})
fatal: [oshyp01.xyz.com]: FAILED! => {"msg": "The conditional check 'inventory_hostname in groups[item.group]' failed. The error was: error while evaluating conditional (inventory_hostname in groups[item.group]): Unable to look up a name or access an attribute in template string ({% if inventory_hostname in groups[item.group] %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/opt/kolla-ansible5/kolla-ansible/ansible/roles/ceph/tasks/config.yml': line 15, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Copying over config.json files for services\n ^ here\n"}
ok: [oscontroller01.xyz.com] => (item={u'group': u'ceph-rgw', u'name': u'ceph-rgw'})
fatal: [oscontroller01.xyz.com]: FAILED! => {"msg": "The conditional check 'inventory_hostname in groups[item.group]' failed. The error was: error while evaluating conditional (inventory_hostname in groups[item.group]): Unable to look up a name or access an attribute in template string ({% if inventory_hostname in groups[item.group] %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/opt/kolla-ansible5/kolla-ansible/ansible/roles/ceph/tasks/config.yml': line 15, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Copying over config.json files for services\n ^ here\n"}
changed: [oshyp02.xyz.com] => (item={u'group': u'ceph-rgw', u'name': u'ceph-rgw'})
fatal: [oshyp02.xyz.com]: FAILED! => {"msg": "The conditional check 'inventory_hostname in groups[item.group]' failed. The error was: error while evaluating conditional (inventory_hostname in groups[item.group]): Unable to look up a name or access an attribute in template string ({% if inventory_hostname in groups[item.group] %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/opt/kolla-ansible5/kolla-ansible/ansible/roles/ceph/tasks/config.yml': line 15, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Copying over config.json files for services\n ^ here\n"}
 to retry, use: --limit @/opt/kolla-ansible5/kolla-ansible/ansible/site.retry

PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0
oscontroller01.xyz.com : ok=82 changed=0 unreachable=0 failed=1
oshyp01.xyz.com : ok=72 changed=9 unreachable=0 failed=1
oshyp02.xyz.com : ok=72 changed=9 unreachable=0 failed=1

Partition label to the disk :

[root@oscontroller01 ~]# parted /dev/sdb print
Number Start End Size File system Name Flags
 1 1049kB 922GB 922GB xfs KOLLA_CEPH_OSD_BOOTSTRAP

[root@oshyp01 ~]# parted /dev/sdb print
Number Start End Size File system Name Flags
 1 1049kB 922GB 922GB KOLLA_CEPH_OSD_BOOTSTRAP

[root@oshyp02 ~]# parted /dev/sdb print
Number Start End Size File system Name Flags
 1 1049kB 922GB 922GB KOLLA_CEPH_OSD_BOOTSTRAP

cat /etc/kolla/multinode :

# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control]
# These hostname must be resolvable from your deployment host
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt
# The above can also be specified as follows:
#control[01:03] ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt

[inner-compute]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt

# external-compute is the groups of compute nodes which can reach
# # outside
[external-compute]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt
# compute01
#
[compute]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt

[monitoring]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt
# When compute nodes and control nodes use different interfaces,
# you can specify "api_interface" and other interfaces like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1

[storage]
oscontroller01.xyz.pvt
oshyp01.xyz.pvt
oshyp02.xyz.pvt

And enabled only following in global.yaml file

enable_ceph: "yes"
enable_ceph_rgw: "yes"
glance_backend_ceph: "yes"
nova_backend_ceph: "{{ enable_ceph }}"

[Sabbir Sakib (sakibsys)]

I'm using kolla-ansible v5.0

[Eduardo Gonzalez (egonzalez90)]

Please, ensure all groups in inventory exists. There are a few new groups, keep it sync with pike's inventory.

Make a diff from your inventory and https://github.com/openstack/kolla-ansible/blob/stable/pike/ansible/inventory/multinode

Regards

[panticz.de (panticz.de)]

Please make sure that all required Ceph groups are defined in your Ansible inventory with are used for kolla-ansible deploy:

# cat all-in-one
..
# Ceph
[ceph-mds:children]
ceph

[ceph-mgr:children]
ceph

[ceph-nfs:children]
ceph

[ceph-mon:children]
ceph

[ceph-rgw:children]
ceph

[ceph-osd:children]
storage
...

[Sabbir Sakib (sakibsys)]

Confirm. This is no longer an issue with queens release.