ceph-ansible error No first item, sequence was empty

[Satish Patel]

I am deploying openstack-ansible with ceph and stuck here, following is my ceph user_variables.yml in openstack-ansible and i am getting same error.

In Openstack-ansible i have br-storage bridge which will be my ceph "public network" but i am getting error "No first item, sequence was empty" when running ceph-install.yml playbook, what i am doing wrong here?

## Common Ceph Overrides
ceph_mons:
 -172.16.200.211
 -172.16.200.76
 -172.16.200.195

ceph_stable_release: luminous

monitor_address_block: "172.16.200.0/24"
public_network: "172.16.200.0/24"
cluster_network: "10.5.1.0/24"
osd_scenario: collocated
journal_size: 10240 # size in MB
osd_objectstore: bluestore

It generated my ceph.conf file on mon node, if you noticed "mon host" list is different then public network, do you think that could be the issue?

mon initial members = ostack-controller-03-ceph-mon-container-f1024d18,ostack-controller-02-ceph-mon-container-b67c137c,ostack-controller-01-ceph-mon-container-a0cfeecb
mon host = 192.168.100.232,192.168.100.203,192.168.100.188

public network = 172.16.200.0/24
cluster network = 10.5.1.0/24

Error

TASK [ceph-config : remove tmp template file for ceph_conf_overrides] ***************************************************************************************
Wednesday 11 July 2018 23:48:15 -0400 (0:00:00.069) 0:02:45.063 ********
changed: [ostack-controller-02_ceph-mon_container-b67c137c -> localhost]

TASK [ceph-config : generate ceph configuration file: ceph.conf] ********************************************************************************************
Wednesday 11 July 2018 23:48:16 -0400 (0:00:00.238) 0:02:45.302 ********
fatal: [ostack-controller-02_ceph-mon_container-b67c137c]: FAILED! => {"failed": true, "msg": "No first item, sequence was empty."}

PLAY RECAP **************************************************************************************************************************************************
ostack-controller-02_ceph-mon_container-b67c137c : ok=42 changed=9 unreachable=0 failed=1

[Satish Patel (satish-txt)]

I found related post but wondering does OSAD support same solution?

https://bugs.launchpad.net/tripleo/+bug/1709485

[Allen (allen-s)]

I reverted to 17.0.5 and am still having the same issue. Complete rebuild of the deployment server and machines.

Is there any known workarounds?

[Satish Patel (satish-txt)]

In my case i did "rm -rf /etc/openstack_deploy/ansible_facts" which fixed issue, look like facts where cached.

[Satish Patel (satish-txt)]

Now i am getting this error when running setup-openstack.yml, I want cinder/glance/ceph_mon/ceph_osd all running on br-storage network, I can ping ceph_mon from glance but still glance saying "No reachable" look like glance finding ceph on br-mgmt interface instead of br-storage.. any thought?

TASK [ceph_client : Fail when no ceph_mon_host is reachable] ************************************************************************************************
Friday 13 July 2018 20:55:13 -0400 (0:00:00.359) 0:06:10.920 ***********
fatal: [ostack-controller-03_glance_container-98fdccc5]: FAILED! => {"changed": false, "failed": true, "msg": "No Ceph Monitor reachable"}

PLAY RECAP **************************************************************************************************************************************************
ostack-controller-03_glance_container-98fdccc5 : ok=83 changed=4 unreachable=0 failed=1

[Satish Patel (satish-txt)]

After talking to @allen found OSA deployment node need br-storage VLAN reachability to make it works, after adding br-storage network on OSA deployment node it works.

[Marc Gariépy (mgariepy)]

Marking as invalid since it was a config issue.