OC deployment with multiple subnets under default networks fails with: msg: 'ipaddr: unknown filter type: 172.120.3.0/24,172.117.3.0/24,172.118.3.0/24,172.119.3.0/24'"

[Harald Jensås]

Log: /var/lib/mistral/overcloud/ceph-ansible/ceph_ansible_command.log

TASK [ceph-facts : set_fact _monitor_address to monitor_address_block ipv4] ****
task path: /usr/share/ceph-ansible/roles/ceph-facts/tasks/set_monitor_address.yml:2
Friday 09 August 2019 23:43:45 +0000 (0:00:00.921) 0:01:17.061 *********
fatal: [overcloud-controller-0]: FAILED! =>
  msg: 'ipaddr: unknown filter type: 172.120.3.0/24,172.117.3.0/24,172.118.3.0/24,172.119.3.0/24'

Code: /usr/share/ceph-ansible/roles/ceph-facts/tasks/set_monitor_address.yml

- name: set_fact _monitor_address to monitor_address_block ipv4
  set_fact:
    _monitor_addresses: "{{ _monitor_addresses | default([]) + [{ 'name': item, 'addr': hostvars[item]['ansible_all_ipv4_addresses'] | ipaddr(hostvars[item]['monitor_address_block']) | first }] }}"
  with_items: "{{ groups.get(mon_group_name, []) }}"
  when:
    - "item not in _monitor_addresses | default([]) | selectattr('name', 'defined') | map(attribute='name') | list"
    - hostvars[item]['monitor_address_block'] is defined
    - hostvars[item]['monitor_address_block'] != 'subnet'
    - ip_version == 'ipv4'

When it is setting the _monitor_addresses fact, ipaddr expects monitor_address_block to be a single IP subnet CIDR not a list of CIDR's.

In tripleo-heat-templates[1] a list of CIDR for all subnets on the CephMonNetwork is passed as . We need to pass a single CIDR for monitor_address_block.

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/ceph-ansible/ceph-base.yaml#L292-L295

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/676178

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: master
Review: https://review.opendev.org/676178
Reason: Proposing a fix in ceph-ansible instead, see: https://github.com/ceph/ceph-ansible/pull/4339

[Harald Jensås (harald-jensas)]

ceph-ansible core is suggesting:

"""
that setting monitor_address_block at the inventory level should be enough:

 [mons]
 mon0 monitor_address_block=192.168.10.0/24
 mon1 monitor_address_block=192.168.20.0/24
 mon2 monitor_address_block=192.168.30.0/24
"""

While the proposed fix works, there is push-back from ceph-ansible core since it introduces a custom filter_plugin.

1. I don't see how we can set this on the inventory level in TripleO.
2. Since we cannot use hiera interpolation (see https://review.opendev.org/553804) I don't see how we can get the correct address block for each mon if we can set it on the inventory level in Tripleo.

NOTE: I will update the patch to also use the filter for the radosgw_address_block.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/679060

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/679410

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/679412

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: master
Review: https://review.opendev.org/679412
Reason: We are most likely landing https://github.com/ceph/ceph-ansible/pull/4339 instead.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: master
Review: https://review.opendev.org/679410
Reason: We are most likely landing https://github.com/ceph/ceph-ansible/pull/4339 instead.

[Harald Jensås (harald-jensas)]

This was fixed in ceph-ansible.