ceph_spec_bootstrap not handling children groups in tripleo-ansible-inventory

[John Fulton]

2022-12-01 07:20:33.491271 | 5254000e-1e92-b58e-24eb-000000000018 | TASK | Create Ceph spec based on baremetal_deployed_path and tripleo_roles
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'hosts'
2022-12-01 07:20:34.051011 | 5254000e-1e92-b58e-24eb-000000000018 | FATAL | Create Ceph spec based on baremetal_deployed_path and tripleo_roles | undercloud | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 107, in <module>\n File \"<stdin>\", line 99, in _ansiballz_main\n File \"<stdin>\", line 47, in invoke_module\n File \"/usr/lib64/python3.9/runpy.py\", line 210, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_ceph_spec_bootstrap_payload_z3x_t18l/ansible_ceph_spec_bootstrap_payload.zip/ansible/modules/ceph_spec_bootstrap.py\", line 576, in <module>\n File \"/tmp/ansible_ceph_spec_bootstrap_payload_z3x_t18l/ansible_ceph_spec_bootstrap_payload.zip/ansible/modules/ceph_spec_bootstrap.py\", line 556, in main\n File \"/tmp/ansible_ceph_spec_bootstrap_payload_z3x_t18l/ansible_ceph_spec_bootstrap_payload.zip/ansible/modules/ceph_spec_bootstrap.py\", line 179, in get_inventory_hosts_to_ips\nKeyError: 'hosts'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
2022-12-01 07:20:34.051577 | 5254000e-1e92-b58e-24eb-000000000018 | TIMING | Create Ceph spec based on baremetal_deployed_path and tripleo_roles | undercloud | 0:00:00.830279 | 0.56s

[John Fulton (jfulton-org)]

The tripleo-ansible-inventory file contains the following:

  CephStorage:
    children:
      overcloud_CephStorage: {}
  overcloud_CephStorage:
    hosts:
      ceph-0:
        ...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/866503

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/866465

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/866503
Committed: https://opendev.org/openstack/tripleo-ansible/commit/04c33a4747769cc414d163fb25cd0f8927ba9206
Submitter: "Zuul (22348)"
Branch: master

commit 04c33a4747769cc414d163fb25cd0f8927ba9206
Author: John Fulton <email address hidden>
Date: Fri Dec 2 21:04:32 2022 +0000

    Handle child groups when building ceph spec from inventory

    If the inventory contains the following:

      CephStorage:
        children:
          overcloud_CephStorage: {}
      overcloud_CephStorage:
        hosts:
          ceph-0:

    and I want to get a list of hosts from the CephStorage group,
    it is wrong to return that CephStorage has no hosts. Instead
    since all hosts in the overcloud_CephStorage group are children
    of the CephStorage group, I should return the hosts of the
    child group. I.e. the hosts of CephStorage above are [ceph-0].

    Closes-Bug: #1998649
    Change-Id: I0405f6729b492af011531642a2ee374bb210fa82

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/866653

[OpenStack Infra (hudson-openstack)]

Change abandoned by "John Fulton <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/866465
Reason: waiting for cherry pick which has merged based on https://review.opendev.org/c/openstack/tripleo-ansible/+/866653

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/866653
Committed: https://opendev.org/openstack/tripleo-ansible/commit/37fc1a9d11fad1ed52e3ae4a7381b90150dcdbe5
Submitter: "Zuul (22348)"
Branch: stable/zed

commit 37fc1a9d11fad1ed52e3ae4a7381b90150dcdbe5
Author: John Fulton <email address hidden>
Date: Fri Dec 2 21:04:32 2022 +0000

    Handle child groups when building ceph spec from inventory

    If the inventory contains the following:

      CephStorage:
        children:
          overcloud_CephStorage: {}
      overcloud_CephStorage:
        hosts:
          ceph-0:

    and I want to get a list of hosts from the CephStorage group,
    it is wrong to return that CephStorage has no hosts. Instead
    since all hosts in the overcloud_CephStorage group are children
    of the CephStorage group, I should return the hosts of the
    child group. I.e. the hosts of CephStorage above are [ceph-0].

    Closes-Bug: #1998649
    Change-Id: I0405f6729b492af011531642a2ee374bb210fa82
    (cherry picked from commit 04c33a4747769cc414d163fb25cd0f8927ba9206)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/866465
Committed: https://opendev.org/openstack/tripleo-ansible/commit/6f1b1fe8e76724a724e7aa765e8779109af4341a
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 6f1b1fe8e76724a724e7aa765e8779109af4341a
Author: John Fulton <email address hidden>
Date: Fri Dec 2 21:04:32 2022 +0000

    Handle child groups when building ceph spec from inventory

    If the inventory contains the following:

      CephStorage:
        children:
          overcloud_CephStorage: {}
      overcloud_CephStorage:
        hosts:
          ceph-0:

    and I want to get a list of hosts from the CephStorage group,
    it is wrong to return that CephStorage has no hosts. Instead
    since all hosts in the overcloud_CephStorage group are children
    of the CephStorage group, I should return the hosts of the
    child group. I.e. the hosts of CephStorage above are [ceph-0].

    Closes-Bug: #1998649
    Change-Id: I0405f6729b492af011531642a2ee374bb210fa82
    (cherry picked from commit 04c33a4747769cc414d163fb25cd0f8927ba9206)
    (cherry picked from commit 37fc1a9d11fad1ed52e3ae4a7381b90150dcdbe5)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-ansible 6.0.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/881720

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Oliver Walsh <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/866410
Reason: wallaby only - https://review.opendev.org/c/openstack/tripleo-ansible/+/881720

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/881720
Committed: https://opendev.org/openstack/tripleo-ansible/commit/92c59b3f997a90fc38b67864f84ea345d5ca6aa3
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 92c59b3f997a90fc38b67864f84ea345d5ca6aa3
Author: Oliver Walsh <email address hidden>
Date: Thu Dec 1 23:18:10 2022 +0000

    Generalize ceph_spec_bootstrap inventory parsing

    Create a generic helper class in module_utils to parse
    inventory yaml. Similar to the ansible-core InventoryManager
    but usable from ansible modules with minimal dependencies
    (just python stdlib and yaml).

    This should be compatible with any current/past tripleo inventory
    structures and potentially future inventory structure changes (within
    reason).

    Related-bug: #1998649
    Change-Id: Id2fb39054bd5f81deb967ed9938c613a2ca6ea2c