tripleo-ansible-inventory multi stack produces group with servers it shouldn't have

[John Fulton]

Multi stack inventories as introduced by https://review.opendev.org/#/c/664065 produce a group which is not correct.

The cell1_CellController group below shows cell1-cellcontrol-0 and cell1-novacompute-0, while it should only have cell1-cellcontrol-0 .

$ /usr/bin/tripleo-ansible-inventory --stack cell1 --static-yaml-inventory test_cell1_inv.yaml

~~~
Undercloud:
  hosts:
    undercloud: {}
  vars:
    ansible_connection: local
    ansible_host: localhost
    ansible_python_interpreter: /usr/bin/python2
    ansible_remote_tmp: /tmp/ansible-${USER}
    auth_url: https://192.168.24.2:13000
    cacert: /etc/pki/ca-trust/source/anchors/cm-local-ca.pem
    os_auth_token: gAAAAABdqbIL1mkfbQerlj5opL4YyneuDSVLzMb80WPuoiTLBckGSDm2YC-c6SE5mfByrxJmWJL66yANWgdNU6RoNQsWQHZX7QgxDGB4N3yEBQa9UcB7r01qhtnkxlJNRySYjhd06EOSOvzs_rUCQx_vjAQoMx1pAmF8jjsDAX7vz7x7VzffB8Q
    overcloud_admin_password: xBXmN3jEIOpeJLlSZ54o411eh
    overcloud_horizon_url: https://10.0.0.5:443/dashboard
    overcloud_keystone_url: https://10.0.0.5:13000
    plan: cell1
    project_name: admin
    undercloud_service_list: [tripleo_nova_compute, tripleo_heat_engine, tripleo_ironic_conductor,
      tripleo_swift_container_server, tripleo_swift_object_server, tripleo_mistral_engine]
    undercloud_swift_url: https://192.168.24.2:13808/v1/AUTH_6c785a90c9344b28932bf821262fa818
    username: admin
CellController:
  hosts:
    cell1-cellcontrol-0:
      ansible_host: 192.168.24.10
      canonical_hostname: cell1-cellcontrol-0.ooo.test
      ctlplane_hostname: cell1-cellcontrol-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.10
      deploy_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_hostname: cell1-cellcontrol-0.external.ooo.test
      external_ip: 10.0.0.10
      internal_api_hostname: cell1-cellcontrol-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.188
      management_ip: 192.168.24.10
      storage_hostname: cell1-cellcontrol-0.storage.ooo.test
      storage_ip: 172.16.1.164
      storage_mgmt_hostname: cell1-cellcontrol-0.storagemgmt.ooo.test
      storage_mgmt_ip: 172.16.3.111
      tenant_hostname: cell1-cellcontrol-0.tenant.ooo.test
      tenant_ip: 172.16.0.203
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389,
    serial: '1', tripleo_role_name: CellController}
Compute:
  hosts:
    cell1-novacompute-0:
      ansible_host: 192.168.24.19
      canonical_hostname: cell1-novacompute-0.ooo.test
      ctlplane_hostname: cell1-novacompute-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.19
      deploy_server_id: 1a163e1b-06de-4d4a-a228-0f9d143897b8
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_ip: 192.168.24.19
      internal_api_hostname: cell1-novacompute-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.236
      management_ip: 192.168.24.19
      storage_hostname: cell1-novacompute-0.storage.ooo.test
      storage_ip: 172.16.1.249
      storage_mgmt_ip: 192.168.24.19
      tenant_hostname: cell1-novacompute-0.tenant.ooo.test
      tenant_ip: 172.16.0.36
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389,
    serial: '1', tripleo_role_name: Compute}
overcloud:
  children:
    CellController: {}
    Compute: {}
  vars: {container_cli: docker, ctlplane_vip: 192.168.24.28, external_vip: 10.0.0.6,
    internal_api_vip: 172.16.2.54, redis_vip: 172.16.2.60, storage_mgmt_vip: 172.16.3.12,
    storage_vip: 172.16.1.208}
kernel:
  children:
    CellController: {}
    Compute: {}
  vars: {ansible_ssh_user: heat-admin}
nova_conductor:
  children:
    CellController: {}
  vars: {ansible_ssh_user: heat-admin}
...
~~~

$ /usr/bin/tripleo-ansible-inventory --stack overcloud,cell1 --static-yaml-inventory test_inv.yaml

~~~
Undercloud:
  hosts:
    undercloud: {}
  vars:
    ansible_connection: local
    ansible_host: localhost
    ansible_python_interpreter: /usr/bin/python2
    ansible_remote_tmp: /tmp/ansible-${USER}
    auth_url: https://192.168.24.2:13000
    cacert: /etc/pki/ca-trust/source/anchors/cm-local-ca.pem
    os_auth_token: gAAAAABdqbC5HUbRB4Y4x_SZB7EHIk0ez12DrMWr2b8iUnYvpXC9dyGnqfABK7YQEbIQCvKuSKO8SSACTedqiF_TP-AORCjuMdrq2-6539aqIzQIz6RLhc6ydarCIH_CX2xYwwMKoWTEc-EST3dN9_ULw_NhIpP1wN7ll7c0-anlNqLs5BpBjEk
    overcloud_admin_password: xBXmN3jEIOpeJLlSZ54o411eh
    overcloud_horizon_url: https://10.0.0.5:443/dashboard
    overcloud_keystone_url: https://10.0.0.5:13000
    plan: ''
    plans: [cell1, overcloud]
    project_name: admin
    undercloud_service_list: [tripleo_nova_compute, tripleo_heat_engine, tripleo_ironic_conductor,
      tripleo_swift_container_server, tripleo_swift_object_server, tripleo_mistral_engine]
    undercloud_swift_url: https://192.168.24.2:13808/v1/AUTH_6c785a90c9344b28932bf821262fa818
    username: admin
cell1_CellController:
  hosts: &id001
    cell1-cellcontrol-0:
      ansible_host: 192.168.24.10
      canonical_hostname: cell1-cellcontrol-0.ooo.test
      ctlplane_hostname: cell1-cellcontrol-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.10
      deploy_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_hostname: cell1-cellcontrol-0.external.ooo.test
      external_ip: 10.0.0.10
      internal_api_hostname: cell1-cellcontrol-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.188
      management_ip: 192.168.24.10
      storage_hostname: cell1-cellcontrol-0.storage.ooo.test
      storage_ip: 172.16.1.164
      storage_mgmt_hostname: cell1-cellcontrol-0.storagemgmt.ooo.test
      storage_mgmt_ip: 172.16.3.111
      tenant_hostname: cell1-cellcontrol-0.tenant.ooo.test
      tenant_ip: 172.16.0.203
    cell1-novacompute-0:
      ansible_host: 192.168.24.19
      canonical_hostname: cell1-novacompute-0.ooo.test
      ctlplane_hostname: cell1-novacompute-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.19
      deploy_server_id: 1a163e1b-06de-4d4a-a228-0f9d143897b8
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_ip: 192.168.24.19
      internal_api_hostname: cell1-novacompute-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.236
      management_ip: 192.168.24.19
      storage_hostname: cell1-novacompute-0.storage.ooo.test
      storage_ip: 172.16.1.249
      storage_mgmt_ip: 192.168.24.19
      tenant_hostname: cell1-novacompute-0.tenant.ooo.test
      tenant_ip: 172.16.0.36
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389,
    serial: '1', tripleo_role_name: CellController}
cell1_Compute:
  hosts: *id001
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: 2b2e4e4d-857c-48be-885f-de2eca619389,
    serial: '1', tripleo_role_name: Compute}
cell1_overcloud: &id002
  children:
    cell1_Compute: {}
  vars: {container_cli: docker, ctlplane_vip: 192.168.24.28, external_vip: 10.0.0.6,
    internal_api_vip: 172.16.2.54, redis_vip: 172.16.2.60, storage_mgmt_vip: 172.16.3.12,
    storage_vip: 172.16.1.208}
cell1: *id002
cell1_kernel:
  children:
    cell1_Compute: {}
  vars: {ansible_ssh_user: heat-admin}
cell1_nova_conductor:
  children:
    cell1_CellController: {}
  vars: {ansible_ssh_user: heat-admin}
...
~~~

[Martin Schuppert (mschuppert)]

$ /usr/bin/tripleo-ansible-inventory --stack overcloud,cell1 --static-yaml-inventory test_inv.yaml
~~~
Undercloud:
  hosts:
    undercloud: {}
  vars:
    ansible_connection: local
    ansible_host: localhost
    ansible_python_interpreter: /usr/bin/python2
    ansible_remote_tmp: /tmp/ansible-${USER}
    auth_url: https://192.168.24.2:13000
    cacert: /etc/pki/ca-trust/source/anchors/cm-local-ca.pem
    os_auth_token: gAAAAABdrvUOK1vzq8A0SmrNJkKq3wO7cKdcERqtywXLmugO9BbgvQE-fMJ34zc0xkysLbJb5wLn6nN1cUSmCWjBqSnc2lP90TWabQxNZ-9cZ67CdU136SAng9w-ZhPyhm9l4ddPaxZOMngZYhNzHvw5VQqSI6HVk28JuOyF8JKNH2-drS_HZPA
    overcloud_admin_password: 1XOD5itCotVL8h2ZYFbEIUKEL
    overcloud_horizon_url: https://10.0.0.5:443/dashboard
    overcloud_keystone_url: https://10.0.0.5:13000
    plan: ''
    plans: [cell1, overcloud]
    project_name: admin
    undercloud_service_list: [tripleo_nova_compute, tripleo_heat_engine, tripleo_ironic_conductor,
      tripleo_swift_container_server, tripleo_swift_object_server, tripleo_mistral_engine]
    undercloud_swift_url: https://192.168.24.2:13808/v1/AUTH_1056be26845a40dab35dc847f66a56b3
    username: admin
cell1_CellController:
  hosts: &id001
    cell1-cellcontrol-0:
      ansible_host: 192.168.24.25
      canonical_hostname: cell1-cellcontrol-0.ooo.test
      ctlplane_hostname: cell1-cellcontrol-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.25
      deploy_server_id: c972375c-90e3-427d-bfb8-d22c51b1aa5f
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_hostname: cell1-cellcontrol-0.external.ooo.test
      external_ip: 10.0.0.224
      internal_api_hostname: cell1-cellcontrol-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.187
      management_ip: 192.168.24.25
      storage_hostname: cell1-cellcontrol-0.storage.ooo.test
      storage_ip: 172.16.1.193
      storage_mgmt_hostname: cell1-cellcontrol-0.storagemgmt.ooo.test
      storage_mgmt_ip: 172.16.3.145
      tenant_hostname: cell1-cellcontrol-0.tenant.ooo.test
      tenant_ip: 172.16.0.237
    cell1-novacompute-0:
      ansible_host: 192.168.24.18
      canonical_hostname: cell1-novacompute-0.ooo.test
      ctlplane_hostname: cell1-novacompute-0.ctlplane.ooo.test
      ctlplane_ip: 192.168.24.18
      deploy_server_id: 2bc8fb7b-dbe8-4c13-a50e-5106388010e0
      enabled_networks: [management, storage, ctlplane, external, internal_api, storage_mgmt,
        tenant]
      external_ip: 192.168.24.18
      internal_api_hostname: cell1-novacompute-0.internalapi.ooo.test
      internal_api_ip: 172.16.2.175
      management_ip: 192.168.24.18
      storage_hostname: cell1-novacompute-0.storage.ooo.test
      storage_ip: 172.16.1.184
      storage_mgmt_ip: 192.168.24.18
      tenant_hostname: cell1-novacompute-0.tenant.ooo.test
      tenant_ip: 172.16.0.134
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: c972375c-90e3-427d-bfb8-d22c51b1aa5f,
    serial: '1', tripleo_role_name: CellController}
cell1_Compute:
  hosts: *id001
  vars: {ansible_ssh_user: heat-admin, bootstrap_server_id: c972375c-90e3-427d-bfb8-d22c51b1aa5f,
    serial: '1', tripleo_role_name: Compute}
cell1_overcloud: &i...

[John Fulton (jfulton-org)]

This was not fixed by https://review.opendev.org/#/c/668560 ( was editing the wrong bug)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/713750

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/713919

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/713750
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=f2e833e74a213f5c011154e76fa25e36f9f29009
Submitter: Zuul
Branch: master

commit f2e833e74a213f5c011154e76fa25e36f9f29009
Author: Oliver Walsh <email address hidden>
Date: Wed Mar 18 22:01:57 2020 +0000

    Fix multistack inventory

    The static inventory merging was setting the role hosts to the _meta hostvars.
    This is a special key to set hostvars for dynamic inventories. It contains all
    hosts in the inventory to prevent the dynamic inventory script being called
    with --host for each host.

    It was not used by the single stack inventory when generating a static
    inventory. Instead an evil magic flag (host_format_dict), switched the
    behaviour to embed the hostvars.

    This flag is now removed and replaced with a param to the list method and
    calling the write_static_inventory methods no longer magically mutates
    the object. The logic for static vs dynamic inventory should be clearer.

    The interface for the multi stack inventory has also change to be more
    consistent with the single stack interface.

    Unit tests now cover both static and dynamic multistack inventories.

    Change-Id: Ife14dbe04dd11db44a944f98373f63e01dfbb8d8
    Depends-On: If022a4647aaebc4b212bba188cc959ff2f8f9311
    Closes-bug: #1849073

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/713919
Committed: https://git.openstack.org/cgit/openstack/tripleo-validations/commit/?id=afe97a302451c26f7f187e31357b6743986229f6
Submitter: Zuul
Branch: master

commit afe97a302451c26f7f187e31357b6743986229f6
Author: Oliver Walsh <email address hidden>
Date: Thu Mar 19 15:51:53 2020 +0000

    Fix multistack inventory

    Switch to new multistack inventory class interface.
    Also TRIPLEO_PLAN_NAME did not work for comma seperated list.

    Depends-On: Ife14dbe04dd11db44a944f98373f63e01dfbb8d8
    Change-Id: I2cc721676005536b14995980f7a042991c92adcc
    Related-bug: #1849073

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/718556

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/718557

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/718789

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/719038

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/718556
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=8e6a86d623c3401f9b6d7d35d710e32296c20d48
Submitter: Zuul
Branch: stable/train

commit 8e6a86d623c3401f9b6d7d35d710e32296c20d48
Author: Oliver Walsh <email address hidden>
Date: Wed Mar 18 22:01:57 2020 +0000

    Fix multistack inventory

    The static inventory merging was setting the role hosts to the _meta hostvars.
    This is a special key to set hostvars for dynamic inventories. It contains all
    hosts in the inventory to prevent the dynamic inventory script being called
    with --host for each host.

    It was not used by the single stack inventory when generating a static
    inventory. Instead an evil magic flag (host_format_dict), switched the
    behaviour to embed the hostvars.

    This flag is now removed and replaced with a param to the list method and
    calling the write_static_inventory methods no longer magically mutates
    the object. The logic for static vs dynamic inventory should be clearer.

    The interface for the multi stack inventory has also change to be more
    consistent with the single stack interface.

    Unit tests now cover both static and dynamic multistack inventories.

    Change-Id: Ife14dbe04dd11db44a944f98373f63e01dfbb8d8
    Depends-On: If022a4647aaebc4b212bba188cc959ff2f8f9311
    Closes-bug: #1849073
    (cherry picked from commit f2e833e74a213f5c011154e76fa25e36f9f29009)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/719038
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=9fc3c5717b61cdcb45f9b0222e7d06a6c9dd65ac
Submitter: Zuul
Branch: stable/train

commit 9fc3c5717b61cdcb45f9b0222e7d06a6c9dd65ac
Author: Oliver Walsh <email address hidden>
Date: Thu Apr 9 21:15:50 2020 +0100

    Include uc plan var when TripleoInventories contains a single plan

    I2cc721676005536b14995980f7a042991c92adcc switched to using TripleoInventories
    for both single and multi stack inventories. However the 'plan' undercloud var
    was no longer set.
    Restore this for single stack inventories only to keep compatibility.

    Change-Id: I5735cfaa5fe59eabde66b3370a2c5838ff0c4b23
    Related-bug: #1849073
    (cherry picked from commit 2afb9771f0532335e971c72c0eb206de2fc38886)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/718557
Committed: https://git.openstack.org/cgit/openstack/tripleo-validations/commit/?id=7b9484343a73376b77ca85d3b3b3001bfee15508
Submitter: Zuul
Branch: stable/train

commit 7b9484343a73376b77ca85d3b3b3001bfee15508
Author: Oliver Walsh <email address hidden>
Date: Thu Mar 19 15:51:53 2020 +0000

    Fix multistack inventory

    Switch to new multistack inventory class interface.
    Also TRIPLEO_PLAN_NAME did not work for comma seperated list.

    Depends-On: Ife14dbe04dd11db44a944f98373f63e01dfbb8d8
    Change-Id: I2cc721676005536b14995980f7a042991c92adcc
    Related-bug: #1849073
    (cherry picked from commit afe97a302451c26f7f187e31357b6743986229f6)