lxc_hosts dynamic inventory group adds non-lxc hosts

[Logan V]

In the inventory below only lsn-mc1006, lsn-mc1007, and lsn-mc1008 host lxc containers. The rest only contain on_metal: true services.

ubuntu@deploy-devcloud:playbooks$ ansible localhost -m debug -a msg="{{ groups['lxc_hosts'] }}"
localhost | SUCCESS => {
    "msg": [
        "lsn-mc1008",
        "lsn-mc1007",
        "lsn-mc1006",
        "lsn-d4325",
        "lsn-d4802",
        "lsn-d5077",
        "lsn-d4999",
        "lsn-d4794",
        "lsn-d4624",
        "lsn-d5211",
        "lsn-d5420",
        "lsn-d5923",
        "lsn-d5376",
        "lsn-d5610",
        "lsn-d5822",
        "lsn-d5382",
        "lsn-d6204",
        "lsn-d5897",
        "lsn-d5667",
        "lsn-d5659"
    ]
}
ubuntu@deploy-devcloud:playbooks$ ansible localhost -m debug -a msg="{{ groups['lsn-d5659_containers'] }}"
localhost | FAILED! => {
    "failed": true,
    "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute 'lsn-d5659_containers'"
}
ubuntu@deploy-devcloud:playbooks$ ansible lsn-d5659 -m debug -a msg="{{ group_names }}"
lsn-d5659 | SUCCESS => {
    "msg": [
        "ceph-osd",
        "ceph-osd_all",
        "ceph-osd_hosts",
        "ceph_all",
        "hosts",
        "lxc_hosts"
    ]
}

ubuntu@deploy-devcloud:playbooks$ cat /etc/openstack_deploy/env.d/ceph.yml
component_skel:
  ceph-mon:
    belongs_to:
    - ceph_all
  ceph-osd:
    belongs_to:
    - ceph_all
  ceph-rgw:
    belongs_to:
    - ceph_all
container_skel:
  ceph-mon_container:
    belongs_to:
    - shared-infra_containers
    contains:
    - ceph-mon
    properties:
      service_name: ceph-mon
  ceph-osd_container:
    belongs_to:
    - ceph-osd_containers
    contains:
    - ceph-osd
    properties:
      is_metal: true
      service_name: ceph
  ceph-rgw_container:
    belongs_to:
    - ceph-rgw_containers
    contains:
    - ceph-rgw
    properties:
      is_metal: true
      service_name: ceph
physical_skel:
  ceph-osd_hosts:
    belongs_to:
    - hosts
  ceph-rgw_hosts:
    belongs_to:
    - hosts

[Logan V (loganv)]

also the output from dynamic_inventory regarding lsn-d5659:
            "lsn-d5659": {
                "ansible_host": "10.12.1.144",
                "ansible_ssh_host": "10.12.1.144",
                "bond_slaves": {
                    "bond0": [
                        "p4p1",
                        "em1"
                    ]
                },
                "component": "ceph-osd",
                "container_address": "10.12.1.144",
                "container_name": "lsn-d5659",
                "container_networks": {},
                "container_types": "lsn-d5659-host_containers",
                "is_metal": true,
                "net_boot": {
                    "bond0": {
                        "ip": "10.10.8.144",
                        "mac": "00:25:90:7f:71:f8"
                    },
                    "dummy1": {
                        "ip": "10.12.1.144",
                        "type": "dummy"
                    }
                },
                "physical_host": "lsn-d5659",
                "physical_host_group": "ceph-osd_hosts",
                "properties": {
                    "is_metal": true,
                    "service_name": "ceph"
                }
            },

[Logan V (loganv)]

Testing out the old jinja removed here https://github.com/openstack/openstack-ansible/commit/518fb381c49a04e2b8af4a7e623e698482033efc I get the expected output:

known_container_hosts: >
  {% set _var = [] -%}
  {% for item in groups['all_containers'] -%}
  {% if hostvars[item]['physical_host'] | default(false) != item -%}
  {% if _var.append(hostvars[item]['physical_host']) -%}
  {% endif -%}
  {% endif -%}
  {% endfor -%}
  {{ _var | unique }}

ubuntu@deploy-devcloud:playbooks$ ansible localhost -m debug -a msg="{{ known_container_hosts }}"
localhost | SUCCESS => {
    "msg": [
        "lsn-mc1008",
        "lsn-mc1007",
        "lsn-mc1006"
    ]
}

[Nolan Brubaker (nolan-brubaker)]

I can confirm this happens with hosts added to `compute_hosts` in automated testing, as well.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427855
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=59aaac0cb5cf935d5d5e9ee899ecfd25b67923a1
Submitter: Jenkins
Branch: master

commit 59aaac0cb5cf935d5d5e9ee899ecfd25b67923a1
Author: Nolan Brubaker <email address hidden>
Date: Wed Feb 1 13:28:58 2017 -0500

    Only insert container hosts into lxc_hosts

    This change introduces a set of functions for dynamically populating the
    lxc_hosts function on each inventory run. Any previous membership is
    overwritten each time. The output is written to the
    openstack_inventory.json still in order to provide parity with the
    values provided to the ansible executable on stdout.

    Previous code was fairly naive, inserting hosts into lxc_hosts when they
    were marked as `is_metal`. This would add hosts, such as Ceph, that
    were on metal but had no LXC containers.

    A previous attempt at fixing this changed the _build_container_hosts
    function to provide more information about the container build process,
    then used that data in the _append_host_containers function to populate
    the lxc_hosts group. However, this approach failed due to limited
    information in each pass of the loop - if a node was an AIO, it might be
    erroneously removed from the lxc_hosts group because a container wasn't
    built on a given pass, and due to ordering, that pass may be the last
    one of the loop.

    To get around such problems, this code instead processes the inventory
    in whole, after all containers have been made. Population into the group
    is determined according to whether or not a given host's `physical_host`
    hostvar matches the host name.

    Change-Id: I9f3336f77cd0ef05fe1c7edeaf7defc6d93c3111
    Closes-Bug: #1660996

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/439378

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/439378
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=db53e5405e880b02d77fa32f487bb8ad319ee91b
Submitter: Jenkins
Branch: stable/ocata

commit db53e5405e880b02d77fa32f487bb8ad319ee91b
Author: Nolan Brubaker <email address hidden>
Date: Wed Feb 1 13:28:58 2017 -0500

    Only insert container hosts into lxc_hosts

    This change introduces a set of functions for dynamically populating the
    lxc_hosts function on each inventory run. Any previous membership is
    overwritten each time. The output is written to the
    openstack_inventory.json still in order to provide parity with the
    values provided to the ansible executable on stdout.

    Previous code was fairly naive, inserting hosts into lxc_hosts when they
    were marked as `is_metal`. This would add hosts, such as Ceph, that
    were on metal but had no LXC containers.

    A previous attempt at fixing this changed the _build_container_hosts
    function to provide more information about the container build process,
    then used that data in the _append_host_containers function to populate
    the lxc_hosts group. However, this approach failed due to limited
    information in each pass of the loop - if a node was an AIO, it might be
    erroneously removed from the lxc_hosts group because a container wasn't
    built on a given pass, and due to ordering, that pass may be the last
    one of the loop.

    To get around such problems, this code instead processes the inventory
    in whole, after all containers have been made. Population into the group
    is determined according to whether or not a given host's `physical_host`
    hostvar matches the host name.

    Change-Id: I9f3336f77cd0ef05fe1c7edeaf7defc6d93c3111
    Closes-Bug: #1660996
    (cherry picked from commit 59aaac0cb5cf935d5d5e9ee899ecfd25b67923a1)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/427885
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=b6eebd5c7abdbcb1d9b3fea8e9b4605ba1de2b28
Submitter: Jenkins
Branch: stable/newton

commit b6eebd5c7abdbcb1d9b3fea8e9b4605ba1de2b28
Author: Nolan Brubaker <email address hidden>
Date: Wed Feb 1 14:52:53 2017 -0500

    Only insert container hosts into lxc_hosts

    This change introduces a set of functions for dynamically populating the
    lxc_hosts function on each inventory run. Any previous membership is
    overwritten each time. The output is written to the
    openstack_inventory.json still in order to provide parity with the
    values provided to the ansible executable on stdout.

    Previous code was fairly naive, inserting hosts into lxc_hosts when they
    were marked as `is_metal`. This would add hosts, such as Ceph, that
    were on metal but had no LXC containers.

    A previous attempt at fixing this changed the _build_container_hosts
    function to provide more information about the container build process,
    then used that data in the _append_host_containers function to populate
    the lxc_hosts group. However, this approach failed due to limited
    information in each pass of the loop - if a node was an AIO, it might be
    erroneously removed from the lxc_hosts group because a container wasn't
    built on a given pass, and due to ordering, that pass may be the last
    one of the loop.

    To get around such problems, this code instead processes the inventory
    in whole, after all containers have been made. Population into the group
    is determined according to whether or not a given host's `physical_host`
    hostvar matches the host name.

    This backport was manual due to code organization changes in
    master/ocata. Some variable changes and import paths were updated in the
    tests, but they should otherwise be identical to the ocata version.

    Change-Id: I9f3336f77cd0ef05fe1c7edeaf7defc6d93c3111
    Closes-Bug: #1660996

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 14.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 15.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 16.0.0.0b1 development milestone.