Enable fact caching in ansible

[Bjoern]

It might be beneficial to enable fact caching in ansible, especially on a hundred of nodes cluster, especially when the node facts are usually static. It can be enabled pretty easy doing :

apt-get install redis

diff --git a/playbooks/ansible.cfg b/playbooks/ansible.cfg
index 5c36952..8794e68 100644
--- a/playbooks/ansible.cfg
+++ b/playbooks/ansible.cfg
@@ -6,6 +6,9 @@ filter_plugins = plugins/filters
 gathering = smart
 hostfile = inventory
 host_key_checking = False
+fact_caching = redis
+fact_caching_timeout = 86400

you can verify if it is working using :

redis-cli : get ansible_facts+<hostname>

like

127.0.0.1:6379> get ansible_factsaio1_keystone_container-8bfd8abe
"{\"ansible_all_ipv4_addresses\": [\"10.0.3.129\", \"172.29.239.104\"],... output orbmitted....

[Bjoern (bjoern-t)]

Proposed change for master : https://review.openstack.org/#/c/225967/

[Bjoern (bjoern-t)]

I disabled the caching by default but left information inside the ansible.cfg so that is can be enabled on request.
It serves also as documentation for all interested / stressed ops folks.

[Jesse Pretorius (jesse-pretorius)]

Can you please give a better outline regarding why this is useful? What problem does it solve and how is it beneficial?

[Jesse Pretorius (jesse-pretorius)]

Marking it incomplete - please provide the details requested and switch the status back to new for discussion in the bug triage.

[Bjoern (bjoern-t)]

Fact caching is self explanatory. Why would you over and over collect possibly same data without caching ...

[Kevin Carter (kevin-carter)]

Tested this the other day when adding 150 compute nodes to an environment. The performance gains are noticeable and quite valuable. I've updated the patch that implements this change and modified it to use the "jsonfile" implementation of fact caching. In this way we add no new dependencies and still receive similar performance gains.

A deployer can choose to use redis but I don't think that it should be the default because the current implementation within Ansible does not allow for the use of user credentials or access to an external redis cluster. If that changes in future releases of Ansible I'd be happy to revisit the default settings.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: kilo
Review: https://review.openstack.org/228645

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/225967
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=1f07924cd2f619d0c0b3c0bf21e3d9cd5a1541b7
Submitter: Jenkins
Branch: master

commit 1f07924cd2f619d0c0b3c0bf21e3d9cd5a1541b7
Author: Bjoern Teipel <email address hidden>
Date: Mon Sep 21 12:50:31 2015 -0500

    Implement fact caching

    Fact caching will improve general Ansible performance by default.
    This has the biggest performance gain on very large environments.
    This implementation will use the JSONfile configuration because
    it has no new dependencies and will provide a similar performance
    gain as noted through the use of Redis.

    Notes:
    * The facts can purged or read by interacting with the default
      JSON file.
    * The caching behavior can be turned off by removing the parameter
      fact_caching from the playbook/ansible.cfg file.

    DocImpact
    UpgradeImpact

    Co-Authored-By: Kevin Carter <email address hidden>

    Implements: blueprint build-facts-archive
    Closes-Bug: 1498111

    Change-Id: I95ab1c2f8567d6af68feb53d7c14b8997eef5c89

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/228645
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=591b3c2cf37ca6570f74f07073ed14e7bf858fde
Submitter: Jenkins
Branch: kilo

commit 591b3c2cf37ca6570f74f07073ed14e7bf858fde
Author: Bjoern Teipel <email address hidden>
Date: Mon Sep 21 12:50:31 2015 -0500

    Implement fact caching

    Fact caching will improve general Ansible performance by default.
    This has the biggest performance gain on very large environments.
    This implementation will use the JSONfile configuration because
    it has no new dependencies and will provide a similar performance
    gain as noted through the use of Redis.

    Notes:
    * The facts can purged or read by interacting with the default
      JSON file.
    * The caching behavior can be turned off by removing the parameter
      fact_caching from the playbook/ansible.cfg file.

    DocImpact
    UpgradeImpact

    Co-Authored-By: Kevin Carter <email address hidden>

    Implements: blueprint build-facts-archive
    Closes-Bug: 1498111

    Change-Id: I95ab1c2f8567d6af68feb53d7c14b8997eef5c89
    (cherry picked from commit 1f07924cd2f619d0c0b3c0bf21e3d9cd5a1541b7)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.