[fluentd] Elasticsearch output enabled even when service disabled

[Scott Shambarger]

Seen on v11, but appears in v12/master too...

Deploying all-in-one with the following in global.yml (elasticsearch disabled by default):

kolla_internal_vip_address: "192.168.60.10"
kolla_internal_fqdn: "cluster.test.lan"

fluentd container contains td-agent.conf with elasticsearch output <store> entries, and fluentd repeatedly fails to connect to it, from fluentd.log:

#0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 192.168.60.10:9200 (Errno::ECONNREFUSED)

THE FIX:
It appears that there's an error in ansible/roles/common/tasks/config.yml (about line 105):

- name: Copying over td-agent.conf
  vars:
    log_direct_to_elasticsearch: >-
      {{ ( enable_elasticsearch | bool or
           ( elasticsearch_address != kolla_internal_vip_address )) and
         not enable_monasca | bool }}

log_direct_to_elasticsearch is treated as True as ansible/group_vars/all.yml (about line 748) has:

elasticsearch_address: "{{ kolla_internal_fqdn }}"

Clearly a name is being compared to an IP. It appears the comparison should be:

    elasticsearch_address != kolla_internal_fqdn

... that, or elasticsearch_address is not using the correct default?

[Mark Goddard (mgoddard)]

Looks like a good bug with an easy solution, as proposed - compare with the FQDN. Can you propose a fix to gerrit?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/791321

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/791321
Committed: https://opendev.org/openstack/kolla-ansible/commit/283b0dec639c032945f5d1d87a66f0c7541c0562
Submitter: "Zuul (22348)"
Branch: master

commit 283b0dec639c032945f5d1d87a66f0c7541c0562
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804334

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804357

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/804337

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804334
Committed: https://opendev.org/openstack/kolla-ansible/commit/25ee0b693a89f44a09df1b5c522c553b1f8717fb
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 25ee0b693a89f44a09df1b5c522c553b1f8717fb
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804357
Committed: https://opendev.org/openstack/kolla-ansible/commit/085a0852c9f70c4149eccd68dc8b8e89be25850b
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 085a0852c9f70c4149eccd68dc8b8e89be25850b
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac
    (cherry picked from commit 283b0dec639c032945f5d1d87a66f0c7541c0562)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/804337
Committed: https://opendev.org/openstack/kolla-ansible/commit/c65e1035b0406807d22f944f534138733a07c451
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit c65e1035b0406807d22f944f534138733a07c451
Author: Scott Shambarger <email address hidden>
Date: Thu May 13 14:08:32 2021 -0700

    fluentd: Fix check for external elasticsearch

    When enable_elasticsearch is False, and kolla_internal_fqdn is set
    and elasticsearch_address is the default, fluentd is still enabling
    output to elasticsearch.

    elasticsearch_address defaults to kolla_internal_fqdn, but the
    check to enable logging to an "external" elasticsearch host
    compares with kolla_internal_vip_address (fqdn != vip address).

    This patch fixes the comparison to use the kolla_internal_fqdn.

    Closes-Bug: #1927880
    Change-Id: If23a6b1273c2639d1296becc9d222546d52f63ac
    (cherry picked from commit 283b0dec639c032945f5d1d87a66f0c7541c0562)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 12.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 13.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 10.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 11.2.0 release.