Enable monasca broken fluentd container

Bug #1830147 reported by Chiawei Xie on 2019-05-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Medium
Mark Goddard
Rocky
Medium
Mark Goddard
Stein
Medium
Mark Goddard

Bug Description

kolla-ansible version: stein
kolla version: stein
source: ubuntu source

When enable monasca, the fluentd container got error messages:

[warn]: 'utc' is deperecated for output plugin. This parameter is used for formatter plugin in compatibility layer. If you want to use same feature, use timekey_use_utc parameter in <buffer> directive instead

[error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown output plugin 'monasca'. Run 'gem search -rd fluent-plugin' to find plugins"

Mark Goddard (mgoddard) wrote :

I cracked open the kolla/ubuntu-source-fluentd:stein image, and found the following gem installed via 'gem list':

fluentd-monasca-output (0.0.1)

I see the same output in the kolla/centos-source-fluentd:stein image. However, the CentOS image also includes a bunch of other fluentd plugins that don't appear in the ubuntu image:

fluent-plugin-elasticsearch (1.9.5.1)
fluent-plugin-grok-parser (0.3.1)
fluent-plugin-kubernetes_metadata_filter (0.27.0)
fluent-plugin-rewrite-tag-filter (1.5.6)
fluent-plugin-secure-forward (0.4.5)

The main difference is that the ubuntu plugins are installed via apt, whereas the centos plugins are installed via gem.

I've now found a command called td-agent-gem in the ubuntu image (fluent is referred to as td-agent in ubuntu for legacy reasons). This lists the expected plugins, other than the monasca one:

fluent-config-regexp-type (1.0.0)
fluent-logger (0.8.0)
fluent-plugin-elasticsearch (3.5.1)
fluent-plugin-grep (0.3.4)
fluent-plugin-grok-parser (2.1.4)
fluent-plugin-kafka (0.9.4)
fluent-plugin-kubernetes_metadata_filter (2.1.6)
fluent-plugin-parser (0.6.1)
fluent-plugin-prometheus (1.4.0)
fluent-plugin-record-modifier (2.0.1)
fluent-plugin-rewrite-tag-filter (2.2.0, 2.0.0)
fluent-plugin-s3 (1.1.10)
fluent-plugin-secure-forward (0.4.5)
fluent-plugin-td (1.0.0)
fluent-plugin-td-monitoring (0.2.4)
fluent-plugin-webhdfs (1.2.3)

When installing the monasca plugin, we currently use 'fluentd-gem install' on ubuntu, we probably need to run 'td-agent-gem install'.

Fix proposed to branch: master
Review: https://review.opendev.org/660953

Changed in kolla:
assignee: nobody → Mark Goddard (mgoddard)
status: New → In Progress
Mark Goddard (mgoddard) on 2019-05-23
Changed in kolla:
importance: Undecided → Medium
status: In Progress → Triaged
milestone: none → train-1
Changed in kolla:
status: Triaged → In Progress

Reviewed: https://review.opendev.org/660953
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c585ca5055c7a92fdbe5826030d3529e5c397540
Submitter: Zuul
Branch: master

commit c585ca5055c7a92fdbe5826030d3529e5c397540
Author: Mark Goddard <email address hidden>
Date: Thu May 23 10:25:28 2019 +0100

    Use td-agent-gem to install fluentd monasca plugin on ubuntu

    If monasca is enabled on ubuntu, we see the following error in fluentd:

    [error]: config error file="/etc/td-agent/td-agent.conf"
    error_class=Fluent::ConfigError error="Unknown output plugin 'monasca'.
    Run 'gem search -rd fluent-plugin' to find plugins"

    This is because we use 'fluent-gem install' to install the gem, which
    works on CentOS but on Ubuntu it is necessary to use 'td-agent-gem
    install'.

    Change-Id: I515f9764ca93a41a871e520696193a9298856d08
    Closes-Bug: #1830147

Changed in kolla:
status: In Progress → Fix Released

Reviewed: https://review.opendev.org/666075
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=40104a393e11cee0f2d37a6b2eea0cbbd388e71b
Submitter: Zuul
Branch: stable/rocky

commit 40104a393e11cee0f2d37a6b2eea0cbbd388e71b
Author: Mark Goddard <email address hidden>
Date: Thu May 23 10:25:28 2019 +0100

    Use td-agent-gem to install fluentd monasca plugin on ubuntu

    If monasca is enabled on ubuntu, we see the following error in fluentd:

    [error]: config error file="/etc/td-agent/td-agent.conf"
    error_class=Fluent::ConfigError error="Unknown output plugin 'monasca'.
    Run 'gem search -rd fluent-plugin' to find plugins"

    This is because we use 'fluent-gem install' to install the gem, which
    works on CentOS but on Ubuntu it is necessary to use 'td-agent-gem
    install'.

    Change-Id: I515f9764ca93a41a871e520696193a9298856d08
    Closes-Bug: #1830147
    (cherry picked from commit c585ca5055c7a92fdbe5826030d3529e5c397540)

Reviewed: https://review.opendev.org/666074
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=588c1076c3da583e9f654dcfbed5fde13788dc0d
Submitter: Zuul
Branch: stable/stein

commit 588c1076c3da583e9f654dcfbed5fde13788dc0d
Author: Mark Goddard <email address hidden>
Date: Thu May 23 10:25:28 2019 +0100

    Use td-agent-gem to install fluentd monasca plugin on ubuntu

    If monasca is enabled on ubuntu, we see the following error in fluentd:

    [error]: config error file="/etc/td-agent/td-agent.conf"
    error_class=Fluent::ConfigError error="Unknown output plugin 'monasca'.
    Run 'gem search -rd fluent-plugin' to find plugins"

    This is because we use 'fluent-gem install' to install the gem, which
    works on CentOS but on Ubuntu it is necessary to use 'td-agent-gem
    install'.

    Change-Id: I515f9764ca93a41a871e520696193a9298856d08
    Closes-Bug: #1830147
    (cherry picked from commit c585ca5055c7a92fdbe5826030d3529e5c397540)

This issue was fixed in the openstack/kolla 8.0.0.0rc2 release candidate.

Mark Goddard (mgoddard) on 2019-08-07
Changed in kolla:
milestone: train-1 → 9.0.0
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers